Repository navigation
feat: e2e tests #1501
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
MrMuzyk
wants to merge
8
commits into
callstack:main
Choose a base branch
from
MrMuzyk:e2e-ci
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+1,001
−169
Open
feat: e2e tests #1501
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
6ff87a3
ci: temp e2e workflow
MrMuzyk c5fa84d
Merge branch 'callstack:main' into main
MrMuzyk 4feac60
feat: repack e2e tests
MrMuzyk e0a0ea5
Merge branch 'main' into e2e-ci
MrMuzyk 0254419
fix: CR fixes
MrMuzyk da3f15e
fix: pin ruby
MrMuzyk 53bca6d
fix: warn
MrMuzyk 62fd861
fix: cr fixes v2
MrMuzyk File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,237 @@ | ||
| name: e2e | ||
|
|
||
| on: | ||
| pull_request: | ||
| paths: | ||
| - 'apps/tester-federation-v2/**' | ||
| - 'packages/**' | ||
| - 'pnpm-lock.yaml' | ||
| - '.github/workflows/e2e.yml' | ||
| push: | ||
| branches: [main] | ||
| paths: | ||
| - 'apps/tester-federation-v2/**' | ||
| - 'packages/**' | ||
| - 'pnpm-lock.yaml' | ||
| - '.github/workflows/e2e.yml' | ||
| workflow_dispatch: | ||
|
|
||
| # one run per PR; a new push cancels the older one | ||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }} | ||
| cancel-in-progress: ${{ github.event_name != 'push' }} | ||
|
|
||
| # keep it read-only | ||
| permissions: | ||
| contents: read | ||
|
|
||
| env: | ||
| E2E_TELEMETRY_DISABLED: '1' | ||
|
|
||
| jobs: | ||
| android: | ||
| name: e2e (Android) | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 45 | ||
|
|
||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # tag=v4.2.2 | ||
|
|
||
| - name: Setup workspace | ||
| uses: ./.github/actions/setup-pnpm-node | ||
| with: | ||
| node-version: '24.18.0' | ||
|
|
||
| - name: Setup Java | ||
| uses: actions/setup-java@de7274f081f381c8f8158605e0321c36c376e2e6 # tag=v6.0.1 | ||
| with: | ||
| distribution: temurin | ||
| java-version: '17' | ||
|
|
||
| - name: Enable KVM | ||
| run: | | ||
| echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' | sudo tee /etc/udev/rules.d/99-kvm4all.rules | ||
| sudo udevadm control --reload-rules | ||
| sudo udevadm trigger --name-match=kvm | ||
|
|
||
| - name: Build workspace packages | ||
| run: pnpm --filter tester-federation-v2... build | ||
|
|
||
| # nohup/disown: the runner kills step processes at the end. /status wait: RNTA picks its bundle source once, never retries | ||
| - name: Start dev servers | ||
| working-directory: apps/tester-federation-v2 | ||
| run: | | ||
| nohup pnpm start:hostapp > /tmp/hostapp.log 2>&1 & | ||
| disown | ||
| nohup pnpm start:miniapp > /tmp/miniapp.log 2>&1 & | ||
| disown | ||
| timeout 180 sh -c 'until [ "$(curl -sf localhost:8081/status)" = "packager-status:running" ] && [ "$(curl -sf localhost:8082/status)" = "packager-status:running" ]; do sleep 2; done' | ||
|
|
||
| # ask for the bundles now so the servers compile them — they only build on first request | ||
| - name: Warm both compilers | ||
| working-directory: apps/tester-federation-v2 | ||
| run: | | ||
| curl -sf --max-time 300 -o /dev/null 'http://localhost:8081/android/index.bundle?platform=android&dev=true&lazy=true&minify=false&app=com.tester.federationV2&modulesOnly=false&runModule=true' | ||
| curl -sf --max-time 300 -o /dev/null 'http://localhost:8082/android/index.bundle?platform=android&dev=true&lazy=true&minify=false&app=com.tester.federationV2&modulesOnly=false&runModule=true' | ||
| echo 'Both bundles served; compilers are warm.' | ||
|
|
||
| - name: Check shared versions are real semver | ||
| run: | | ||
| for p in 8081 8082; do | ||
| M=$(curl -sf --retry 3 --retry-delay 2 "http://localhost:$p/android/mf-manifest.json") || { echo "::error::port $p: mf-manifest.json not served"; exit 1; } | ||
| [ "$(jq '.shared | length' <<<"$M")" -gt 0 ] || { echo "::error::port $p: manifest has no shared entries"; exit 1; } | ||
| BAD=$(jq -r '.shared[] | select((.version // "") | test("^[0-9]+\\.[0-9]+\\.[0-9]+") | not) | "\(.name): \(.version)"' <<<"$M") | ||
| [ -z "$BAD" ] || { echo "::error::port $p has non-semver shared versions: $BAD"; exit 1; } | ||
| done | ||
|
|
||
| - name: Build Android app | ||
| working-directory: apps/tester-federation-v2/android | ||
| run: ./gradlew :app:assembleDebug -PreactNativeArchitectures=x86_64 | ||
|
|
||
| # expose dev servers to the emulator, install the APK, run the tests | ||
| - name: Run e2e on an emulator | ||
| uses: reactivecircus/android-emulator-runner@a421e43855164a8197daf9d8d40fe71c6996bb0d # tag=v2.38.0 | ||
| with: | ||
| api-level: 35 | ||
| target: google_apis | ||
| arch: x86_64 | ||
| profile: pixel_7 | ||
| avd-name: e2e | ||
| working-directory: apps/tester-federation-v2 | ||
| script: | | ||
| adb shell settings put global hide_error_dialogs 1 | ||
| for i in 1 2 3; do adb shell "uiautomator dump /sdcard/d.xml >/dev/null && grep -q aerr_ /sdcard/d.xml" && { echo 'ANR dialog on screen, clearing it'; adb shell am force-stop com.google.android.apps.nexuslauncher; }; sleep 2; done | ||
| adb reverse tcp:8081 tcp:8081 | ||
| adb reverse tcp:8082 tcp:8082 | ||
| adb reverse --list | ||
| adb reverse --list | grep -q 'tcp:8081' || { echo '::error::adb reverse missing for tcp:8081'; exit 1; } | ||
| adb reverse --list | grep -q 'tcp:8082' || { echo '::error::adb reverse missing for tcp:8082'; exit 1; } | ||
| adb install -r -g android/app/build/outputs/apk/debug/app-debug.apk || { echo '::error::adb install failed'; exit 1; } | ||
| E2E_DEVICE="${ANDROID_SERIAL:-$(adb get-serialno)}" pnpm exec e2e run --target android --reporter list,junit; ec=$?; adb logcat -d -v time > .e2e/logcat.txt 2>&1; exit $ec | ||
|
|
||
| - name: Collect dev server logs | ||
| if: ${{ !cancelled() }} | ||
| working-directory: apps/tester-federation-v2 | ||
| run: | | ||
| mkdir -p .e2e | ||
| cp /tmp/hostapp.log /tmp/miniapp.log .e2e/ | ||
|
|
||
| - name: Upload e2e artifacts | ||
| if: ${{ !cancelled() }} | ||
| uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # tag=v7.0.1 | ||
| with: | ||
| name: e2e-android | ||
| path: apps/tester-federation-v2/.e2e | ||
| include-hidden-files: true | ||
| if-no-files-found: warn | ||
| retention-days: 7 | ||
|
|
||
| - name: Show dev server logs | ||
| if: ${{ !cancelled() }} | ||
| run: tail -n 80 /tmp/hostapp.log /tmp/miniapp.log | ||
|
|
||
| # dispatch + main only for now — no PR triggers until this lane has greens | ||
| ios: | ||
| name: e2e (iOS) | ||
| if: github.event_name == 'push' || github.event_name == 'workflow_dispatch' | ||
| runs-on: macos-15 | ||
| timeout-minutes: 45 | ||
|
|
||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # tag=v4.2.2 | ||
|
|
||
| - name: Setup Ruby | ||
| uses: ruby/setup-ruby@651ff0d62c5c79bed6d53ec518387aae20e72c1a # tag=v1.329.0 | ||
| with: | ||
| ruby-version: '3.4' | ||
| bundler-cache: true | ||
| working-directory: apps/tester-federation-v2 | ||
|
|
||
| - name: Setup workspace | ||
| uses: ./.github/actions/setup-pnpm-node | ||
| with: | ||
| node-version: '24.18.0' | ||
|
|
||
| - name: Build workspace packages | ||
| run: pnpm --filter tester-federation-v2... build | ||
|
|
||
| # start both dev servers in the background, wait until both answer /status | ||
| - name: Start dev servers | ||
| working-directory: apps/tester-federation-v2 | ||
| run: | | ||
| nohup pnpm start:hostapp > /tmp/hostapp.log 2>&1 & | ||
| disown | ||
| nohup pnpm start:miniapp > /tmp/miniapp.log 2>&1 & | ||
| disown | ||
| for p in 8081 8082; do curl -sf --retry 90 --retry-delay 2 --retry-all-errors --retry-connrefused -o /dev/null "http://localhost:$p/status"; done | ||
|
|
||
| # ask for the iOS bundles now so the servers compile them — they only build on first request | ||
| - name: Warm both compilers | ||
| working-directory: apps/tester-federation-v2 | ||
| run: | | ||
| curl -sf --max-time 300 -o /dev/null 'http://localhost:8081/index.bundle?platform=ios&dev=true&lazy=true&minify=false&modulesOnly=false&runModule=true&app=com.tester.federationV2' | ||
| curl -sf --max-time 300 -o /dev/null 'http://localhost:8082/index.bundle?platform=ios&dev=true&lazy=true&minify=false&modulesOnly=false&runModule=true&app=com.tester.federationV2' | ||
| echo 'Both bundles served; compilers are warm.' | ||
|
|
||
| - name: Check shared versions are real semver | ||
| run: | | ||
| for p in 8081 8082; do | ||
| M=$(curl -sf --retry 3 --retry-delay 2 "http://localhost:$p/ios/mf-manifest.json") || { echo "::error::port $p: mf-manifest.json not served"; exit 1; } | ||
| [ "$(jq '.shared | length' <<<"$M")" -gt 0 ] || { echo "::error::port $p: manifest has no shared entries"; exit 1; } | ||
| BAD=$(jq -r '.shared[] | select((.version // "") | test("^[0-9]+\\.[0-9]+\\.[0-9]+") | not) | "\(.name): \(.version)"' <<<"$M") | ||
| [ -z "$BAD" ] || { echo "::error::port $p has non-semver shared versions: $BAD"; exit 1; } | ||
| done | ||
|
|
||
| # SIM_RUNTIME_PREFIX is tied to the runner image: macos-15's default Xcode | ||
| # (16.4) targets iOS 18.x. If the image moves, this check fails every run — | ||
| # bump the prefix (and SIM_DEVICE) to the new SDK's major. | ||
| - name: Boot simulator | ||
| env: | ||
| SIM_RUNTIME_PREFIX: com.apple.CoreSimulator.SimRuntime.iOS-18- | ||
| SIM_DEVICE: iPhone 16 | ||
| run: | | ||
| RT=$(xcrun simctl list runtimes -j | jq -r --arg prefix "$SIM_RUNTIME_PREFIX" '.runtimes | map(select(.isAvailable and (.identifier | startswith($prefix)))) | sort_by(.version) | last | .identifier') | ||
| echo "using $RT" | ||
| [ "$RT" != "null" ] && [ -n "$RT" ] | ||
| xcrun simctl create e2e "$SIM_DEVICE" "$RT" | ||
| xcrun simctl boot e2e | ||
| xcrun simctl bootstatus e2e -b | ||
|
|
||
| # pod install generates the Xcode project the workspace points at | ||
| - name: Build iOS app | ||
| working-directory: apps/tester-federation-v2/ios | ||
| run: bundle install && bundle exec pod install && xcodebuild -workspace ModuleFederationTesterV2.xcworkspace -scheme ReactTestApp -configuration Debug -destination 'generic/platform=iOS Simulator' -derivedDataPath build CODE_SIGNING_ALLOWED=NO ARCHS=arm64 build | ||
|
|
||
| - name: Run e2e | ||
| working-directory: apps/tester-federation-v2 | ||
| env: | ||
| E2E_DEVICE: e2e | ||
| run: | | ||
| xcrun simctl install booted ios/build/Build/Products/Debug-iphonesimulator/*.app | ||
| pnpm exec e2e run --target ios --reporter list,junit | ||
|
|
||
| - name: Collect dev server logs | ||
| if: ${{ !cancelled() }} | ||
| working-directory: apps/tester-federation-v2 | ||
| run: | | ||
| mkdir -p .e2e | ||
| cp /tmp/hostapp.log /tmp/miniapp.log .e2e/ | ||
| xcrun simctl spawn booted log show --style compact --last 30m \ | ||
| --predicate 'process == "ReactTestApp"' \ | ||
| > .e2e/ios-syslog.txt 2>&1 || true | ||
|
|
||
| - name: Upload e2e artifacts | ||
| if: ${{ !cancelled() }} | ||
| uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # tag=v7.0.1 | ||
| with: | ||
| name: e2e-ios | ||
| path: apps/tester-federation-v2/.e2e | ||
| include-hidden-files: true | ||
| if-no-files-found: warn | ||
| retention-days: 7 | ||
|
|
||
| - name: Show dev server logs | ||
| if: ${{ !cancelled() }} | ||
| run: tail -n 80 /tmp/hostapp.log /tmp/miniapp.log | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -62,3 +62,6 @@ yarn-error.log | |
|
|
||
| # MF | ||
| **/@mf-types | ||
|
|
||
| # e2e | ||
| .e2e/ | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| import nativePkg from '@react-navigation/native/package.json' with { | ||
| type: 'json', | ||
| }; | ||
| import nativeStackPkg from '@react-navigation/native-stack/package.json' with { | ||
| type: 'json', | ||
| }; | ||
| import reactPkg from 'react/package.json' with { type: 'json' }; | ||
| import reactNativePkg from 'react-native/package.json' with { type: 'json' }; | ||
| import safeAreaPkg from 'react-native-safe-area-context/package.json' with { | ||
| type: 'json', | ||
| }; | ||
| import screensPkg from 'react-native-screens/package.json' with { | ||
| type: 'json', | ||
| }; | ||
|
|
||
| const SHARED_SINGLETONS: Array<[name: string, version: string]> = [ | ||
| ['react', reactPkg.version], | ||
| ['react-native', reactNativePkg.version], | ||
| ['@react-navigation/native', nativePkg.version], | ||
| ['@react-navigation/native-stack', nativeStackPkg.version], | ||
| ['react-native-safe-area-context', safeAreaPkg.version], | ||
| ['react-native-screens', screensPkg.version], | ||
| ]; | ||
|
|
||
| export function buildSharedSingletons(eager: boolean) { | ||
| return Object.fromEntries( | ||
| SHARED_SINGLETONS.map(([name, version]) => [ | ||
| name, | ||
| { singleton: true, eager, version, requiredVersion: version }, | ||
| ]) | ||
| ); | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The iOS lane can't catch the bug this PR exists to fix. Same app, same device, only the configs differing:
versionmain(pkg.dependencies[...])catalog:testers5.8.0/4.26.2iOS passes with the bug fully active. It isn't silently absent — the host iOS bundle I served contains
version: "catalog:testers"/requiredVersion: "catalog:testers"for bothreact-native-safe-area-contextandreact-native-screens, and the Gallery test still went green.Because with
singleton: trueMF2 only warns and carries on (@module-federation/runtime-coredist/utils/share.js:189-193—strictVersionis false becausesingletonis true), andsatisfy("catalog:testers","catalog:testers")isfalseeven against itself. The module loads fine. On Android the visible symptom is the warning banner covering the Mini Apps row, which eats the tap — the trace showstap group "🖼️, Gallery"→screen: unchanged since step 3→ 327 failed reads ofminiapp-gallery. iOS has no banner, so the tap lands and the test passes.So the description's "the Gallery test passing with zero suppression is the regression test for this" holds on Android only. Making it platform-agnostic, ~5s:
Verified: against
main's configs it prints exactly the two offenders; against this PR's configs it's silent and exits 0. Both ports serve a manifest, so the loop covers host and remote.Related, same root cause: iOS collects no device logs — Android does
adb logcat -d > .e2e/logcat.txt(line 102), this job copies only the two dev-server logs. A JS console warning is the only symptom on iOS, so an iOS-only federation regression gives a green run and nothing to debug from even when it does fail. Worth asimctl spawn booted log showin the collect step.