chore(e2e): add e2e tests on browserstack via Maestro#56
Conversation
still need to wait and poll test run statuses
Browserstack's Maestro support does not seem to support this command (https://www.browserstack.com/docs/app-automate/maestro/references/supported-commands)
gmaclennan
left a comment
There was a problem hiding this comment.
Just some minor bugs and nits. I'll push changes for these to re-run CI and check it all works.
| key: nodejs-mobile-${{ env.NODEJS_MOBILE_VERSION }}-ios-${{ hashFiles('scripts/download-nodejs-mobile.sh') }} | ||
|
|
||
| - name: Download nodejs-mobile binaries | ||
| if: steps.cache-libnode.outputs.cache-hit != 'true' |
| # TODO: Enable when we know this job works | ||
| # needs: [build-android, build-ios] |
There was a problem hiding this comment.
I think we need to re-enable this now?
There was a problem hiding this comment.
Should we create a follow-up to address this? Mostly hygenic to prevent uploading test suites to browserstack when the jobs for building + uploading the apps fail
- Set jasmine.DEFAULT_TIMEOUT_INTERVAL to 60s — the 5s default was tripping the CRUD tests that fan out 100 IPC create()/delete() calls on slower BrowserStack devices. - Track every opened project in project-crud and close them in afterEach so listeners don't accumulate across tests (was producing MaxListenersExceededWarning on SocketMessagePort / LocalPeers and slowing later tests). - Add an `all-tests-done` testID alongside `all-tests-passed` / `all-tests-failed` so Maestro can fail fast on a failing run instead of waiting for the full timeout. - Bump Maestro `extendedWaitUntil` to 5 min and assert pass after done. - Log spec start / pass / fail (with stacks) via console.log so they appear under the ReactNativeJS tag in logcat / device logs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The root package.json `prepare` script (`expo-module prepare`, plus `scripts/check-sentry-cocoa-pin.mjs`) needs to run before the e2e app prebuilds, otherwise the module's native scaffolding isn't regenerated and the iOS Pods build fails to resolve `SentrySDK.startTransaction` through the `Sentry/HybridSDK` module. The `iOS Tests` workflow that builds `apps/example` runs `npm install` (no `--ignore-scripts`) and passes — match that. `patch-package` is already wired up as `postinstall` in both package.jsons, so the explicit `npx patch-package` calls are redundant once scripts run. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Seems like the older sodium-native (libsodium) was crashing Android v8.1
|
Warning Review the following alerts detected in dependencies. According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.
|
…or Xcode 26 compat" This reverts commit 8012789.
The Oppo Reno 6-11.0 is no longer supported by Browserstack
…ules npm list --parseable --long appends flag fields (:OVERRIDDEN, :INVALID, ...) after name@version, so taking the last :-field read "OVERRIDDEN" for the override-pinned sodium-native@5.1.0. The module was silently dropped from prebuild download + jniLibs packaging while the bundled JS still referenced it, crashing every device with ERR_DLOPEN_FAILED: libsodium-native__5.1.0.so not found. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Xcode 26's Swift compiler hides SentrySDK.startTransaction (used by SentryNativeBridge) when sentry-cocoa builds from source without BUILD_LIBRARY_FOR_DISTRIBUTION (getsentry/sentry-cocoa#7950). Set it on Sentry pod targets from the library config plugin so every consumer's prebuild gets the workaround, injected inside the existing post_install block (CocoaPods allows only one post_install hook — the previous attempt added a second block, which fails pod install). Register the plugin in the e2e app, which didn't apply it before. Verified locally with Xcode 26.3: the exact CI archive command now succeeds, and the Release app boots on the iOS simulator. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The backend build reads the Node ABI from android/libnode headers on every platform. The iOS job only created them on a cache miss (the download script fetches both platforms); a cache hit restored just ios/NodeMobile.xcframework, so any rerun failed with ENOENT on node_version.h. Cache both paths and bump the key so ios-only caches can't hit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
gmaclennan
left a comment
There was a problem hiding this comment.
@achou11 this is now passing on CI - the failed tests on Browserstack revealed a bug / incompatibility with older Android version (<9) which is now fixed (needed to rebuild the native deps). I think this is good to merge, it's already proven its worth.
* origin/main: chore(e2e): add e2e tests on browserstack via Maestro (#56) fix(sentry): make exit telemetry lossless and stop cross-process clobbering (#84) feat(sentry): land Phases 6 + 7a — Android exit reasons & iOS MetricKit app-exit telemetry (#72) chore(build): use npm list instead of custom traversal to get native module versions (#70)
## Optic Release Automation This **draft** PR is opened by Github action [optic-release-automation-action](https://github.com/nearform-actions/optic-release-automation-action). A new **draft** GitHub release [v1.0.0-pre.2](https://github.com/digidem/comapeo-core-react-native/releases/tag/untagged-c499977757c9745e56b2) has been created. Release author: @gmaclennan #### If you want to go ahead with the release, please merge this PR. When you merge: - The GitHub release will be published - The npm package with tag pre will be published according to the publishing rules you have configured - No major or minor tags will be updated as configured #### If you close the PR - The new draft release will be deleted and nothing will change ## What's Changed * Android Testing Infrastructure & Bug Fixes by @gmaclennan in #3 * chore: prebuild example/android; harden instrumented tests by @gmaclennan in #10 * Integrate @comapeo/core via IPC over Unix sockets by @gmaclennan in #5 * chore: adjust repo setup by @achou11 in #12 * chore: minor fixes based on expo-doctor by @achou11 in #13 * Add iOS support & test infrastructure by @gmaclennan in #6 * chore: add architecture docs & plans by @gmaclennan in #11 * update some native deps used in backend by @achou11 in #14 * iOS Phase 1: unified JS bundle + smoke test (simulator-only) by @gmaclennan in #15 * iOS Phase 2: xcframework Embed & Sign for native addons by @gmaclennan in #16 * Phase 2 Android: jniLibs packaging + unified rollup loader plugin by @gmaclennan in #17 * chore: post-Phase-2 cleanup — comments, plan docs, agents.md by @gmaclennan in #33 * android: read abiFilters from reactNativeArchitectures (#30) by @gmaclennan in #35 * refactor: simplify build-backend.ts; rollup writes directly to native asset trees by @gmaclennan in #34 * chore: fix eslint configuration by @achou11 in #41 * android: audit 16 KB page alignment on every shipped .so by @gmaclennan in #43 * Add rootkey persistence and lifecycle state management by @gmaclennan in #36 * chore: move example app into apps directory by @achou11 in #18 * refactor: per-component lifecycle state with derived ComapeoState by @gmaclennan in #47 * android: fold waitForFile into connect retry loop by @gmaclennan in #52 * chore: add e2e testing app by @achou11 in #49 * fix(android): drop setUnlockedDeviceRequired from rootkey wrapper key by @gmaclennan in #57 * fix(backend): cache stopping/error frames for late joiners by @gmaclennan in #58 * fix(ios-tests): wait for STOPPING before signalling node exit by @gmaclennan in #59 * fix(android): drain JNI stdio pumps before returning from node::Start by @gmaclennan in #60 * Sentry integration: Phase 1 + Phase 2a + Phase 2b by @gmaclennan in #54 * feat(backend): polywasm-backed undici on iOS, re-enable maps plugin by @gmaclennan in #62 * ci: drop unreliable Android emulator snapshot caching by @gmaclennan in #64 * feat(sentry): land Phase 3 — backend loader + RPC tracing by @gmaclennan in #63 * fix(ios-tests): serialise STOPPING/STOPPED observers in testFullLifecycleStateTransitions by @gmaclennan in #71 * use npm list instead of custom traversal to get native module versions by @achou11 in #70 * feat(sentry): land Phases 6 + 7a — Android exit reasons & iOS MetricKit app-exit telemetry by @gmaclennan in #72 * fix(sentry): make exit telemetry lossless and stop cross-process clobbering by @gmaclennan in #84 * chore(e2e): add e2e tests on browserstack via Maestro by @achou11 in #56 * feat(sentry): migrate to @sentry/react-native v8; exit telemetry as Application Metrics by @gmaclennan in #73 * Map server integration by @gmaclennan in #86 * chore(deps): upgrade to Expo SDK 56 (React Native 0.85) by @gmaclennan in #87 * chore(ci): add release workflow by @gmaclennan in #90 * chore: fix npm script and release build script by @gmaclennan in #91 * chore(pack): don't try to package build files by @gmaclennan in #92 * fix: start fastify listening by @gmaclennan in #93 * perf(backend): switch bundler from rollup to rolldown by @gmaclennan in #94 * fix(ci): ignore-scripts in ios npm installs by @gmaclennan in #96 * fix(ci): replace --ignore-scripts with npm strict-allow-scripts allowlist by @gmaclennan in #106 * feat(config): let the consuming app supply the default project config by @gmaclennan in #95 * chore(release): merge prerelease branch. by @gmaclennan in #110 ## New Contributors * @achou11 made their first contribution in #12 **Full Changelog**: https://github.com/digidem/comapeo-core-react-native/commits/v1.0.0-pre.2 <!-- <release-meta>{"id":342868678,"version":"v1.0.0-pre.2","npmTag":"pre","opticUrl":"https://optic-zf3votdk5a-ew.a.run.app/api/generate/"}</release-meta> -->
## Optic Release Automation This **draft** PR is opened by Github action [optic-release-automation-action](https://github.com/nearform-actions/optic-release-automation-action). A new **draft** GitHub release [v1.0.0-pre.2](https://github.com/digidem/comapeo-core-react-native/releases/tag/untagged-352a6c41c12fd02dec37) has been created. Release author: @gmaclennan #### If you want to go ahead with the release, please merge this PR. When you merge: - The GitHub release will be published - The npm package with tag pre will be published according to the publishing rules you have configured - No major or minor tags will be updated as configured #### If you close the PR - The new draft release will be deleted and nothing will change <!-- Release notes generated using configuration in .github/release.yml at 7fe80b4 --> ## What's Changed ### 🚀 Features * Integrate @comapeo/core via IPC over Unix sockets by @gmaclennan in #5 * Add iOS support & test infrastructure by @gmaclennan in #6 * iOS Phase 1: unified JS bundle + smoke test (simulator-only) by @gmaclennan in #15 * iOS Phase 2: xcframework Embed & Sign for native addons by @gmaclennan in #16 * Phase 2 Android: jniLibs packaging + unified rollup loader plugin by @gmaclennan in #17 * android: read abiFilters from reactNativeArchitectures (#30) by @gmaclennan in #35 * Add rootkey persistence and lifecycle state management by @gmaclennan in #36 * Sentry integration: Phase 1 + Phase 2a + Phase 2b by @gmaclennan in #54 * feat(backend): polywasm-backed undici on iOS, re-enable maps plugin by @gmaclennan in #62 * feat(sentry): land Phase 3 — backend loader + RPC tracing by @gmaclennan in #63 * feat(sentry): land Phases 6 + 7a — Android exit reasons & iOS MetricKit app-exit telemetry by @gmaclennan in #72 * feat(sentry): migrate to @sentry/react-native v8; exit telemetry as Application Metrics by @gmaclennan in #73 * Map server integration by @gmaclennan in #86 * feat(config): let the consuming app supply the default project config by @gmaclennan in #95 ### 🐛 Bug Fixes * fix(android): drop setUnlockedDeviceRequired from rootkey wrapper key by @gmaclennan in #57 * fix(backend): cache stopping/error frames for late joiners by @gmaclennan in #58 * fix(ios-tests): wait for STOPPING before signalling node exit by @gmaclennan in #59 * fix(android): drain JNI stdio pumps before returning from node::Start by @gmaclennan in #60 * fix(ios-tests): serialise STOPPING/STOPPED observers in testFullLifecycleStateTransitions by @gmaclennan in #71 * fix(sentry): make exit telemetry lossless and stop cross-process clobbering by @gmaclennan in #84 * fix: start fastify listening by @gmaclennan in #93 * fix(ci): ignore-scripts in ios npm installs by @gmaclennan in #96 * fix(ci): replace --ignore-scripts with npm strict-allow-scripts allowlist by @gmaclennan in #106 * fix(release): stop `npm pack --dry-run` leaking dry-run into backend install by @gmaclennan in #129 ### ⚡ Performance * perf(backend): switch bundler from rollup to rolldown by @gmaclennan in #94 ### ⬆️ Dependencies * update some native deps used in backend by @achou11 in #14 * chore(deps): upgrade to Expo SDK 56 (React Native 0.85) by @gmaclennan in #87 ### 🏗️ Maintenance * Android Testing Infrastructure & Bug Fixes by @gmaclennan in #3 * chore: prebuild example/android; harden instrumented tests by @gmaclennan in #10 * chore: adjust repo setup by @achou11 in #12 * chore: minor fixes based on expo-doctor by @achou11 in #13 * chore: add architecture docs & plans by @gmaclennan in #11 * chore: post-Phase-2 cleanup — comments, plan docs, agents.md by @gmaclennan in #33 * refactor: simplify build-backend.ts; rollup writes directly to native asset trees by @gmaclennan in #34 * chore: fix eslint configuration by @achou11 in #41 * android: audit 16 KB page alignment on every shipped .so by @gmaclennan in #43 * chore: move example app into apps directory by @achou11 in #18 * refactor: per-component lifecycle state with derived ComapeoState by @gmaclennan in #47 * android: fold waitForFile into connect retry loop by @gmaclennan in #52 * chore: add e2e testing app by @achou11 in #49 * ci: drop unreliable Android emulator snapshot caching by @gmaclennan in #64 * use npm list instead of custom traversal to get native module versions by @achou11 in #70 * chore(e2e): add e2e tests on browserstack via Maestro by @achou11 in #56 * chore(ci): add release workflow by @gmaclennan in #90 * chore: fix npm script and release build script by @gmaclennan in #91 * chore(pack): don't try to package build files by @gmaclennan in #92 * chore(release): merge prerelease branch. by @gmaclennan in #110 * ci(e2e): retry BrowserStack builds on infra-class flakes by @gmaclennan in #113 ### Other Changes * ci: derive changelog labels from PR titles + add Dependabot by @gmaclennan in #114 ## New Contributors * @achou11 made their first contribution in #12 * @optic-release-automation[bot] made their first contribution in #112 **Full Changelog**: https://github.com/digidem/comapeo-core-react-native/commits/v1.0.0-pre.2 <!-- <release-meta>{"id":342970724,"version":"v1.0.0-pre.2","npmTag":"pre","opticUrl":"https://optic-zf3votdk5a-ew.a.run.app/api/generate/"}</release-meta> -->
Closes #61
Introduces e2e test automation using Maestro, configured to run as part of CI.
Current workflow diagram (will update as needed):
Relevant links:
TODO: