From 8087a59ec4c38fcc506fe0ede38413ca55bb0711 Mon Sep 17 00:00:00 2001 From: Nan Date: Mon, 15 Jun 2026 11:52:49 -0700 Subject: [PATCH] chore(appium): harden local runner with preflights and clearer docs Surface setup problems early instead of as cryptic late failures, based on issues hit running the suite on a fresh machine. run-local.sh: * fail-fast preflight for appium, the platform driver, vpx, and auto-install of node_modules when missing, each with remediation * auto-export WDIO_USE_NATIVE_FETCH=1 on Node >= 26 (webdriverio/undici fetch incompatibility; CI on Node 24 is unaffected) * fail fast on missing ONESIGNAL_APP_ID/API_KEY, directing users to the OneSignal app dedicated to Appium tests (not a general/shared app, whose live in-app campaigns can cover the UI and break tests) * graceful iOS simulator/runtime fallback (booted sim, else newest available runtime + matching device) * uninstall the app on iOS reset so the notification permission prompt reappears for local notification tests docs: * document the Vite+/vpx prerequisite and Node version expectations * note to use the dedicated Appium test app for credentials * explain CI-vs-local notification-test gating (itSkipBsIos / isBrowserStackIos) and the Enterprise signing-cert reason Co-authored-by: Cursor --- appium/README.md | 11 ++- appium/scripts/README.md | 31 ++++++-- appium/scripts/run-local.sh | 138 ++++++++++++++++++++++++++++++++---- 3 files changed, 160 insertions(+), 20 deletions(-) diff --git a/appium/README.md b/appium/README.md index 8fb4429..e10f9c6 100644 --- a/appium/README.md +++ b/appium/README.md @@ -4,7 +4,8 @@ End-to-end tests for OneSignal mobile SDKs using [Appium](https://appium.io/) an ## Prerequisites -- [Node.js](https://nodejs.org/) (or [Bun](https://bun.sh/)) +- [Node.js](https://nodejs.org/) (or [Bun](https://bun.sh/)) — CI runs Node 24. Node 26+ works locally: `scripts/run-local.sh` sets `WDIO_USE_NATIVE_FETCH=1` automatically to work around webdriverio's undici dispatcher being rejected by Node 26+'s `fetch` (`UND_ERR_INVALID_ARG`). +- [Vite+](https://vite.plus) (`curl -fsSL https://vite.plus | bash`) — provides the `vpx` command used to run WebdriverIO here (the `vpx` symlink is created on `vp`'s first run). - [Appium](https://appium.io/docs/en/latest/quickstart/install/) (`npm i -g appium`) - Appium drivers: - iOS: `appium driver install xcuitest` @@ -13,6 +14,12 @@ End-to-end tests for OneSignal mobile SDKs using [Appium](https://appium.io/) an - Xcode with iOS simulators (for iOS) - Android SDK with an AVD configured (for Android) +## CI vs Local + +CI (`.github/workflows/appium-e2e.yml`) runs these tests on BrowserStack devices with Node 24 and does **not** use `scripts/run-local.sh` — it calls `vpx wdio run "wdio..conf.ts"` directly. + +Notification-dependent tests currently only run locally: `02_push.spec.ts` and `12_activity.spec.ts` mark them with `itSkipBsIos` (`isBrowserStackIos()` from `tests/helpers/app.ts`, true when `BROWSERSTACK_USERNAME` is set and the platform is iOS). BrowserStack requires the app to be built with an Enterprise Signing Certificate for these notification flows, which we don't have yet — a temporary signing limitation, not an inherent device capability limit. The skip helper exists so these tests can be re-enabled once signing support is available. Until then, run `scripts/run-local.sh` on iOS to cover them. + ## Directory Structure ``` @@ -36,7 +43,7 @@ Quick start: ```bash cd scripts -cp .env.example .env # add your OneSignal credentials +cp .env.example .env # add the OneSignal app dedicated to Appium tests ./run-local.sh --platform=ios --sdk=flutter ``` diff --git a/appium/scripts/README.md b/appium/scripts/README.md index 18b8ae3..40802ee 100644 --- a/appium/scripts/README.md +++ b/appium/scripts/README.md @@ -18,20 +18,31 @@ cp .env.example .env ``` - At minimum, set your OneSignal credentials: + At minimum, set your OneSignal credentials (the script fails fast without them). Use the OneSignal app **dedicated to Appium tests** — not a general or shared app, whose live in-app marketing campaigns can cover the UI and cause misleading "element not displayed" failures: ``` - ONESIGNAL_APP_ID=your-app-id - ONESIGNAL_API_KEY=your-api-key + ONESIGNAL_APP_ID=your-appium-test-app-id + ONESIGNAL_API_KEY=your-appium-test-api-key ``` 3. **Install Appium and drivers** (if not already): ```bash npm i -g appium - appium driver install xcuitest + appium driver install xcuitest # iOS + appium driver install uiautomator2 # Android ``` +4. **Install [Vite+](https://vite.plus)** (if not already) — it provides the `vpx` command the script uses to run WebdriverIO (the `vpx` symlink is created on `vp`'s first run): + + ```bash + curl -fsSL https://vite.plus | bash + ``` + +The script checks all of these up front and prints the exact install command for anything missing; `node_modules` in `appium/` is installed automatically on first run. + +> **CI vs local:** CI runs on BrowserStack (Node 24) without this script. Notification-dependent tests (in `02_push.spec.ts` and `12_activity.spec.ts`) are skipped on BrowserStack iOS via `isBrowserStackIos()` because BrowserStack requires an Enterprise Signing Certificate for those notification flows, which we don't have yet (temporary — they'll be re-enabled once signing support is available), so for now they only run locally. If your local Node is 26+, the script sets `WDIO_USE_NATIVE_FETCH=1` automatically. + ## Usage ```bash @@ -118,6 +129,14 @@ All env vars can be set in `.env` or exported in your shell. See [`.env.example` ./run-local.sh --platform=ios --sdk=flutter ``` -- **Simulator not found**: Check available simulators with `xcrun simctl list devices available` and update `IOS_SIMULATOR` / `IOS_RUNTIME` in your `.env`. +- **Simulator not found**: The script falls back automatically to the booted simulator, or to the newest installed iOS runtime, when the requested device/runtime isn't on your machine. To pin a specific one, check `xcrun simctl list devices available` and set `DEVICE` / `OS_VERSION` / `IOS_RUNTIME` in your `.env`. + +- **Appium fails to start**: Make sure Appium and the required drivers are installed (`appium driver list --installed`). The script checks both up front and prints the install command for anything missing. + +- **`vpx: command not found`**: Install [Vite+](https://vite.plus) with `curl -fsSL https://vite.plus | bash`. If `vp` is installed but `vpx` is missing, run `vp --version` once — `vp` creates the `vpx` symlink on its first run. + +- **`UND_ERR_INVALID_ARG` / fetch errors on Node 26+**: webdriverio's undici dispatcher is rejected by Node 26+'s `fetch`. The script exports `WDIO_USE_NATIVE_FETCH=1` automatically when it detects Node 26+; if you invoke `vpx wdio run` manually, export it yourself. + +- **Test waiting for the notification permission alert fails**: A reused simulator remembers a previously-decided notification permission, and `simctl privacy` can't reset it. The script's app reset uninstalls the app, which restores the prompt — avoid `--skip`/`--skip-reset` when running the push specs. -- **Appium fails to start**: Make sure Appium and the required drivers are installed (`appium driver list --installed`). +- **Misleading "element not displayed" failures**: Live in-app marketing campaigns on the configured app can cover the UI. Use the OneSignal app dedicated to Appium tests (set `ONESIGNAL_APP_ID`/`ONESIGNAL_API_KEY` in `.env`) rather than a general or shared app. diff --git a/appium/scripts/run-local.sh b/appium/scripts/run-local.sh index af54ac3..a582807 100755 --- a/appium/scripts/run-local.sh +++ b/appium/scripts/run-local.sh @@ -199,6 +199,64 @@ if [[ "$SDK_TYPE" == "ios" && "$PLATFORM" != "ios" ]]; then exit 0 fi +# ── Preflight checks ────────────────────────────────────────────────────────── +# Fail fast on missing local tooling with the exact remediation, instead of +# surfacing as cryptic failures much later (e.g. a bare `appium: command not +# found` followed by a 30s startup timeout). CI never runs this script — it +# calls `vpx wdio run` directly on BrowserStack — so these checks are +# local-only by construction. +preflight() { + command -v appium >/dev/null 2>&1 \ + || error "appium not found on PATH. Install it with: npm i -g appium" + + local driver + if [[ "$PLATFORM" == "ios" ]]; then driver="xcuitest"; else driver="uiautomator2"; fi + appium driver list --installed 2>&1 | grep -q "$driver" \ + || error "Appium driver '$driver' is not installed. Install it with: appium driver install $driver + (check what's installed with: appium driver list --installed)" + + if ! command -v vpx >/dev/null 2>&1; then + if command -v vp >/dev/null 2>&1; then + error "vpx not found on PATH. Vite+ creates the vpx symlink on vp's first run — run 'vp --version' once, or reinstall: curl -fsSL https://vite.plus | bash" + fi + error "vpx not found on PATH. Install Vite+ with: curl -fsSL https://vite.plus | bash" + fi + + if [[ ! -d "$APPIUM_DIR/node_modules" ]]; then + # package.json declares "packageManager": "bun@…"; fall back to vp (which + # run_tests already uses) when bun isn't installed. + if command -v bun >/dev/null 2>&1; then + info "node_modules missing in $APPIUM_DIR — running 'bun install'..." + (cd "$APPIUM_DIR" && bun install) + elif command -v vp >/dev/null 2>&1; then + info "node_modules missing in $APPIUM_DIR — running 'vp install'..." + (cd "$APPIUM_DIR" && vp install) + else + error "node_modules missing in $APPIUM_DIR. Run 'bun install' (or 'vp install') there first." + fi + fi + + # webdriverio 9.x ships an undici-v6 dispatcher that Node 26+'s fetch + # rejects with UND_ERR_INVALID_ARG. WDIO_USE_NATIVE_FETCH=1 makes wdio skip + # the custom dispatcher. CI is on Node 24 and unaffected. + local node_major="" + if command -v node >/dev/null 2>&1; then + node_major="$(node -v 2>/dev/null | sed -E 's/^v([0-9]+).*/\1/' || true)" + fi + if [[ "${node_major:-0}" =~ ^[0-9]+$ ]] && (( ${node_major:-0} >= 26 )) && [[ -z "${WDIO_USE_NATIVE_FETCH:-}" ]]; then + export WDIO_USE_NATIVE_FETCH=1 + info "Node $node_major detected — setting WDIO_USE_NATIVE_FETCH=1 (works around webdriverio's undici dispatcher being rejected by Node 26+ fetch)." + fi + + if [[ -z "${ONESIGNAL_APP_ID:-}" || -z "${ONESIGNAL_API_KEY:-}" ]]; then + error "ONESIGNAL_APP_ID / ONESIGNAL_API_KEY not set. Use the OneSignal app + dedicated to Appium tests (not a general/shared app — its live in-app + marketing campaigns can cover the UI and cause misleading 'element not + displayed' failures). Set both in $SCRIPT_DIR/.env (cp .env.example .env)." + fi +} +preflight + # ── Real-device validation + signing setup ──────────────────────────────────── # When --device-real is set, we need a physical-device build and codesigning # inputs. Centralised here so the rest of the script stays simulator-shaped @@ -1731,11 +1789,6 @@ start_ios_simulator() { return fi - if xcrun simctl list devices booted 2>/dev/null | grep -q "Booted"; then - info "Simulator already running" - return - fi - local udid udid=$(xcrun simctl list devices available -j \ | python3 -c " @@ -1748,8 +1801,65 @@ for runtime, devices in data['devices'].items(): print(d['udid']); sys.exit(0) " 2>/dev/null || true) + # Requested device/runtime isn't installed on this machine (e.g. defaults + # assume iOS 26.2 but only 26.5 is installed). Fall back to the booted + # simulator if there is one, else the newest installed iOS runtime, and + # align DEVICE/OS_VERSION so the Appium session targets what actually runs. if [[ -z "$udid" ]]; then - error "Simulator '$IOS_SIMULATOR' ($IOS_RUNTIME) not found. Run: xcrun simctl list devices available" + warn "Simulator '$IOS_SIMULATOR' ($IOS_RUNTIME) not found on this machine." + local fallback + fallback=$(xcrun simctl list devices -j \ + | python3 -c " +import json, sys +data = json.load(sys.stdin) +for runtime, devices in data['devices'].items(): + if '.iOS-' not in runtime: + continue + for d in devices: + if d['state'] == 'Booted': + rt = runtime.rsplit('.', 1)[-1] + print(d['udid'] + '|' + d['name'] + '|' + rt + '|' + rt.replace('iOS-', '').replace('-', '.')) + sys.exit(0) +" 2>/dev/null || true) + if [[ -z "$fallback" ]]; then + fallback=$(xcrun simctl list devices available -j \ + | python3 -c " +import json, sys +data = json.load(sys.stdin) +runtimes = [] +for runtime, devices in data['devices'].items(): + rt = runtime.rsplit('.', 1)[-1] + if not rt.startswith('iOS-'): + continue + try: + ver = tuple(int(p) for p in rt.replace('iOS-', '').split('-')) + except ValueError: + continue + avail = [d for d in devices if d.get('isAvailable')] + if avail: + runtimes.append((ver, rt, avail)) +for ver, rt, avail in sorted(runtimes, reverse=True): + exact = [d for d in avail if d['name'] == '$IOS_SIMULATOR'] + iphones = sorted((d for d in avail if d['name'].startswith('iPhone')), key=lambda d: d['name']) + pick = exact[0] if exact else (iphones[-1] if iphones else avail[0]) + print(pick['udid'] + '|' + pick['name'] + '|' + rt + '|' + '.'.join(str(p) for p in ver)) + sys.exit(0) +" 2>/dev/null || true) + fi + if [[ -z "$fallback" ]]; then + error "No usable iOS simulator found. Run: xcrun simctl list devices available, then set DEVICE / OS_VERSION / IOS_RUNTIME in $SCRIPT_DIR/.env" + fi + udid="${fallback%%|*}" + IOS_SIMULATOR="$(cut -d'|' -f2 <<<"$fallback")" + IOS_RUNTIME="$(cut -d'|' -f3 <<<"$fallback")" + OS_VERSION="$(cut -d'|' -f4 <<<"$fallback")" + DEVICE="$IOS_SIMULATOR" + info "Falling back to '$IOS_SIMULATOR' (iOS $OS_VERSION, $udid). Set DEVICE / OS_VERSION / IOS_RUNTIME in $SCRIPT_DIR/.env to pin a different one." + fi + + if xcrun simctl list devices booted 2>/dev/null | grep -q "Booted"; then + info "Simulator already running" + return fi info "Booting simulator '$IOS_SIMULATOR' ($udid)..." @@ -1887,6 +1997,9 @@ cleanup_android_automation() { reset_app() { if [[ "$SKIP_RESET" == true ]]; then info "Skipping app reset (--skip-reset)" + if [[ "$PLATFORM" == "ios" ]]; then + warn "iOS notification-permission state persists with --skip-reset; the test that waits for the permission alert will fail if it was already decided. Re-run without --skip/--skip-reset to reset." + fi return fi @@ -1901,12 +2014,13 @@ reset_app() { xcrun devicectl device uninstall app --device "$UDID" "$bundle" 2>/dev/null || true else local sim_target="${UDID:-booted}" - if xcrun simctl listapps "$sim_target" 2>/dev/null | grep -q "$bundle"; then - info "Uninstalling $bundle..." - xcrun simctl uninstall "$sim_target" "$bundle" 2>/dev/null || true - else - info "App not installed — nothing to reset" - fi + # Uninstall unconditionally: a previously-decided notification permission + # survives reinstalls and makes the permission-alert test fail, and + # `simctl privacy` cannot reset it (notifications is SpringBoard state, + # not a TCC service — it's absent from `simctl privacy`'s service list). + # Uninstalling the app is the only reliable way to get the prompt back. + info "Uninstalling $bundle (also resets notification-permission state)..." + xcrun simctl uninstall "$sim_target" "$bundle" 2>/dev/null || true fi else local package="${BUNDLE_ID:-}"