ci: PR smoke test for a11y-scan SPM plugin (end-to-end scan)

[Crash0v3rrid3]

What

Adds .github/workflows/spm-smoke-test.yml — a GitHub Actions workflow that smoke-tests the a11y-scan SwiftPM command plugin on every PR, with two jobs:

1. a11y-scan end-to-end (SwiftPM) (macOS) — the credentialed scan. cd tests/spm && swift build compiles the a11y-scan command plugin (via the path dependency) and the sample sources — the repo root is a plugin-only package with no buildable target, so it isn't built directly. Then cd tests/spm && swift test with RUN_A11Y_SCAN=1 + credentials un-gates the repo's existing testA11yScanPluginRuns, which drives scripts/run-a11y-scan.sh — a real scan that downloads the BrowserStack CLI, authenticates, and scans the sample SwiftUI sources with intentional a11y issues. It reuses the harness the repo already ships rather than duplicating the swift package plugin … scan invocation, so there's a single source of truth for the invocation.
2. Launcher scripts (bash syntax) (Ubuntu, no secrets) — bash -n over every launcher under scripts/. The bash/zsh/fish variants are all bash scripts (they differ only in which login shell they source creds from), so one syntax gate covers them. Static-syntax only — the scripts self-update, register git hooks and need creds, so they aren't executed here; checksum integrity stays covered by verify-selfupdate-checksums.yml.

This also corrects a latent bug in the harness the scan depends on: run-a11y-scan.sh (both the spm and xcode-app copies) passed --allow-network-connections 'all(ports: [])' — PackageDescription API syntax that isn't a valid CLI value and doesn't satisfy the plugin's declared need for ports 80/443, so SwiftPM refused the scan. Corrected to all:80,443.

Design notes

• Fork-safe. The scan needs BROWSERSTACK_USERNAME / BROWSERSTACK_ACCESS_KEY, and secrets are never exposed to fork PRs, so the scan job is gated with github.event.pull_request.head.repo.fork == false (plus workflow_dispatch for manual runs). The scripts-lint job needs no secrets and runs on all PRs including forks.
• Degrades gracefully. The scan step is guarded on the secrets being non-empty (if: env.BROWSERSTACK_USERNAME != '' && …). GitHub exposes an unset secret as an empty string (present, not nil), so without the guard the scan would run with empty creds and fail; with it, an unconfigured repo simply skips the scan and the job stays green on the build step.
• Proves the scan worked, not just ran. testA11yScanPluginRuns runs the scan twice and checks the tool's own exit-code contract: --non-strict must exit 0 (downloaded, authenticated, ran cleanly) and strict must exit non-zero (the intentional issues were actually detected). A clean-but-empty scan (silent no-op) fails the strict check.
• Resilient to transient upstream failures. The scan hits BrowserStack (network + auth + CLI download), so the step retries up to 3x with backoff; a consistent failure still fails the gate. swift test reuses the first attempt's build, so retries only re-run the scan.
• actions/checkout is pinned by SHA (v4.2.2) to match Semgrep.yml / verify-selfupdate-checksums.yml (supply-chain hardening, DEVA11Y-476).

Verified (CI + local)

• CI is green on both jobs. The end-to-end run exercises the two-run fidelity check with real credentials — strict mode exits non-zero, i.e. the sample's planted a11y issues are genuinely detected, not just "the plugin ran."
• Locally (macOS, Swift 6.2): tests/spm builds (Compiling plugin a11y-scan + sample sources); swift test with no RUN_A11Y_SCAN skips the e2e test and passes, confirming the no-secrets path stays green.

Notes / open questions

• BROWSERSTACK_USERNAME / BROWSERSTACK_ACCESS_KEY are already configured on the repo (the credentialed scan runs green); no secret setup needed.
• Covers the SwiftPM harness (tests/spm/). The tests/xcode-app/ build-phase harness is not wired up here — happy to add an Xcode job in a follow-up.
• actions/checkout@v4.2.2 emits a non-blocking Node 20 deprecation warning; left as-is to match the repo's existing pinned workflows.
• No DEVA11Y ticket linked yet — branch is chore/spm-pr-smoke-test; rename/link as needed.

🤖 Generated with Claude Code

[Crash0v3rrid3]

Smoke gate is green ✅ — and it caught a real pre-existing bug

The first CI run failed at the scan step:

error: Plugin 'a11y-scan' wants permission to allow all network connections on ports: 80, 443. Use `--allow-network-connections all:80,443` to allow this.

Root cause (not in the workflow): the harness scripts passed --allow-network-connections 'all(ports: [])' — that's PackageDescription API syntax pasted into a CLI flag, and the empty port list didn't satisfy the plugin's declared need for ports 80/443. This had never been exercised in CI before, so it went unnoticed. Fixed to all:80,443 in:

• tests/spm/scripts/run-a11y-scan.sh
• tests/xcode-app/scripts/run-a11y-scan.sh
• tests/spm/README.md

After the fix, the end-to-end scan runs and passes (42s) with the repo secrets. BROWSERSTACK_USERNAME / BROWSERSTACK_ACCESS_KEY are clearly already configured on the repo — the scan authenticated and completed.

(Non-blocking) The Node 20 deprecation warning is from actions/checkout v4.2.2, which the repo's other workflows also pin — left as-is for consistency.

[Crash0v3rrid3]

Added: scripts-lint job — syntax-checks all scripts/ launchers

Per request, the workflow now has a second job that runs bash -n on every launcher script under scripts/ (all six: bash/zsh/fish × cli/spm). They're all bash scripts — the zsh/fish variants only differ in which login shell they source creds from — so one bash syntax gate covers them.

• Runs on ubuntu-latest, needs no secrets, so it runs on all PRs including forks (unlike the credentialed e2e job).
• Scripts are not executed (they self-update, register git hooks and need creds) — this is a static syntax gate. Checksum integrity stays covered by verify-selfupdate-checksums.yml.

Latest run: both jobs green — a11y-scan end-to-end (SwiftPM) ✅ and Launcher scripts (bash syntax) ✅ (all 6 scripts bash -n OK).

Out of scope, FYI: shellcheck flags SC2096 on the #!/usr/bin/env bash -il shebang in all six scripts (env accepts only one arg on most OSes → breaks direct ./script.sh execution on Linux). Pre-existing on main; not touched here. Worth a follow-up.

[Crash0v3rrid3]

Both open review findings addressed ✅ — CI green (c24c577)

#2 — smoke fidelity. The e2e test (testA11yScanPluginRuns) previously asserted only --non-strict exit 0, so a scan that authenticated but found nothing (silent no-op) would pass green. It now runs the scan twice and uses the tool's own exit-code contract:

• --non-strict -> exit 0 (downloaded CLI, authenticated, ran cleanly), and
• strict -> exit non-zero (the intentional issues in SampleViews.swift were actually detected).

Exit codes, not brittle output-matching. Added a runScan() helper that drains stdout+stderr to EOF before waitUntilExit (no pipe-buffer deadlock) and includes output in failure messages. Also tightened the cred guard to skip on empty-string (not just nil), matching the workflow guard. Validated on CI with real creds: the green run means strict exited non-zero, i.e. the scan genuinely flagged the sample's a11y issues.

#3 — flaky prod-dependent gate. The scan hits BrowserStack (network + auth + CLI download) on every same-repo PR. Wrapped the scan step in a bounded retry (3 attempts, 20s backoff) so a transient upstream hiccup doesn't red-block a PR; a consistent failure still fails the gate. swift test reuses the first attempt's build, so retries only re-run the scan (CLI is cached). Retry didn't need to fire this run.

Verified: tests compile, no-creds path still skips cleanly, both CI jobs green with the new logic exercised. Remaining non-blocking item: the Node 20 deprecation warning on actions/checkout@v4.2.2 (repo-wide pin).