Visual regression infrastructure for design-system migrations (#554/#556 follow-up)

[oleksandr-korin]

Summary

The design-system sweep PRs (#554) claim "visual output is unchanged" — currently we trust this based on palette equivalence math (status-success aliases colors.green directly). Adding automated visual regression would convert this claim into a CI-enforced contract.

The Playwright harness (#556 / PR #571) supports toHaveScreenshot() out of the box. The blocker is cross-platform snapshot rendering: macOS and Linux render fonts and anti-aliasing slightly differently, so baselines captured on a developer's mac fail in CI's Linux runner.

What's needed

Pick one of the standard fixes:

Option A — Capture baselines in CI, commit them

• CI workflow has an explicit --update-snapshots mode (manual trigger or label-gated)
• Workflow runs Playwright with --update-snapshots, commits the resulting PNGs back to the branch
• Subsequent runs check against those committed baselines
• Pros: zero local-dev burden, baselines are always Linux-rendered
• Cons: requires write-access from CI back to the branch (token + commit step)

Option B — Capture locally via Playwright Docker image

• Devs run snapshots inside mcr.microsoft.com/playwright:latest to match CI's Linux runtime
• Add a npm run test:e2e:update:linux script that wraps Docker
• Pros: no CI write permissions needed
• Cons: extra step for developers; slower locally

Option C — Platform-specific snapshot directories

• Use Playwright's snapshotPathTemplate to keep separate snapshots-darwin/ and snapshots-linux/ dirs
• Pros: simple
• Cons: doubles maintenance; baselines drift between platforms over time

Recommended approach

Option A — capture in CI, commit back. Most teams converge here; it's the lowest ongoing burden. The CI commit step needs:

• A PAT or GITHUB_TOKEN with contents: write permission
• A guard to only run on PRs with the ui label AND a special opt-in (e.g. update-snapshots label)
• Commit message convention so the auto-commits are easy to identify

Specs to capture once infra is in place

The design-system migrations would each get a corresponding visual spec:

• /monitoring — health summary cards + alerts banner (deferred from refactor(frontend): migrate Monitoring view to design-system tokens (#554) #595)
• /operating-room — queue list + queue card colors (refactor(frontend): migrate operator queue to design-system tokens (#554) #569 follow-up)
• Each future #554 slice as it lands

Acceptance

• One of A/B/C implemented and documented in e2e/README.md
• At least 2 visual regression specs capturing baselines for already-migrated slices
• CI demonstrably catches an intentional color change (e.g. swap status-success to colors.emerald) and fails

Why this matters

Design-system PRs are easy to review in token names but hard to verify in pixels. Without visual regression, any reviewer's "looks fine" rubber-stamp could miss a subtle palette swap. The token validator (added in #553) catches typos and palette-alias drift; visual regression closes the loop on rendered output.

Cross-reference

• Tracking issue: Sweep remaining frontend status colors to design-system tokens (#67 follow-up) #554 (design-system sweep)
• Harness lives at: src/frontend/e2e/ (added in feat(frontend): Playwright e2e harness gated on ui label (#556) #571)
• Test runner skill: /test-runner-ui-e2e already supports --update-snapshots mode (see feat(skills): add /test-runner-ui-e2e for Playwright e2e suite (#556) #582)