Skip to content

ci: make the Chromium regression suite merge-blocking#593

Merged
BigSimmo merged 1 commit into
mainfrom
claude/audit-ci-browser-gate-2026-07-13
Jul 13, 2026
Merged

ci: make the Chromium regression suite merge-blocking#593
BigSimmo merged 1 commit into
mainfrom
claude/audit-ci-browser-gate-2026-07-13

Conversation

@BigSimmo

Copy link
Copy Markdown
Owner

Summary

2026-07-13 audit finding 8: the broad Chromium suite ran as "Advisory UI regression" with continue-on-error: true and no place in the required aggregate, so required CI could pass while real browser regressions failed — the stale /privacy heading assertion (independently fixed on main since) shipped exactly that way.

  • ui-regression now runs the stable set via test:e2e:regression (--grep-invert "@critical|@quarantine"), gates pr-required on UI changes, and matches ui-critical's trigger scope.
  • New advisory ui-quarantine job runs @quarantine-tagged specs (test:e2e:quarantine, --pass-with-no-tests).
  • No test is quarantined at flip time: the full 142-test stable set was run locally during triage — the only genuine failure was the /privacy drift; three cold-dev-server timing timeouts pass on a warmed server and have no CI failure history, documented as a watch-list in docs/process-hardening.md (tag @quarantine if one flakes rather than reverting the gate).

Verification

  • Full 142-test regression set exercised locally during triage (one real failure fixed; remainder green)
  • npm run check:github-actions, npm run check:ci-scope, YAML parse, schema/yaml-contract vitest suites on the rebased head
  • test:e2e:quarantine exits 0 with no tagged tests (--pass-with-no-tests)
  • Branch-protection contexts unchanged: the aggregate remains CI / PR required (no new required-context names)

Notes

  • Merge-gating change for every UI-touching PR; revert is a single-commit rollback.

🤖 Generated with Claude Code

The broad Chromium suite ran as "Advisory UI regression" with
continue-on-error and no place in the required aggregate, so required CI
could pass while real browser regressions failed (2026-07-13 audit,
finding 8) — the stale /privacy heading assertion fixed here shipped
exactly that way. Changes:

- ui-regression now runs the stable set (test:e2e:regression,
  --grep-invert "@critical|@quarantine"), gates pr-required on UI changes,
  and matches ui-critical's trigger scope.
- New advisory ui-quarantine job runs @quarantine-tagged specs
  (test:e2e:quarantine, --pass-with-no-tests). No test is quarantined at
  flip time: the full 142-test set was green locally after fixing the
  /privacy drift; three cold-server timing timeouts that pass warmed and
  have no CI failure history are documented as a watch-list in
  docs/process-hardening.md.
- Fix the stale /privacy assertions in ui-smoke (page title is
  "Privacy & data handling" since the footer simplification).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@supabase

supabase Bot commented Jul 13, 2026

Copy link
Copy Markdown

This pull request has been ignored for the connected project sjrfecxgysukkwxsowpy because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

@coderabbitai

coderabbitai Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@BigSimmo, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 1 minute

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: b7262391-f879-4fe8-b316-0453dd2a2354

📥 Commits

Reviewing files that changed from the base of the PR and between d430b03 and 09be064.

📒 Files selected for processing (3)
  • .github/workflows/ci.yml
  • docs/process-hardening.md
  • package.json
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/audit-ci-browser-gate-2026-07-13
✨ Simplify code
  • Create PR with simplified code
  • Commit simplified code in branch claude/audit-ci-browser-gate-2026-07-13

Comment @coderabbitai help to get the list of available commands.

@BigSimmo
BigSimmo enabled auto-merge (squash) July 13, 2026 14:28
@BigSimmo
BigSimmo merged commit 3325a12 into main Jul 13, 2026
17 of 18 checks passed
BigSimmo added a commit that referenced this pull request Jul 13, 2026
#593 already promoted ui-regression (test:e2e:regression with the
@quarantine escape hatch); this branch keeps only its unique delta —
the chromium-mockups project split, the advisory ui-mockups lane, and
the cross-mode spec hardening.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@BigSimmo
BigSimmo deleted the claude/audit-ci-browser-gate-2026-07-13 branch July 13, 2026 18:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant