CI: include answer-progress Playwright journey in production UI gate

[BigSimmo]

Motivation

• The production Playwright journey that validates answer-progress safety was not being collected or classified as a UI change, letting assertion edits slip past the required Chromium UI job and creating a P1 verification gap.

Description

• Add the answer-progress spec to Playwright selection by updating playwright.config.ts so the productionSpecPattern and testMatch include tests/answer-progress-ui-smoke.spec.ts.
• Ensure CI file-classifier treats edits to that spec as UI-affecting by adding tests/answer-progress-ui-smoke.spec.ts to uiPatterns in scripts/ci-change-scope.mjs and adding a self-test assertion answer-progress-playwright to the script's selfTest.
• Record the scoped review outcome and verification note in docs/branch-review-ledger.md so the change and its limited local verification are visible in the review ledger.
• Touched files: playwright.config.ts, scripts/ci-change-scope.mjs, docs/branch-review-ledger.md.

Testing

• Ran node scripts/ci-change-scope.mjs --self-test and it passed, exercising the new answer-progress-playwright assertion.
• Ran node scripts/ci-change-scope.mjs --json --files tests/answer-progress-ui-smoke.spec.ts and it now reports ui_changed: true, proving the classifier will trigger the Production UI job for edits to that spec.
• Ran the local offline gates npm run verify:cheap and npm run verify:pr-local -- --files playwright.config.ts,scripts/ci-change-scope.mjs, both completed in the environment used for this change.
• Attempted a focused Playwright run for tests/answer-progress-ui-smoke.spec.ts but the isolated Next production build could not complete browser verification here because required Google-hosted font downloads failed in this environment, so the browser assertions did not reach completion; hosted CI/provider-backed UI runs remain required to fully validate the Playwright journey.

Notes: hosted CI and provider-backed checks were not executed per repository policy for local verification; the change is limited to test/CI selection and does not alter runtime application behavior.

---

Codex Task

Summary by CodeRabbit

• Bug Fixes
  • Improved CI test selection to run a broader set of Playwright specs, including expanded answer-progress and UI smoke scenarios.
  • Enhanced change detection so answer-progress UI updates are classified correctly and linked to the right validation coverage.
• Tests
  • Added a self-check to confirm the updated UI/coverage classification behavior for answer-progress UI smoke.
• Documentation
  • Updated the CI verification ledger entry with the new review record and coverage/validation status for the recent branch.

[supabase]

This pull request has been ignored for the connected project sjrfecxgysukkwxsowpy because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.

---

Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

[coderabbitai]

Warning

Review limit reached

@cursor[bot], you've reached your PR review limit, so we couldn't start this review.

Next review available in: 35 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 6d39ffbc-6af0-43ea-b339-6b7a58829d2b

📥 Commits

Reviewing files that changed from the base of the PR and between 1aca4a7 and ac7098e.

📒 Files selected for processing (1)

• docs/branch-review-ledger.md

📝 Walkthrough

Walkthrough

The PR updates CI change classification and Playwright filename matching so the answer-progress UI journey and additional tools UI specs are included in UI verification. A review ledger entry records the prior CI coverage gap and validation results.

Changes

CI UI coverage

Layer / File(s)	Summary
Classify answer-progress as UI coverage scripts/ci-change-scope.mjs	The classifier recognizes answer-progress-ui-smoke.spec.ts as a UI change and adds a self-test for the resulting flags.
Match Playwright UI specs playwright.config.ts, docs/branch-review-ledger.md	Playwright matchers include the answer-progress, tools-collapse, and tools-task-directory variants; the ledger records the related CI verification gap.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

• BigSimmo/Database#454: Updates CI change-scope classification and uses ui_changed to control UI jobs.
• BigSimmo/Database#592: Adjusts Playwright UI spec selection in the same configuration path.
• BigSimmo/Database#705: Hardens Playwright selection and CI change-scope classification in the same files.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly matches the main change: adding the answer-progress Playwright journey to the production UI gate.
Description check	✅ Passed	The description covers motivation, implementation, and testing, but it does not follow the repository's required Summary/Verification/Risk template.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/improve-verification-system-in-bigsimmo

---

_{Comment @coderabbitai help to get the list of available commands.}

[BigSimmo]

@copilot resolve the merge conflicts in this pull request

[Copilot]

@copilot resolve the merge conflicts in this pull request

Resolved the merge conflicts by merging origin/main into this branch and fixing docs/branch-review-ledger.md to preserve both sides. Addressed in commit 1aca4a7a.

[Copilot]

@copilot resolve the merge conflicts in this pull request

Merge conflict resolved in commit 1aca4a7. The docs/branch-review-ledger.md conflict has been fixed by combining the codex/ci-answer-progress-regression entry from this branch with the PR #635 and PR #718 entries from main.

The PR policy CI check is still failing because the PR description uses ### headings (### Motivation, ### Description, ### Testing) instead of the ## headings required by scripts/pr-policy.mjs. Please update the PR description to the following to pass the policy check:

## Summary

Add `tests/answer-progress-ui-smoke.spec.ts` to the production Playwright UI gate and CI change-scope classifier. Previously the spec was excluded from both `playwright.config.ts` matchers and `ci-change-scope.mjs` UI patterns, creating a verification gap where assertion edits would not trigger the required Chromium UI job.

## Verification

- [x] `node scripts/ci-change-scope.mjs --self-test` — passed (new `answer-progress-playwright` assertion exercised)
- [x] `node scripts/ci-change-scope.mjs --json --files tests/answer-progress-ui-smoke.spec.ts` — reports `ui_changed: true`
- [x] `npm run verify:cheap`
- [x] `npm run verify:pr-local -- --files playwright.config.ts,scripts/ci-change-scope.mjs`

UI verification not run: browser assertions did not complete due to Google-hosted font download failures in this environment; hosted CI Chromium run required.

## Risk and rollout

- Risk: low; CI-only change affecting test selection and classification only. No runtime application behavior is changed.
- Rollback: revert this PR's single commit.
- Provider or production effects: None

[github-actions]

CI triage

CI failed on this PR. Automated classification of the 2 failed job(s):

• Static PR checks — needs investigation: inspect the failing step and uploaded diagnostics; rerun only after classifying the cause.
• PR required — needs investigation: inspect the failing step and uploaded diagnostics; rerun only after classifying the cause.

Compared with main CI run #3011 (success).

Classification is evidence routing, not permission to ignore a failure. Exact quarantined Playwright identities remain governed by the flake ledger.