ci: enforce safer pull request evidence

.github/pull_request_template.md

@@ -18,6 +18,16 @@ For retrieval, ranking, selection, chunking, source/citation rendering, or answe
 - [ ] `npm run check:production-readiness` when clinical workflow, privacy, environment, Supabase, source governance, or deployment behavior changed
 - [ ] `npm run check:deployment-readiness` when deployment startup, hosting, or rollout behavior changed
 
+<!-- Use `Verification not run: <reason>` or `UI verification not run: <reason>` when a required local gate cannot be run; do not leave an unchecked box as the only evidence. -->
+
+## Risk and rollout
+
+Complete this section for clinical, data, API, auth/privacy, workflow, dependency, build, or deployment changes.
+
+- Risk:
+- Rollback:
+- Provider or production effects: None / describe the explicitly authorized effect
+
 ## Clinical Governance Preflight
 
 Complete this section when the change touches ingestion, answer generation, search/ranking, source rendering, document access, privacy, production env, or clinical output.

.github/workflows/ci-triage.yml

@@ -1,7 +1,8 @@
 # CI failure triage. When CI fails on a pull request, post ONE comment that
 # classifies the failure so attention is not spent on failures that are not the
 # author's fault:
-#   - main-side: the same job is also failing on the latest default-branch run
+#   - main-side: the same job is also failing on the latest completed run of the
+#     same workflow on the default branch
 #     (CI merges the PR branch with current main, so a main regression surfaces on
 #     every open PR — this has cost debugging time before).
 #   - needs investigation: everything else.
@@ -35,15 +36,24 @@ jobs:
       github.event.workflow_run.conclusion == 'failure' &&
       github.event.workflow_run.event == 'pull_request'
     steps:
-      - name: Checkout default branch (trusted) for the flake ledger
+      - name: Checkout default branch (trusted) for triage policy
         uses: actions/checkout@9f698171ed81b15d1823a05fc7211befd50c8ae0 # v6.0.3
         with:
+          ref: ${{ github.event.repository.default_branch }}
           persist-credentials: false
 
       - name: Post triage comment
         uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
         with:
           script: |
+            const { pathToFileURL } = require("node:url");
+            const moduleUrl = pathToFileURL(`${process.env.GITHUB_WORKSPACE}/scripts/ci-triage.mjs`).href;
+            const {
+              buildTriageBody,
+              classifyFailedJobs,
+              failedJobNames,
+              selectLatestDefaultBranchRun,
+            } = await import(moduleUrl);
             const run = context.payload.workflow_run;
 
             // Resolve the PR for this run (empty for fork PRs → skip quietly).
@@ -60,55 +70,47 @@ jobs:
               run_id: run.id,
               per_page: 100,
             });
-            const failed = jobs.filter((j) => j.conclusion === "failure").map((j) => j.name);
+            const failed = failedJobNames(jobs);
             if (failed.length === 0) {
               core.info("No failed jobs found; skipping.");
               return;
             }
 
-            // Is the same job failing on the latest default-branch CI run? → main-side.
-            let mainFailingJobs = new Set();
+            // Compare only with the latest completed run of this exact workflow
+            // on the default branch. A repository-wide per_page=1 lookup can
+            // accidentally select a different workflow and misattribute a PR failure.
+            let mainRun;
+            let mainFailingJobs = [];
             try {
-              const { data: mainRuns } = await github.rest.actions.listWorkflowRunsForRepo({
+              const { data: mainRuns } = await github.rest.actions.listWorkflowRuns({
                 owner: context.repo.owner,
                 repo: context.repo.repo,
+                workflow_id: run.workflow_id,
                 event: "push",
                 branch: context.payload.repository.default_branch,
-                per_page: 1,
+                status: "completed",
+                per_page: 10,
               });
-              // reuse: find the CI workflow's latest main run
-              const latestMain = (mainRuns.workflow_runs || []).find((r) => r.name === run.name);
-              if (latestMain && latestMain.conclusion === "failure") {
+              mainRun = selectLatestDefaultBranchRun(mainRuns.workflow_runs, {
+                currentRunId: run.id,
+                defaultBranch: context.payload.repository.default_branch,
+              });
+              if (mainRun?.conclusion === "failure") {
                 const mainJobs = await github.paginate(github.rest.actions.listJobsForWorkflowRun, {
                   owner: context.repo.owner,
                   repo: context.repo.repo,
-                  run_id: latestMain.id,
+                  run_id: mainRun.id,
                   per_page: 100,
                 });
-                mainFailingJobs = new Set(mainJobs.filter((j) => j.conclusion === "failure").map((j) => j.name));
+                mainFailingJobs = failedJobNames(mainJobs);
               }
             } catch (e) {
               core.info(`main-side check skipped: ${e.message}`);
             }
 
-            const looksUi = (name) => /ui|playwright|browser|e2e/i.test(name);
-
-            const lines = failed.map((name) => {
-              if (mainFailingJobs.has(name)) return `- \`${name}\` — **main-side**: also failing on the latest \`main\` run, likely not your change.`;
-              if (looksUi(name)) return `- \`${name}\` — **needs investigation**: use the uploaded JUnit classification and trace; job type alone is not evidence of a known flake.`;
-              return `- \`${name}\` — **needs investigation**.`;
-            });
-
+            const classifications = classifyFailedJobs(failed, mainRun, mainFailingJobs);
+            const body = buildTriageBody(classifications, mainRun);
             const marker = "<!-- ci-triage -->";
-            const body = [
-              marker,
-              `### CI triage`,
-              `CI failed on this PR. Automated classification of the ${failed.length} failed job(s):`,
-              "",
-              ...lines,
-              "",
-              "_Heuristic only — a main-side or flake label is a starting point, not a verdict._",
-            ].join("\n");
 
             // Replace a prior triage comment instead of stacking.
             const { data: comments } = await github.rest.issues.listComments({

.github/workflows/ci.yml

@@ -82,6 +82,12 @@ jobs:
       - name: CI scope self-test
         run: npm run check:ci-scope
 
+      - name: CI triage self-test
+        run: npm run check:ci-triage
+
+      - name: PR policy self-test
+        run: npm run check:pr-policy
+
       - name: Format check
         run: npm run format:check
 

.github/workflows/pr-policy.yml

@@ -0,0 +1,74 @@
+name: PR Policy
+
+on:
+  pull_request_target:
+    branches: [main]
+    types: [opened, edited, synchronize, reopened, ready_for_review, labeled, unlabeled]
+  merge_group:
+
+concurrency:
+  group: pr-policy-${{ github.event.pull_request.number || github.ref }}
+  cancel-in-progress: true
+
+permissions:
+  contents: read
+  pull-requests: read
+
+jobs:
+  policy:
+    name: PR policy
+    runs-on: ubuntu-24.04
+    timeout-minutes: 5
+    steps:
+      - name: Accept merge queue validation
+        if: github.event_name == 'merge_group'
+        run: echo "PR metadata was validated before merge-queue entry."
+
+      # pull_request_target runs trusted base-branch code. Pin the checkout to
+      # the PR's exact base SHA and never execute the PR head or persist credentials.
+      - name: Checkout trusted policy
+        if: github.event_name == 'pull_request_target'
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        with:
+          ref: ${{ github.event.pull_request.base.sha }}
+          persist-credentials: false
+
+      - name: Validate pull request evidence
+        if: github.event_name == 'pull_request_target'
+        uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
+        with:
+          script: |
+            const { pathToFileURL } = require("node:url");
+            const moduleUrl = pathToFileURL(`${process.env.GITHUB_WORKSPACE}/scripts/pr-policy.mjs`).href;
+            const { evaluatePullRequestPolicy } = await import(moduleUrl);
+            const pr = context.payload.pull_request;
+
+            if (pr.draft) {
+              core.notice("Draft PR: metadata policy will be enforced when the PR is marked ready for review.");
+              return;
+            }
+
+            const changedFiles = await github.paginate(github.rest.pulls.listFiles, {
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              pull_number: pr.number,
+              per_page: 100,
+            });
+            const result = evaluatePullRequestPolicy({
+              title: pr.title,
+              body: pr.body || "",
+              headRef: pr.head.ref,
+              files: changedFiles.map((file) => file.filename),
+            });
+
+            await core.summary
+              .addHeading("PR policy")
+              .addRaw(`Clinical risk: ${result.classification.clinicalRisk ? "yes" : "no"}<br>`)
+              .addRaw(`Operational risk: ${result.classification.operationalRisk ? "yes" : "no"}<br>`)
+              .addRaw(`UI change: ${result.classification.ui ? "yes" : "no"}<br>`)
+              .write();
+
+            if (!result.ok) {
+              for (const error of result.errors) core.error(error);
+              core.setFailed(`PR policy found ${result.errors.length} issue(s). Edit the PR title/body and rerun.`);
+            }

docs/branch-review-ledger.md

@@ -572,3 +572,5 @@ Use this ledger to prevent repeated branch and PR reviews when the reviewed HEAD
 | 2026-07-17 | codex/chat-forms-import-6914 | e5caaa46cad5fb9a937f1dc43312723799b98abb + working-tree diff | shared Forms/Services catalogue access and LOCAL_NO_AUTH_OWNER_ID review | Fixed one P1 availability/design defect: authenticated reads materialized a private copy of the shared catalogue on first access, creating drift, unnecessary writes, and possible registry-corpus side effects. Forms/Services now merge the reviewed shared baseline with private owner overrides for list, detail, and universal search; older partial overrides retain missing shared metadata; no registry GET seeds or embeds. Private rows and linked documents remain owner-scoped. The ignored local owner setting was corrected to the verified sole live-owner UUID and source validation now requires a UUID. No remaining high-confidence defect in scope. | Focused registry/universal Vitest initially exposed four local expectation/count mismatches; corrected registry suite passed 17/17 and registry/logging suite passed 23/23. Full TypeScript passed. `verify:cheap` passed runtime, action pins, sitemap, brand, type/icon/function guards, full lint, and TypeScript; full Vitest reached 2,588 passing with one stale logging-guard failure, which was fixed and focused-verified. The final full-suite rerun was terminated by the 5-minute host timeout without a reported assertion failure. `git diff --check` passed. No Supabase/OpenAI/provider call or schema/RLS mutation was run for this review. |
 | 2026-07-17 | codex/scrolling-cleanup-20260717 | ff77cd06c + latest origin/main sync | cross-page scrolling and interaction stability review | Fixed two confirmed P2 defects: desktop action-popup placement performed synchronous geometry work for every captured scroll event, and submitted differential searches with zero document matches fell back to the home state. Placement is now coalesced per animation frame with passive scroll listeners, and submitted empty-evidence results remain visible. Hardened three popup/navigation browser helpers that reproduced hydration timing failures. No other high-confidence defect remains in the scoped diff. | Scroll-focused Chromium 28/28; final affected Chromium 5/5; source regressions 2/2; scoped ESLint; Prettier; TypeScript and production build passed before the final upstream-only sync; `git diff --check`. The aggregate local Vitest/UI runs were affected by concurrent-worktree resource contention, so exact-head hosted CI remains required before merge. No Supabase/OpenAI/live-provider checks run. |
 | 2026-07-17 | PR #713 / codex/chat-workflow-ideas-0916 | b52112df6aa36311d7420189064acd79dcf2c3f5 + reviewed follow-up diff | workflow toolkit review follow-up | Fixed all 14 actionable Codex and CodeRabbit threads: cross-platform path fixtures, installation-managed preflight guidance, complete Supabase-backed API database scoping, per-command approval boundaries, plugin-ignore narrowing, isolated CI-scope proof, remote-Git command guarding, repository-skill verification classification, `TypeError` diagnosis, strict CLI option values, machine-parseable JSON evidence output, and preservation of baseline database/clinical approval gates in the RAG lab. No unresolved actionable finding remains in the reviewed scope. | `npm run verify:cheap` passed with 273 files and 2,599 tests; focused toolkit Vitest 20/20; CI-scope self-test; plugin-ignore proof; `git diff --check`. Exact-head hosted CI remains required after the follow-up push. No Supabase, OpenAI, or other live product-provider command was run. |
+| 2026-07-17 | PR #699 / codex/test-reliability-hardening | 6202835ab1cf3703af311b9afdf372f74c63e040 | branch-cleanup-superseded | Closed as fully superseded by merged PR #705 (`e5caaa46c`). Range-diff maps the original implementation commit to #705's first integration commit; #705 then adds seven focused reliability fixes, while #699's remaining commit is merge-only and contributes no unique relevant patch. The exact-SHA remote ref and the unregistered local predecessor ref (`b518c1de9`) were deleted after final rechecks. | Fresh GitHub PR/head/status inventory; exact `ls-remote` and local-ref checks; cherry-pick-aware log; range-diff against PR #705's merged head; merge-parent verification; exact leased remote deletion and exact-old-value local `update-ref` deletion. No Supabase/OpenAI/product-provider checks run. |
+| 2026-07-17 | PR #716 / codex/pr-process-hardening-20260717 | 220de891f10f82df11eb2e8137367514c139a206 + reviewed implementation/follow-up diff | PR metadata, CI triage, review routing, Actions permissions, secret scanning, and branch-hygiene hardening | No remaining high-confidence P0-P2 defect in the reviewed process diff. Added a trusted base-SHA PR metadata/risk policy with draft and merge-queue handling; corrected CI triage to compare only the same workflow's latest completed `main` run; wired self-tests into local/hosted gates; documented the policy; created the missing durable Codex routing labels; removed four unused per-head routing labels; and applied read-only default Actions tokens, immutable Action pinning, no Actions-authored approvals, push protection, and automatic merged-branch deletion. Four policy defects were fixed before merge: API-only paths no longer trigger UI evidence, slash-bearing outcome titles are accepted, all seven governance items are required exactly, and placeholder risk/rollback text is rejected. | Offline `check:pr-policy`, `check:ci-triage`, `check:ci-scope`, action-pin check, scoped Prettier, and `git diff --check` passed. Initial hosted CI passed static, safety, coverage, build, images, Semgrep, Gitleaks, GitGuardian, and the aggregate gate; exact-head CI is required after the review fix. Broader local gates were deferred because other registered worktrees repeatedly held the heavyweight lock. GitHub provider inspection/settings changes were explicitly authorized. No Supabase/OpenAI/product-provider command ran. |

docs/codex-review-protocol.md

@@ -11,6 +11,7 @@ Use this protocol for every Codex review, audit, bug hunt, PR review, release-re
 - Before branch or PR review, check `docs/branch-review-ledger.md`: resolve the target with `git rev-parse`, compare the HEAD and scope, and skip unchanged completed reviews unless the user asks for a fresh pass.
 - Treat GitHub automatic review as one pass per pull request. A repair commit or later head does not authorize another automatic pass; require an explicit human request before re-reviewing.
 - Route automatic repair only for high-risk paths, at least 10 changed non-test source files, at least 300 changed non-test source lines, or an explicit `codex-review` label. `skip-codex-review` always opts out, including when both labels are present. Small low-risk, docs-only, test-only, and generated-only changes should not receive the automatic repair request.
+- Ready PRs must pass the trusted `PR policy` metadata check. It reads only the base-branch policy implementation, never executes PR code, and requires concrete verification plus risk/rollback evidence for high-risk changes.
 
 ## Review Output
 

docs/process-hardening.md

@@ -351,6 +351,26 @@ the durable index for the tooling; `docs/operator-backlog.md` tracks the human-o
   as main-side or needs-investigation. Inert until repo var `CI_TRIAGE_ENABLED=true` (now set). UI jobs use
   their uploaded JUnit classification and trace; job names alone never produce a known-flake verdict.
   The workflow reads only trusted default-branch job metadata and never runs PR code.
+- **PR metadata policy** (`.github/workflows/pr-policy.yml`, `scripts/pr-policy.mjs`): ready PRs to `main`
+  must use an outcome-focused title, complete Summary and Verification evidence, and provide risk/rollback
+  evidence for clinical or operationally sensitive paths. UI changes require `verify:ui` evidence (or an
+  explicit reason it could not run), while clinical-risk changes must fully disposition the governance
+  checklist. The `pull_request_target` job checks out the exact base SHA, has read-only permissions, and
+  never executes PR-head code. Drafts remain non-blocking until marked ready; merge-queue runs emit the
+  same stable `PR policy` check name.
+- **Default-branch failure attribution** (`scripts/ci-triage.mjs`): triage now compares a failed PR only
+  with the latest completed run of the same workflow on `main`. It no longer samples the latest arbitrary
+  repository workflow, which could incorrectly label a PR failure as main-side. A main-side label remains
+  routing evidence only; it never suppresses the required failure.
+- **Repository permission baseline (applied 2026-07-17):** Actions receive read-only tokens by default,
+  cannot approve pull requests, and must reference external actions by immutable SHA. Workflows that post
+  issues/comments retain narrow explicit permissions. Secret-scanning push protection is enabled and
+  merged branches are deleted automatically. Non-provider pattern scanning and credential-validity checks
+  remain unavailable for this user-owned repository/plan, so the ordinary secret scan, push protection,
+  Gitleaks check, and local secret-surface guards remain the active layers.
+- **Review-routing labels (applied 2026-07-17):** `codex-review` is the explicit opt-in for a normally
+  low-risk PR, and `skip-codex-review` is the unconditional opt-out. Four obsolete per-head `codex-ar-*`
+  labels from the retired routing mechanism were removed after confirming no open PR used them.
 - **Repo hygiene:** `check:env-parity` (env-var NAME reconciliation across `env.ts`, `check-ci-env.mjs`,
   and — opt-in, names-only — `gh secret list` / Railway) and `sweep:branch-ledger` (report-only branch
   inventory, cherry-pick-aware).