From f05afc1ad416c8abb715b4b22b51dd0aba902004 Mon Sep 17 00:00:00 2001 From: BigSimmo <87357024+BigSimmo@users.noreply.github.com> Date: Fri, 24 Jul 2026 11:27:15 +0800 Subject: [PATCH 01/12] fix: fail closed on missing answer relevance --- docs/branch-review-ledger.md | 1 + src/components/ClinicalDashboard.tsx | 4 ++-- src/lib/answer-render-policy.ts | 7 +++++-- tests/answer-render-policy.test.ts | 24 ++++++++++++++++++++++++ 4 files changed, 32 insertions(+), 4 deletions(-) diff --git a/docs/branch-review-ledger.md b/docs/branch-review-ledger.md index a6af85953..2ca2cef69 100644 --- a/docs/branch-review-ledger.md +++ b/docs/branch-review-ledger.md @@ -20,6 +20,7 @@ Use this ledger to prevent repeated branch and PR reviews when the reviewed HEAD | Date | Branch or ref | Reviewed HEAD | Scope | Outcome | Checks | | ---------- | -------------------------------------------------------- | ---------------------------------------- | ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-07-24 | `codex/answer-relevance-fail-closed` | `5ca1ae5cf82e398bf3861517b834dc629bff45ca` + reviewed working diff | Missing answer-relevance metadata across render policy and live dashboard grounding | APPROVE. No P0-P2 finding. A shared `isAnswerSourceBacked` predicate now requires explicit `isSourceBacked: true`; missing metadata cannot retain high render trust or a grounded dashboard state, explicit false remains conservative, and explicit true preserves supported behavior. Retrieval, ranking, generation, source selection and stored data are unchanged. Highest residual risk is deliberate compatibility tightening for older answer payloads without relevance metadata; they render low-trust and expose review sources rather than richer evidence blocks. | Red policy proof failed with `high` before the fix; focused render/provenance/clinical-safety tests 37/37; typecheck; offline RAG 21 suites/308 tests and 36/36 fixtures; production-readiness READY against `Clinical KB Database` using the configured environment read-only; `verify:pr-local` passed runtime, formatting, lint, typecheck, all 366 test files (3,254 passed/1 skipped), production build (1,677 pages), client-bundle secret scan and RAG fixture validation. An earlier `verify:cheap` attempt passed all non-test gates and 3,253 tests before known issue #067 timed out; its isolated suite passed 5/5, and the subsequent full PR-local suite passed. No live RAG, OpenAI request, Supabase mutation, Railway action, production data operation or deployment ran. | | 2026-07-24 | `codex/safety-plan-no-patient-data-contract` | `b94987c94537f3114a3429848fa908bdecd1d80a` | Safety Plan Generator identifier, local-state, copy, print, privacy-notice and PIA contract | APPROVE. No P0-P2 finding. The patient name/initials field is removed; the builder now asks for identifier-free minimum content, retains working state only in the mounted React component, and makes clipboard/print/PDF export an explicit handling boundary. The PIA and product privacy copy distinguish this local-only tool from provider-backed questions. Highest residual risk is outside Clinical KB: users must handle exported copies under an approved clinical-record process, which the UI now states at the export controls. | Privacy/component DOM 3/3 plus updated privacy-copy 2/2; focused Chromium copy/print/no-fetch-or-XHR 1/1; `verify:cheap` passed all 21 gates, 366 files and 3,245 tests with 1 skip; production-readiness READY using the existing canonical environment without a provider call; production build and client-bundle secret scan passed; offline RAG fixture/manifest 36 cases/21 suites passed. `verify:pr-local` passed runtime, formatting, lint and typecheck, then stopped on the unrelated load-sensitive `reconciliation-preflight` 30-second timeout; that test passed 5/5 isolated and the preceding full suite passed, so the unchanged five-minute gate was not retried. No Supabase, OpenAI, Railway, live RAG, production data or deployment action ran. | | 2026-07-24 | PR #1114 / `codex/universal-ledger-main-followup-20260724` | `eedbe594a89eaa0aad8a517ee8808b089cb5ff57` + reviewed working diff | Streamlined-sidebar handoff in the merged universal ledger | APPROVE. PRs #1112, #1116, and #1117 landed the universal-ledger baseline, two preserved UI tasks, and credential/hook hardening; this follow-up preserves that current-main schema and adds only the still-missing sidebar delivery as collision-free `#066`. The queue remains gap-free and every recommended ID has one open row. Sidebar implementation stays recoverable from remote branch `origin/codex/sidebar-test-fix-20260723` at full commit `cd54e68fbf7b07b5dffe3220e36af2caa528da54` until its local/browser/build/hosted merge proof is complete. | Focused sidebar/favourites tests previously passed 18/18 on the recorded implementation commit; this ledger follow-up passed Prettier, remote-ref recovery, 36-row contiguous-order/46-open-ID/next-ID invariants, and `git diff --check`. The broader reconciled precursor passed `verify:cheap` (365 files, 3,242 passed/1 skipped); fresh exact-head hosted CI is required on PR #1114. No OpenAI, Supabase, Railway, deployment, live-app, credential, or production-data action ran. | | 2026-07-24 | `codex/universal-ledger-main-final` | `527988c2ccabc98b4d0673d33360c971df65fa0e` + reviewed working diff | Current-main universal-ledger reconciliation after PR #1106 superseded PR #1109 | READY. Kept PR #1106's current-main ledger and IDs, carried forward only non-duplicate recommended work from the superseded branch, and fixed the confirmed SessionStart empty-open defect. Every active recommendation now has a durable open ID; exposed-GitHub-token containment is the first A1 item; the Safety Plan privacy contract, absent-relevance fail-closed rule, stranded-upload recovery, threshold-conflict design, catalogue-toolbar convergence, and Current Clinical Work brief are retained without duplicating #1106's legal/config/release/staging/seed packages. Resolved #014/#034 claims stay archived, and PR #1110's scheduled-diagnostics priority remains intact. Highest residual risk is manual queue/open-table drift. | Empty-open fixture and real-ledger Bash execution passed; 33 contiguous recommendations reference tracked open IDs; 43 open items; no duplicate queued IDs; `docs:check-links`, `docs:check-index`, `docs:check-scripts`, `check:skills`, Prettier, and `git diff --check` passed. No OpenAI, Supabase, Railway, production, deployment, live-app, or credential action. | diff --git a/src/components/ClinicalDashboard.tsx b/src/components/ClinicalDashboard.tsx index 51a31a707..a95f2fbc9 100644 --- a/src/components/ClinicalDashboard.tsx +++ b/src/components/ClinicalDashboard.tsx @@ -206,7 +206,7 @@ import { maxStoredAnswerTurns, savePersistedAnswerThread, } from "@/lib/answer-thread-storage"; -import { buildAnswerRenderModel } from "@/lib/answer-render-policy"; +import { buildAnswerRenderModel, isAnswerSourceBacked } from "@/lib/answer-render-policy"; import { frontendSourceGovernanceWarnings, groupSourceGovernanceWarnings, @@ -2907,7 +2907,7 @@ export function ClinicalDashboard({ const answerGrounded = answer?.grounded === true && answer.confidence !== "unsupported" && - currentRelevance?.isSourceBacked !== false && + isAnswerSourceBacked(answer) && answerRenderModel?.trust !== "unsupported"; const sourceLookup = useMemo(() => new Map(sources.map((source) => [source.id, source])), [sources]); const answerPreformatted = isPreformattedGroundedAnswer(answer); diff --git a/src/lib/answer-render-policy.ts b/src/lib/answer-render-policy.ts index 642144598..17e311ac3 100644 --- a/src/lib/answer-render-policy.ts +++ b/src/lib/answer-render-policy.ts @@ -139,10 +139,13 @@ function answerRelevance(answer: RagAnswer): EvidenceRelevance | undefined { return answer.relevance ?? answer.smartPanel?.relevance; } +export function isAnswerSourceBacked(answer: RagAnswer): boolean { + return answerRelevance(answer)?.isSourceBacked === true; +} + function deriveTrust(answer: RagAnswer): AnswerRenderTrust { - const relevance = answerRelevance(answer); const retrievalBlocked = answer.retrievalDiagnostics?.gateStatus === "blocked"; - const sourceBacked = relevance?.isSourceBacked !== false; + const sourceBacked = isAnswerSourceBacked(answer); const hasFaithfulnessWarning = Boolean(answer.faithfulnessWarning || answer.unverifiedNumericTokens?.length); const evidenceGap = answer.responseMode === "evidence_gap"; diff --git a/tests/answer-render-policy.test.ts b/tests/answer-render-policy.test.ts index 917b7db36..3239cc5ea 100644 --- a/tests/answer-render-policy.test.ts +++ b/tests/answer-render-policy.test.ts @@ -97,6 +97,17 @@ function answer(overrides: Partial = {}): RagAnswer { answer: "For red-range blood results, withhold clozapine and contact the monitoring service.", grounded: true, confidence: "high", + relevance: { + verdict: "direct", + label: "Direct source support", + matchedTerms: ["clozapine", "monitoring"], + missingTerms: [], + directSourceCount: 1, + weakSourceCount: 0, + score: 0.95, + supportReason: "A direct source supports the answer.", + isSourceBacked: true, + }, citations: [citation()], sources: [baseSource], answerSections: [ @@ -278,6 +289,19 @@ describe("answer render policy", () => { expect(model.warnings).not.toContain("A supporting source is due for review."); }); + it("fails closed when answer relevance metadata is absent", () => { + const model = buildAnswerRenderModel( + answer({ + relevance: undefined, + smartPanel: undefined, + }), + ); + + expect(model.trust).toBe("low"); + expect(model.allowedBlocks).not.toContain("quoteCards"); + expect(model.allowedBlocks).not.toContain("relatedDocuments"); + }); + it("does not render high trust for high-risk claims supported only by unverified evidence", () => { const model = buildAnswerRenderModel( answer({ From 4725f19ce0af107adc2a4f0fea0b512ea743b193 Mon Sep 17 00:00:00 2001 From: BigSimmo <87357024+BigSimmo@users.noreply.github.com> Date: Fri, 24 Jul 2026 11:43:37 +0800 Subject: [PATCH 02/12] docs: record answer relevance UI verification --- docs/branch-review-ledger.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/branch-review-ledger.md b/docs/branch-review-ledger.md index 2ca2cef69..3bf33918b 100644 --- a/docs/branch-review-ledger.md +++ b/docs/branch-review-ledger.md @@ -20,7 +20,7 @@ Use this ledger to prevent repeated branch and PR reviews when the reviewed HEAD | Date | Branch or ref | Reviewed HEAD | Scope | Outcome | Checks | | ---------- | -------------------------------------------------------- | ---------------------------------------- | ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 2026-07-24 | `codex/answer-relevance-fail-closed` | `5ca1ae5cf82e398bf3861517b834dc629bff45ca` + reviewed working diff | Missing answer-relevance metadata across render policy and live dashboard grounding | APPROVE. No P0-P2 finding. A shared `isAnswerSourceBacked` predicate now requires explicit `isSourceBacked: true`; missing metadata cannot retain high render trust or a grounded dashboard state, explicit false remains conservative, and explicit true preserves supported behavior. Retrieval, ranking, generation, source selection and stored data are unchanged. Highest residual risk is deliberate compatibility tightening for older answer payloads without relevance metadata; they render low-trust and expose review sources rather than richer evidence blocks. | Red policy proof failed with `high` before the fix; focused render/provenance/clinical-safety tests 37/37; typecheck; offline RAG 21 suites/308 tests and 36/36 fixtures; production-readiness READY against `Clinical KB Database` using the configured environment read-only; `verify:pr-local` passed runtime, formatting, lint, typecheck, all 366 test files (3,254 passed/1 skipped), production build (1,677 pages), client-bundle secret scan and RAG fixture validation. An earlier `verify:cheap` attempt passed all non-test gates and 3,253 tests before known issue #067 timed out; its isolated suite passed 5/5, and the subsequent full PR-local suite passed. No live RAG, OpenAI request, Supabase mutation, Railway action, production data operation or deployment ran. | +| 2026-07-24 | `codex/answer-relevance-fail-closed` | `5ca1ae5cf82e398bf3861517b834dc629bff45ca` + reviewed working diff | Missing answer-relevance metadata across render policy and live dashboard grounding | APPROVE. No P0-P2 finding. A shared `isAnswerSourceBacked` predicate now requires explicit `isSourceBacked: true`; missing metadata cannot retain high render trust or a grounded dashboard state, explicit false remains conservative, and explicit true preserves supported behavior. Retrieval, ranking, generation, source selection and stored data are unchanged. Highest residual risk is deliberate compatibility tightening for older answer payloads without relevance metadata; they render low-trust and expose review sources rather than richer evidence blocks. | Red policy proof failed with `high` before the fix; focused render/provenance/clinical-safety tests 37/37; typecheck; offline RAG 21 suites/308 tests and 36/36 fixtures; production-readiness READY against `Clinical KB Database` using the configured environment read-only; `verify:pr-local` passed runtime, formatting, lint, typecheck, all 366 test files (3,254 passed/1 skipped), production build (1,677 pages), client-bundle secret scan and RAG fixture validation; `verify:ui` passed all 267 Chromium scenarios against the isolated production build. An earlier `verify:cheap` attempt passed all non-test gates and 3,253 tests before known issue #067 timed out; its isolated suite passed 5/5, and the subsequent full PR-local suite passed. No live RAG, OpenAI request, Supabase mutation, Railway action, production data operation or deployment ran. | | 2026-07-24 | `codex/safety-plan-no-patient-data-contract` | `b94987c94537f3114a3429848fa908bdecd1d80a` | Safety Plan Generator identifier, local-state, copy, print, privacy-notice and PIA contract | APPROVE. No P0-P2 finding. The patient name/initials field is removed; the builder now asks for identifier-free minimum content, retains working state only in the mounted React component, and makes clipboard/print/PDF export an explicit handling boundary. The PIA and product privacy copy distinguish this local-only tool from provider-backed questions. Highest residual risk is outside Clinical KB: users must handle exported copies under an approved clinical-record process, which the UI now states at the export controls. | Privacy/component DOM 3/3 plus updated privacy-copy 2/2; focused Chromium copy/print/no-fetch-or-XHR 1/1; `verify:cheap` passed all 21 gates, 366 files and 3,245 tests with 1 skip; production-readiness READY using the existing canonical environment without a provider call; production build and client-bundle secret scan passed; offline RAG fixture/manifest 36 cases/21 suites passed. `verify:pr-local` passed runtime, formatting, lint and typecheck, then stopped on the unrelated load-sensitive `reconciliation-preflight` 30-second timeout; that test passed 5/5 isolated and the preceding full suite passed, so the unchanged five-minute gate was not retried. No Supabase, OpenAI, Railway, live RAG, production data or deployment action ran. | | 2026-07-24 | PR #1114 / `codex/universal-ledger-main-followup-20260724` | `eedbe594a89eaa0aad8a517ee8808b089cb5ff57` + reviewed working diff | Streamlined-sidebar handoff in the merged universal ledger | APPROVE. PRs #1112, #1116, and #1117 landed the universal-ledger baseline, two preserved UI tasks, and credential/hook hardening; this follow-up preserves that current-main schema and adds only the still-missing sidebar delivery as collision-free `#066`. The queue remains gap-free and every recommended ID has one open row. Sidebar implementation stays recoverable from remote branch `origin/codex/sidebar-test-fix-20260723` at full commit `cd54e68fbf7b07b5dffe3220e36af2caa528da54` until its local/browser/build/hosted merge proof is complete. | Focused sidebar/favourites tests previously passed 18/18 on the recorded implementation commit; this ledger follow-up passed Prettier, remote-ref recovery, 36-row contiguous-order/46-open-ID/next-ID invariants, and `git diff --check`. The broader reconciled precursor passed `verify:cheap` (365 files, 3,242 passed/1 skipped); fresh exact-head hosted CI is required on PR #1114. No OpenAI, Supabase, Railway, deployment, live-app, credential, or production-data action ran. | | 2026-07-24 | `codex/universal-ledger-main-final` | `527988c2ccabc98b4d0673d33360c971df65fa0e` + reviewed working diff | Current-main universal-ledger reconciliation after PR #1106 superseded PR #1109 | READY. Kept PR #1106's current-main ledger and IDs, carried forward only non-duplicate recommended work from the superseded branch, and fixed the confirmed SessionStart empty-open defect. Every active recommendation now has a durable open ID; exposed-GitHub-token containment is the first A1 item; the Safety Plan privacy contract, absent-relevance fail-closed rule, stranded-upload recovery, threshold-conflict design, catalogue-toolbar convergence, and Current Clinical Work brief are retained without duplicating #1106's legal/config/release/staging/seed packages. Resolved #014/#034 claims stay archived, and PR #1110's scheduled-diagnostics priority remains intact. Highest residual risk is manual queue/open-table drift. | Empty-open fixture and real-ledger Bash execution passed; 33 contiguous recommendations reference tracked open IDs; 43 open items; no duplicate queued IDs; `docs:check-links`, `docs:check-index`, `docs:check-scripts`, `check:skills`, Prettier, and `git diff --check` passed. No OpenAI, Supabase, Railway, production, deployment, live-app, or credential action. | From 0072226794de9681b5f76bc24a19b4876e002c90 Mon Sep 17 00:00:00 2001 From: BigSimmo <87357024+BigSimmo@users.noreply.github.com> Date: Fri, 24 Jul 2026 12:01:32 +0800 Subject: [PATCH 03/12] fix: suppress unverified visual evidence --- src/lib/answer-render-policy.ts | 3 ++- tests/answer-render-policy.test.ts | 4 ++++ 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/src/lib/answer-render-policy.ts b/src/lib/answer-render-policy.ts index 17e311ac3..7973ca92d 100644 --- a/src/lib/answer-render-policy.ts +++ b/src/lib/answer-render-policy.ts @@ -745,7 +745,8 @@ export function buildAnswerRenderModel( const rawQuotes = answer.quoteCards ?? answer.smartPanel?.quotes ?? []; const rawVisualEvidence = answer.visualEvidence ?? answer.smartPanel?.visualEvidence ?? []; const rawRelatedDocuments = answer.relatedDocuments ?? answer.smartPanel?.relatedDocuments ?? []; - const visualLimit = hasDirectVisualNeed(answer) || trust === "high" ? caps.visual : 0; + const visualLimit = + isAnswerSourceBacked(answer) && (hasDirectVisualNeed(answer) || trust === "high") ? caps.visual : 0; const quoteCards = dedupeQuotes(rawQuotes, primarySources, caps.quotes); const visualEvidence = dedupeVisualEvidence(rawVisualEvidence, primarySources, visualLimit); const tables = buildCanonicalTables(visualEvidence); diff --git a/tests/answer-render-policy.test.ts b/tests/answer-render-policy.test.ts index 3239cc5ea..d7c73345d 100644 --- a/tests/answer-render-policy.test.ts +++ b/tests/answer-render-policy.test.ts @@ -299,7 +299,11 @@ describe("answer render policy", () => { expect(model.trust).toBe("low"); expect(model.allowedBlocks).not.toContain("quoteCards"); + expect(model.allowedBlocks).not.toContain("visualEvidence"); expect(model.allowedBlocks).not.toContain("relatedDocuments"); + expect(model.visualEvidence).toEqual([]); + expect(model.tables).toEqual([]); + expect(model.copyText).not.toContain("Displayed table evidence"); }); it("does not render high trust for high-risk claims supported only by unverified evidence", () => { From 25b9c3ed95ab86916c26837df6493586bf55b1b2 Mon Sep 17 00:00:00 2001 From: BigSimmo <87357024+BigSimmo@users.noreply.github.com> Date: Fri, 24 Jul 2026 12:04:47 +0800 Subject: [PATCH 04/12] docs: record answer relevance review fix --- docs/branch-review-ledger.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/branch-review-ledger.md b/docs/branch-review-ledger.md index 3bf33918b..c661edfd6 100644 --- a/docs/branch-review-ledger.md +++ b/docs/branch-review-ledger.md @@ -20,7 +20,7 @@ Use this ledger to prevent repeated branch and PR reviews when the reviewed HEAD | Date | Branch or ref | Reviewed HEAD | Scope | Outcome | Checks | | ---------- | -------------------------------------------------------- | ---------------------------------------- | ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 2026-07-24 | `codex/answer-relevance-fail-closed` | `5ca1ae5cf82e398bf3861517b834dc629bff45ca` + reviewed working diff | Missing answer-relevance metadata across render policy and live dashboard grounding | APPROVE. No P0-P2 finding. A shared `isAnswerSourceBacked` predicate now requires explicit `isSourceBacked: true`; missing metadata cannot retain high render trust or a grounded dashboard state, explicit false remains conservative, and explicit true preserves supported behavior. Retrieval, ranking, generation, source selection and stored data are unchanged. Highest residual risk is deliberate compatibility tightening for older answer payloads without relevance metadata; they render low-trust and expose review sources rather than richer evidence blocks. | Red policy proof failed with `high` before the fix; focused render/provenance/clinical-safety tests 37/37; typecheck; offline RAG 21 suites/308 tests and 36/36 fixtures; production-readiness READY against `Clinical KB Database` using the configured environment read-only; `verify:pr-local` passed runtime, formatting, lint, typecheck, all 366 test files (3,254 passed/1 skipped), production build (1,677 pages), client-bundle secret scan and RAG fixture validation; `verify:ui` passed all 267 Chromium scenarios against the isolated production build. An earlier `verify:cheap` attempt passed all non-test gates and 3,253 tests before known issue #067 timed out; its isolated suite passed 5/5, and the subsequent full PR-local suite passed. No live RAG, OpenAI request, Supabase mutation, Railway action, production data operation or deployment ran. | +| 2026-07-24 | `codex/answer-relevance-fail-closed` | `0072226794de9681b5f76bc24a19b4876e002c90` | Missing answer-relevance metadata across render policy and live dashboard grounding | APPROVE after resolving one review P2. A shared `isAnswerSourceBacked` predicate now requires explicit `isSourceBacked: true`; missing or explicitly negative relevance cannot retain high render trust, a grounded dashboard state, or visual/table evidence. Explicitly source-backed answers preserve supported behavior. Retrieval, ranking, generation, source selection and stored data are unchanged. Highest residual risk is deliberate compatibility tightening for older answer payloads without relevance metadata; they render low-trust and expose review sources rather than richer evidence blocks. | Initial red policy proof failed with `high`; the review follow-up red proof exposed retained visual evidence and then passed after the source-backed visual gate. Focused render/provenance/clinical-safety tests passed 37/37 and typecheck passed after the final code change. Offline RAG passed 21 suites/308 tests and 36/36 fixtures; production-readiness was READY against `Clinical KB Database` using the configured environment read-only; `verify:pr-local` passed runtime, formatting, lint, typecheck, all 366 test files (3,254 passed/1 skipped), production build (1,677 pages), client-bundle secret scan and RAG fixture validation; `verify:ui` passed all 267 Chromium scenarios against the isolated production build. An earlier `verify:cheap` attempt passed all non-test gates and 3,253 tests before known issue #067 timed out; its isolated suite passed 5/5, and the subsequent full PR-local suite passed. Fresh hosted checks are required at the final head. No live RAG, OpenAI request, Supabase mutation, Railway action, production data operation or deployment ran. | | 2026-07-24 | `codex/safety-plan-no-patient-data-contract` | `b94987c94537f3114a3429848fa908bdecd1d80a` | Safety Plan Generator identifier, local-state, copy, print, privacy-notice and PIA contract | APPROVE. No P0-P2 finding. The patient name/initials field is removed; the builder now asks for identifier-free minimum content, retains working state only in the mounted React component, and makes clipboard/print/PDF export an explicit handling boundary. The PIA and product privacy copy distinguish this local-only tool from provider-backed questions. Highest residual risk is outside Clinical KB: users must handle exported copies under an approved clinical-record process, which the UI now states at the export controls. | Privacy/component DOM 3/3 plus updated privacy-copy 2/2; focused Chromium copy/print/no-fetch-or-XHR 1/1; `verify:cheap` passed all 21 gates, 366 files and 3,245 tests with 1 skip; production-readiness READY using the existing canonical environment without a provider call; production build and client-bundle secret scan passed; offline RAG fixture/manifest 36 cases/21 suites passed. `verify:pr-local` passed runtime, formatting, lint and typecheck, then stopped on the unrelated load-sensitive `reconciliation-preflight` 30-second timeout; that test passed 5/5 isolated and the preceding full suite passed, so the unchanged five-minute gate was not retried. No Supabase, OpenAI, Railway, live RAG, production data or deployment action ran. | | 2026-07-24 | PR #1114 / `codex/universal-ledger-main-followup-20260724` | `eedbe594a89eaa0aad8a517ee8808b089cb5ff57` + reviewed working diff | Streamlined-sidebar handoff in the merged universal ledger | APPROVE. PRs #1112, #1116, and #1117 landed the universal-ledger baseline, two preserved UI tasks, and credential/hook hardening; this follow-up preserves that current-main schema and adds only the still-missing sidebar delivery as collision-free `#066`. The queue remains gap-free and every recommended ID has one open row. Sidebar implementation stays recoverable from remote branch `origin/codex/sidebar-test-fix-20260723` at full commit `cd54e68fbf7b07b5dffe3220e36af2caa528da54` until its local/browser/build/hosted merge proof is complete. | Focused sidebar/favourites tests previously passed 18/18 on the recorded implementation commit; this ledger follow-up passed Prettier, remote-ref recovery, 36-row contiguous-order/46-open-ID/next-ID invariants, and `git diff --check`. The broader reconciled precursor passed `verify:cheap` (365 files, 3,242 passed/1 skipped); fresh exact-head hosted CI is required on PR #1114. No OpenAI, Supabase, Railway, deployment, live-app, credential, or production-data action ran. | | 2026-07-24 | `codex/universal-ledger-main-final` | `527988c2ccabc98b4d0673d33360c971df65fa0e` + reviewed working diff | Current-main universal-ledger reconciliation after PR #1106 superseded PR #1109 | READY. Kept PR #1106's current-main ledger and IDs, carried forward only non-duplicate recommended work from the superseded branch, and fixed the confirmed SessionStart empty-open defect. Every active recommendation now has a durable open ID; exposed-GitHub-token containment is the first A1 item; the Safety Plan privacy contract, absent-relevance fail-closed rule, stranded-upload recovery, threshold-conflict design, catalogue-toolbar convergence, and Current Clinical Work brief are retained without duplicating #1106's legal/config/release/staging/seed packages. Resolved #014/#034 claims stay archived, and PR #1110's scheduled-diagnostics priority remains intact. Highest residual risk is manual queue/open-table drift. | Empty-open fixture and real-ledger Bash execution passed; 33 contiguous recommendations reference tracked open IDs; 43 open items; no duplicate queued IDs; `docs:check-links`, `docs:check-index`, `docs:check-scripts`, `check:skills`, Prettier, and `git diff --check` passed. No OpenAI, Supabase, Railway, production, deployment, live-app, or credential action. | From ac0d4305478a0bc8fef03894b78ec5911912c08a Mon Sep 17 00:00:00 2001 From: BigSimmo <87357024+BigSimmo@users.noreply.github.com> Date: Fri, 24 Jul 2026 12:25:26 +0800 Subject: [PATCH 05/12] fix: gate clinical notes table evidence --- .../answer-result-surface.tsx | 1 + .../clinical-dashboard/evidence-panels.tsx | 26 ++++--- tests/visual-evidence-tabs.dom.test.tsx | 75 +++++++++++++++++++ 3 files changed, 93 insertions(+), 9 deletions(-) diff --git a/src/components/clinical-dashboard/answer-result-surface.tsx b/src/components/clinical-dashboard/answer-result-surface.tsx index 6051ac0e0..644bda03b 100644 --- a/src/components/clinical-dashboard/answer-result-surface.tsx +++ b/src/components/clinical-dashboard/answer-result-surface.tsx @@ -291,6 +291,7 @@ function StagedAnswerResultSurfaceImpl({ > item.accessibleTableMarkdown || item.tableRows?.length, - ).length; -} - function clinicalNotesRowsForTab( sections: ClinicalDetailSection[], tab: ClinicalNotesTabId, @@ -605,6 +599,7 @@ export function clinicalNotesDisplayCountForAnswer(answer: RagAnswer, viewMode: export function ClinicalNotesChecklistPanel({ answer, + visualEvidence, viewMode, evidenceMapRows, sourceLinks = [], @@ -614,6 +609,7 @@ export function ClinicalNotesChecklistPanel({ onOpenTables, }: { answer: RagAnswer; + visualEvidence: VisualEvidenceCard[]; viewMode: AnswerViewMode; evidenceMapRows: AnswerEvidenceMapRow[]; sourceLinks?: SourceLink[]; @@ -622,7 +618,12 @@ export function ClinicalNotesChecklistPanel({ onCopy: () => void; onOpenTables?: () => void; }) { - const detailSections = clinicalNotesDetailSectionsForAnswer(answer, viewMode); + const renderableAnswer: RagAnswer = { + ...answer, + visualEvidence, + smartPanel: answer.smartPanel ? { ...answer.smartPanel, visualEvidence } : answer.smartPanel, + }; + const detailSections = clinicalNotesDetailSectionsForAnswer(renderableAnswer, viewMode); const tabs = clinicalNotesAvailableTabs(detailSections); const defaultTab = tabs.find((tab) => tab.id === "actions")?.id ?? tabs[0]?.id ?? "actions"; const [requestedTab, setRequestedTab] = useState(defaultTab); @@ -632,14 +633,21 @@ export function ClinicalNotesChecklistPanel({ const notesPanelId = `${tabBaseId}-panel`; const activeTab = tabs.some((tab) => tab.id === requestedTab) ? requestedTab : defaultTab; const rows = clinicalNotesRowsForTab(detailSections, activeTab, sourceLinks, bestSource); - const tableEvidenceCount = clinicalNotesTableEvidenceCount(answer); + const tableEvidenceCount = visualEvidence.filter( + (item) => item.accessibleTableMarkdown || item.tableRows?.length, + ).length; const [added, setAdded] = useState(false); const warningRows = clinicalNotesRowsForTab(detailSections, "safety", sourceLinks, bestSource); const warningCount = warningRows.filter((row) => row.tone === "warn").length || warningRows.length; if (!tabs.length || rows.length === 0) { return ( - + ); } diff --git a/tests/visual-evidence-tabs.dom.test.tsx b/tests/visual-evidence-tabs.dom.test.tsx index 9db75dbec..ffb974c95 100644 --- a/tests/visual-evidence-tabs.dom.test.tsx +++ b/tests/visual-evidence-tabs.dom.test.tsx @@ -2,6 +2,7 @@ import { render, screen, within } from "@testing-library/react"; import userEvent from "@testing-library/user-event"; import { describe, expect, it, vi } from "vitest"; +import { ClinicalNotesChecklistPanel } from "@/components/clinical-dashboard/evidence-panels"; import { MobileEvidenceSheetContent } from "@/components/clinical-dashboard/visual-evidence"; import type { AnswerRenderModel } from "@/lib/answer-render-policy"; import type { RagAnswer } from "@/lib/types"; @@ -137,3 +138,77 @@ describe("MobileEvidenceSheetContent tabs (jsdom)", () => { expect(claims).toHaveAttribute("tabindex", "-1"); }); }); + +describe("ClinicalNotesChecklistPanel visual-evidence boundary (jsdom)", () => { + it("does not expose raw table evidence suppressed by the render model", () => { + const answerWithRawTable: RagAnswer = { + ...answer, + answer: "Monitor renal function and escalate review for vomiting, dehydration, tremor, confusion, or ataxia.", + queryClass: "table_threshold", + responseMode: "threshold_table", + citations: [ + { + chunk_id: "chunk-1", + document_id: "doc-1", + title: "Lithium source", + file_name: "lithium.pdf", + page_number: 1, + chunk_index: 0, + }, + ], + answerSections: [ + { + heading: "Monitoring", + body: "Check lithium level, renal function, thyroid function, calcium, and interacting medicines.", + citation_chunk_ids: ["chunk-1"], + kind: "monitoring_timing", + supportLevel: "direct", + }, + { + heading: "Escalation", + body: "Escalate review for vomiting, dehydration, tremor, confusion, or ataxia.", + citation_chunk_ids: ["chunk-1"], + kind: "escalation_risk", + supportLevel: "direct", + }, + ], + visualEvidence: [ + { + id: "raw-table", + document_id: "doc-1", + image_id: "image-1", + source_chunk_id: "chunk-1", + title: "Raw threshold table", + file_name: "raw-table.pdf", + page_number: 1, + chunk_index: 0, + caption: "Raw table", + tableColumns: ["Threshold", "Action"], + tableRows: [ + ["0.49", "Withhold"], + ["1.0", "Monitor"], + ], + signed_url_endpoint: "/api/images/image-1", + viewer_href: "/documents/doc-1?page=1", + }, + ], + }; + + render( + , + ); + + expect(screen.queryByRole("button", { name: "Tables" })).not.toBeInTheDocument(); + expect(screen.queryByText("0.49")).not.toBeInTheDocument(); + expect(screen.queryByText("Withhold")).not.toBeInTheDocument(); + }); +}); From 5aa98cf170461a5fba5a27fc8b4b13ab6241fbd3 Mon Sep 17 00:00:00 2001 From: BigSimmo <87357024+BigSimmo@users.noreply.github.com> Date: Fri, 24 Jul 2026 12:39:04 +0800 Subject: [PATCH 06/12] docs: record clinical notes review fix --- docs/branch-review-ledger.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/branch-review-ledger.md b/docs/branch-review-ledger.md index c661edfd6..e28eabf59 100644 --- a/docs/branch-review-ledger.md +++ b/docs/branch-review-ledger.md @@ -20,7 +20,7 @@ Use this ledger to prevent repeated branch and PR reviews when the reviewed HEAD | Date | Branch or ref | Reviewed HEAD | Scope | Outcome | Checks | | ---------- | -------------------------------------------------------- | ---------------------------------------- | ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 2026-07-24 | `codex/answer-relevance-fail-closed` | `0072226794de9681b5f76bc24a19b4876e002c90` | Missing answer-relevance metadata across render policy and live dashboard grounding | APPROVE after resolving one review P2. A shared `isAnswerSourceBacked` predicate now requires explicit `isSourceBacked: true`; missing or explicitly negative relevance cannot retain high render trust, a grounded dashboard state, or visual/table evidence. Explicitly source-backed answers preserve supported behavior. Retrieval, ranking, generation, source selection and stored data are unchanged. Highest residual risk is deliberate compatibility tightening for older answer payloads without relevance metadata; they render low-trust and expose review sources rather than richer evidence blocks. | Initial red policy proof failed with `high`; the review follow-up red proof exposed retained visual evidence and then passed after the source-backed visual gate. Focused render/provenance/clinical-safety tests passed 37/37 and typecheck passed after the final code change. Offline RAG passed 21 suites/308 tests and 36/36 fixtures; production-readiness was READY against `Clinical KB Database` using the configured environment read-only; `verify:pr-local` passed runtime, formatting, lint, typecheck, all 366 test files (3,254 passed/1 skipped), production build (1,677 pages), client-bundle secret scan and RAG fixture validation; `verify:ui` passed all 267 Chromium scenarios against the isolated production build. An earlier `verify:cheap` attempt passed all non-test gates and 3,253 tests before known issue #067 timed out; its isolated suite passed 5/5, and the subsequent full PR-local suite passed. Fresh hosted checks are required at the final head. No live RAG, OpenAI request, Supabase mutation, Railway action, production data operation or deployment ran. | +| 2026-07-24 | `codex/answer-relevance-fail-closed` | `ac0d4305478a0bc8fef03894b78ec5911912c08a` | Missing answer-relevance metadata across render policy and live dashboard grounding | APPROVE after resolving two review P2s. A shared `isAnswerSourceBacked` predicate now requires explicit `isSourceBacked: true`; missing or explicitly negative relevance cannot retain high render trust, a grounded dashboard state, visual/table evidence, or a clinical-notes table bypass. Explicitly source-backed answers preserve supported behavior. Retrieval, ranking, generation, source selection and stored data are unchanged. Highest residual risk is deliberate compatibility tightening for older answer payloads without relevance metadata; they render low-trust and expose review sources rather than richer evidence blocks. | Initial red policy proof failed with `high`; two later red proofs exposed retained visual evidence and the clinical-notes raw-table affordance, then passed after both render-model gates. Focused render/provenance/clinical-safety tests passed 37/37; the focused DOM/policy pair passed 31/31. Offline RAG passed 21 suites/308 tests and 36/36 fixtures; production-readiness was READY against `Clinical KB Database` read-only; `verify:pr-local` passed runtime, formatting, lint, typecheck, all 366 test files (3,254 passed/1 skipped), production build (1,677 pages), client-bundle secret scan and RAG fixture validation; the earlier local `verify:ui` passed 267/267. After the final UI fix, `verify:cheap` again passed all 20 non-test gates, lint, typecheck and 3,254 tests, with only tracked issue #067 timing out under machine load; its isolated retry also exceeded the same 30-second limit and was not repeated. Fresh exact-head hosted checks are required. No live RAG, OpenAI request, Supabase mutation, Railway action, production data operation or deployment ran. | | 2026-07-24 | `codex/safety-plan-no-patient-data-contract` | `b94987c94537f3114a3429848fa908bdecd1d80a` | Safety Plan Generator identifier, local-state, copy, print, privacy-notice and PIA contract | APPROVE. No P0-P2 finding. The patient name/initials field is removed; the builder now asks for identifier-free minimum content, retains working state only in the mounted React component, and makes clipboard/print/PDF export an explicit handling boundary. The PIA and product privacy copy distinguish this local-only tool from provider-backed questions. Highest residual risk is outside Clinical KB: users must handle exported copies under an approved clinical-record process, which the UI now states at the export controls. | Privacy/component DOM 3/3 plus updated privacy-copy 2/2; focused Chromium copy/print/no-fetch-or-XHR 1/1; `verify:cheap` passed all 21 gates, 366 files and 3,245 tests with 1 skip; production-readiness READY using the existing canonical environment without a provider call; production build and client-bundle secret scan passed; offline RAG fixture/manifest 36 cases/21 suites passed. `verify:pr-local` passed runtime, formatting, lint and typecheck, then stopped on the unrelated load-sensitive `reconciliation-preflight` 30-second timeout; that test passed 5/5 isolated and the preceding full suite passed, so the unchanged five-minute gate was not retried. No Supabase, OpenAI, Railway, live RAG, production data or deployment action ran. | | 2026-07-24 | PR #1114 / `codex/universal-ledger-main-followup-20260724` | `eedbe594a89eaa0aad8a517ee8808b089cb5ff57` + reviewed working diff | Streamlined-sidebar handoff in the merged universal ledger | APPROVE. PRs #1112, #1116, and #1117 landed the universal-ledger baseline, two preserved UI tasks, and credential/hook hardening; this follow-up preserves that current-main schema and adds only the still-missing sidebar delivery as collision-free `#066`. The queue remains gap-free and every recommended ID has one open row. Sidebar implementation stays recoverable from remote branch `origin/codex/sidebar-test-fix-20260723` at full commit `cd54e68fbf7b07b5dffe3220e36af2caa528da54` until its local/browser/build/hosted merge proof is complete. | Focused sidebar/favourites tests previously passed 18/18 on the recorded implementation commit; this ledger follow-up passed Prettier, remote-ref recovery, 36-row contiguous-order/46-open-ID/next-ID invariants, and `git diff --check`. The broader reconciled precursor passed `verify:cheap` (365 files, 3,242 passed/1 skipped); fresh exact-head hosted CI is required on PR #1114. No OpenAI, Supabase, Railway, deployment, live-app, credential, or production-data action ran. | | 2026-07-24 | `codex/universal-ledger-main-final` | `527988c2ccabc98b4d0673d33360c971df65fa0e` + reviewed working diff | Current-main universal-ledger reconciliation after PR #1106 superseded PR #1109 | READY. Kept PR #1106's current-main ledger and IDs, carried forward only non-duplicate recommended work from the superseded branch, and fixed the confirmed SessionStart empty-open defect. Every active recommendation now has a durable open ID; exposed-GitHub-token containment is the first A1 item; the Safety Plan privacy contract, absent-relevance fail-closed rule, stranded-upload recovery, threshold-conflict design, catalogue-toolbar convergence, and Current Clinical Work brief are retained without duplicating #1106's legal/config/release/staging/seed packages. Resolved #014/#034 claims stay archived, and PR #1110's scheduled-diagnostics priority remains intact. Highest residual risk is manual queue/open-table drift. | Empty-open fixture and real-ledger Bash execution passed; 33 contiguous recommendations reference tracked open IDs; 43 open items; no duplicate queued IDs; `docs:check-links`, `docs:check-index`, `docs:check-scripts`, `check:skills`, Prettier, and `git diff --check` passed. No OpenAI, Supabase, Railway, production, deployment, live-app, or credential action. | From ac007733e0b01d3bf6eb1507cbb834d491e8e151 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Fri, 24 Jul 2026 04:43:36 +0000 Subject: [PATCH 07/12] fix(clinical-notes): gate sections and quotes on source-backed trust MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit When relevance.isSourceBacked is false, clear answerSections, quoteCards, and smartPanel quotes before Clinical Notes renders — not only visualEvidence. Addresses Codex review on PR #1125. Co-authored-by: BigSimmo --- .../clinical-dashboard/evidence-panels.tsx | 40 ++++++++++++--- tests/visual-evidence-tabs.dom.test.tsx | 50 +++++++++++++++++++ 2 files changed, 82 insertions(+), 8 deletions(-) diff --git a/src/components/clinical-dashboard/evidence-panels.tsx b/src/components/clinical-dashboard/evidence-panels.tsx index acbbf6eca..62075c259 100644 --- a/src/components/clinical-dashboard/evidence-panels.tsx +++ b/src/components/clinical-dashboard/evidence-panels.tsx @@ -62,7 +62,7 @@ import { toneSuccess, toneWarning, } from "@/components/ui-primitives"; -import { type AnswerRenderModel, type SourceLink } from "@/lib/answer-render-policy"; +import { isAnswerSourceBacked, type AnswerRenderModel, type SourceLink } from "@/lib/answer-render-policy"; import { documentCitationHref, formatCitationLabel, formatCompactCitationLabel } from "@/lib/citations"; import { extractSafetyFindings, @@ -568,6 +568,32 @@ function clinicalNotesAvailableTabs(sections: ClinicalDetailSection[]) { .filter((tab) => tab.count > 0); } +/** + * Align clinical-notes inputs with the fail-closed render model: when an answer + * is not explicitly source-backed, strip structured clinical payloads so the + * notes sheet cannot reconstruct actionable monitoring/escalation content from + * untrusted `answerSections` / quotes (visual evidence is passed separately). + */ +function trustGatedAnswerForClinicalNotes( + answer: RagAnswer, + visualEvidence: VisualEvidenceCard[] = answer.visualEvidence ?? [], +): RagAnswer { + if (isAnswerSourceBacked(answer)) { + return { + ...answer, + visualEvidence, + smartPanel: answer.smartPanel ? { ...answer.smartPanel, visualEvidence } : answer.smartPanel, + }; + } + return { + ...answer, + answerSections: [], + quoteCards: [], + visualEvidence, + smartPanel: answer.smartPanel ? { ...answer.smartPanel, visualEvidence, quotes: [] } : answer.smartPanel, + }; +} + /** * Builds the non-empty clinical detail sections used by the clinical notes view. * @@ -592,7 +618,9 @@ function clinicalNotesDetailSectionsForAnswer(answer: RagAnswer, viewMode: Answe } export function clinicalNotesDisplayCountForAnswer(answer: RagAnswer, viewMode: AnswerViewMode, fallback: number) { - const tabs = clinicalNotesAvailableTabs(clinicalNotesDetailSectionsForAnswer(answer, viewMode)); + const tabs = clinicalNotesAvailableTabs( + clinicalNotesDetailSectionsForAnswer(trustGatedAnswerForClinicalNotes(answer), viewMode), + ); const largestTabCount = tabs.reduce((largest, tab) => Math.max(largest, tab.count), 0); return Math.max(1, largestTabCount || fallback); } @@ -618,11 +646,7 @@ export function ClinicalNotesChecklistPanel({ onCopy: () => void; onOpenTables?: () => void; }) { - const renderableAnswer: RagAnswer = { - ...answer, - visualEvidence, - smartPanel: answer.smartPanel ? { ...answer.smartPanel, visualEvidence } : answer.smartPanel, - }; + const renderableAnswer = trustGatedAnswerForClinicalNotes(answer, visualEvidence); const detailSections = clinicalNotesDetailSectionsForAnswer(renderableAnswer, viewMode); const tabs = clinicalNotesAvailableTabs(detailSections); const defaultTab = tabs.find((tab) => tab.id === "actions")?.id ?? tabs[0]?.id ?? "actions"; @@ -1035,7 +1059,7 @@ export function evidenceTabCount({ } export function clinicalNotesCount(answer: RagAnswer) { - return buildHighYieldClinicalOutputSections(answer).filter((section) => + return buildHighYieldClinicalOutputSections(trustGatedAnswerForClinicalNotes(answer)).filter((section) => ["action", "escalation", "thresholds", "cautions", "monitoring", "medication", "source-gap"].includes(section.id), ).length; } diff --git a/tests/visual-evidence-tabs.dom.test.tsx b/tests/visual-evidence-tabs.dom.test.tsx index ffb974c95..937c2d5a9 100644 --- a/tests/visual-evidence-tabs.dom.test.tsx +++ b/tests/visual-evidence-tabs.dom.test.tsx @@ -211,4 +211,54 @@ describe("ClinicalNotesChecklistPanel visual-evidence boundary (jsdom)", () => { expect(screen.queryByText("0.49")).not.toBeInTheDocument(); expect(screen.queryByText("Withhold")).not.toBeInTheDocument(); }); + + it("does not reconstruct clinical-notes sections from untrusted answerSections", () => { + const untrusted: RagAnswer = { + ...answer, + grounded: true, + // Missing relevance.isSourceBacked → fail closed for structured clinical UI. + answerSections: [ + { + heading: "Monitoring", + body: "Check lithium level every 3 months when stable.", + citation_chunk_ids: ["chunk-1"], + kind: "monitoring_timing", + supportLevel: "direct", + }, + { + heading: "Escalation", + body: "Escalate for tremor, confusion, or ataxia.", + citation_chunk_ids: ["chunk-1"], + kind: "escalation_risk", + supportLevel: "direct", + }, + ], + quoteCards: [ + { + chunk_id: "chunk-1", + document_id: "doc-1", + title: "Lithium source", + file_name: "lithium.pdf", + page_number: 1, + quote: "Check lithium level every 3 months when stable.", + }, + ], + }; + + render( + , + ); + + expect(screen.queryByText("Check lithium level every 3 months when stable.")).not.toBeInTheDocument(); + expect(screen.queryByText("Escalate for tremor, confusion, or ataxia.")).not.toBeInTheDocument(); + }); }); From 41ad7a3ad924b3a7920243da19ed3776bb382a88 Mon Sep 17 00:00:00 2001 From: BigSimmo <87357024+BigSimmo@users.noreply.github.com> Date: Fri, 24 Jul 2026 12:51:51 +0800 Subject: [PATCH 08/12] test: complete clinical notes quote fixture --- tests/visual-evidence-tabs.dom.test.tsx | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/visual-evidence-tabs.dom.test.tsx b/tests/visual-evidence-tabs.dom.test.tsx index 937c2d5a9..e697fc76f 100644 --- a/tests/visual-evidence-tabs.dom.test.tsx +++ b/tests/visual-evidence-tabs.dom.test.tsx @@ -240,6 +240,8 @@ describe("ClinicalNotesChecklistPanel visual-evidence boundary (jsdom)", () => { title: "Lithium source", file_name: "lithium.pdf", page_number: 1, + chunk_index: 0, + section_heading: "Monitoring", quote: "Check lithium level every 3 months when stable.", }, ], From d0df4f9160e7b0fb057bf16ab45c8fbe571d1862 Mon Sep 17 00:00:00 2001 From: BigSimmo <87357024+BigSimmo@users.noreply.github.com> Date: Fri, 24 Jul 2026 13:02:18 +0800 Subject: [PATCH 09/12] fix: clear comparison data in untrusted clinical notes Strip documentBreakdown and comparison metadata when relevance is not source-backed so Clinical Notes cannot rebuild comparison tables via ClinicalOutputPanel fallback. Co-authored-by: Cursor --- .../clinical-dashboard/evidence-panels.tsx | 8 ++- tests/visual-evidence-tabs.dom.test.tsx | 49 +++++++++++++++++++ 2 files changed, 55 insertions(+), 2 deletions(-) diff --git a/src/components/clinical-dashboard/evidence-panels.tsx b/src/components/clinical-dashboard/evidence-panels.tsx index 62075c259..8f34b3363 100644 --- a/src/components/clinical-dashboard/evidence-panels.tsx +++ b/src/components/clinical-dashboard/evidence-panels.tsx @@ -571,8 +571,9 @@ function clinicalNotesAvailableTabs(sections: ClinicalDetailSection[]) { /** * Align clinical-notes inputs with the fail-closed render model: when an answer * is not explicitly source-backed, strip structured clinical payloads so the - * notes sheet cannot reconstruct actionable monitoring/escalation content from - * untrusted `answerSections` / quotes (visual evidence is passed separately). + * notes sheet cannot reconstruct actionable monitoring/escalation/comparison + * content from untrusted sections, quotes, or documentBreakdown (visual + * evidence is passed separately). */ function trustGatedAnswerForClinicalNotes( answer: RagAnswer, @@ -589,6 +590,9 @@ function trustGatedAnswerForClinicalNotes( ...answer, answerSections: [], quoteCards: [], + documentBreakdown: [], + comparisonMatrix: undefined, + comparisonEvaluationState: undefined, visualEvidence, smartPanel: answer.smartPanel ? { ...answer.smartPanel, visualEvidence, quotes: [] } : answer.smartPanel, }; diff --git a/tests/visual-evidence-tabs.dom.test.tsx b/tests/visual-evidence-tabs.dom.test.tsx index e697fc76f..f1da1682b 100644 --- a/tests/visual-evidence-tabs.dom.test.tsx +++ b/tests/visual-evidence-tabs.dom.test.tsx @@ -263,4 +263,53 @@ describe("ClinicalNotesChecklistPanel visual-evidence boundary (jsdom)", () => { expect(screen.queryByText("Check lithium level every 3 months when stable.")).not.toBeInTheDocument(); expect(screen.queryByText("Escalate for tremor, confusion, or ataxia.")).not.toBeInTheDocument(); }); + + it("does not render comparison tables from untrusted documentBreakdown", () => { + const comparisonAnswer: RagAnswer = { + ...answer, + grounded: true, + responseMode: "comparison_matrix", + queryClass: "comparison", + documentBreakdown: [ + { + document_id: "doc-a", + title: "Guideline A", + file_name: "a.pdf", + top_similarity: 0.9, + source_strength: "strong", + source_count: 1, + quote_count: 1, + pages: [1], + best_quote: "Guideline A prefers weekly lithium monitoring when unstable.", + }, + { + document_id: "doc-b", + title: "Guideline B", + file_name: "b.pdf", + top_similarity: 0.8, + source_strength: "moderate", + source_count: 1, + quote_count: 1, + pages: [2], + best_quote: "Guideline B prefers monthly lithium monitoring when stable.", + }, + ], + }; + + render( + , + ); + + expect(screen.queryByText("Guideline A prefers weekly lithium monitoring when unstable.")).not.toBeInTheDocument(); + expect(screen.queryByText("Guideline B prefers monthly lithium monitoring when stable.")).not.toBeInTheDocument(); + }); }); From 8d9fb2408f13e305138749655214baa0020fcfd4 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Fri, 24 Jul 2026 05:05:20 +0000 Subject: [PATCH 10/12] test: harden untrusted comparison clinical-notes regression Assert the comparison-detail caption and matrix values stay hidden when relevance is missing, even if documentBreakdown and comparisonMatrix are present. --- tests/visual-evidence-tabs.dom.test.tsx | 32 +++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/tests/visual-evidence-tabs.dom.test.tsx b/tests/visual-evidence-tabs.dom.test.tsx index f1da1682b..6855c6821 100644 --- a/tests/visual-evidence-tabs.dom.test.tsx +++ b/tests/visual-evidence-tabs.dom.test.tsx @@ -268,6 +268,8 @@ describe("ClinicalNotesChecklistPanel visual-evidence boundary (jsdom)", () => { const comparisonAnswer: RagAnswer = { ...answer, grounded: true, + // Missing relevance.isSourceBacked → fail closed; documentBreakdown/comparison + // metadata must not rebuild ClinicalOutputPanel comparison-detail tables. responseMode: "comparison_matrix", queryClass: "comparison", documentBreakdown: [ @@ -294,6 +296,33 @@ describe("ClinicalNotesChecklistPanel visual-evidence boundary (jsdom)", () => { best_quote: "Guideline B prefers monthly lithium monitoring when stable.", }, ], + comparisonMatrix: { + documents: [ + { documentId: "doc-a", title: "Guideline A", fileName: "a.pdf" }, + { documentId: "doc-b", title: "Guideline B", fileName: "b.pdf" }, + ], + rows: [ + { + parameter: "Monitoring interval", + status: "conflict", + entries: [ + { + documentId: "doc-a", + chunkIds: ["chunk-a"], + value: "weekly when unstable", + qualifiers: [], + }, + { + documentId: "doc-b", + chunkIds: ["chunk-b"], + value: "monthly when stable", + qualifiers: [], + }, + ], + }, + ], + }, + comparisonEvaluationState: "evaluated", }; render( @@ -309,7 +338,10 @@ describe("ClinicalNotesChecklistPanel visual-evidence boundary (jsdom)", () => { />, ); + expect(screen.queryByText("Clinical comparison detail")).not.toBeInTheDocument(); expect(screen.queryByText("Guideline A prefers weekly lithium monitoring when unstable.")).not.toBeInTheDocument(); expect(screen.queryByText("Guideline B prefers monthly lithium monitoring when stable.")).not.toBeInTheDocument(); + expect(screen.queryByText("weekly when unstable")).not.toBeInTheDocument(); + expect(screen.queryByText("monthly when stable")).not.toBeInTheDocument(); }); }); From 3cc50e05de2ef77b6b9d3fbbb504049a5dedefa3 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Fri, 24 Jul 2026 05:05:39 +0000 Subject: [PATCH 11/12] docs: record comparison clinical-notes review disposition Ledger the cleared documentBreakdown/comparison-metadata fail-closed fix and hardened jsdom regression for PR #1125. --- docs/branch-review-ledger.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/branch-review-ledger.md b/docs/branch-review-ledger.md index e28eabf59..172bb3427 100644 --- a/docs/branch-review-ledger.md +++ b/docs/branch-review-ledger.md @@ -20,6 +20,7 @@ Use this ledger to prevent repeated branch and PR reviews when the reviewed HEAD | Date | Branch or ref | Reviewed HEAD | Scope | Outcome | Checks | | ---------- | -------------------------------------------------------- | ---------------------------------------- | ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-07-24 | PR #1125 / `codex/answer-relevance-fail-closed` | `8d9fb240` | Follow-up: clear comparison/`documentBreakdown` in untrusted clinical notes | APPROVE for the scoped P2. `trustGatedAnswerForClinicalNotes` now clears `documentBreakdown`, `comparisonMatrix`, and `comparisonEvaluationState` when relevance is not source-backed, so Clinical Notes → ClinicalOutputPanel cannot rebuild comparison-detail tables from raw `best_quote` values. Prior visual/section/quote gates remain. Residual risk is still deliberate low-trust rendering for legacy payloads without `isSourceBacked: true`. | Focused jsdom/policy regressions: `tests/visual-evidence-tabs.dom.test.tsx` 5/5 after hardening the comparison case (caption + matrix values absent). Thread disposition posted and resolved. No live RAG/OpenAI/Supabase mutation. | | 2026-07-24 | `codex/answer-relevance-fail-closed` | `ac0d4305478a0bc8fef03894b78ec5911912c08a` | Missing answer-relevance metadata across render policy and live dashboard grounding | APPROVE after resolving two review P2s. A shared `isAnswerSourceBacked` predicate now requires explicit `isSourceBacked: true`; missing or explicitly negative relevance cannot retain high render trust, a grounded dashboard state, visual/table evidence, or a clinical-notes table bypass. Explicitly source-backed answers preserve supported behavior. Retrieval, ranking, generation, source selection and stored data are unchanged. Highest residual risk is deliberate compatibility tightening for older answer payloads without relevance metadata; they render low-trust and expose review sources rather than richer evidence blocks. | Initial red policy proof failed with `high`; two later red proofs exposed retained visual evidence and the clinical-notes raw-table affordance, then passed after both render-model gates. Focused render/provenance/clinical-safety tests passed 37/37; the focused DOM/policy pair passed 31/31. Offline RAG passed 21 suites/308 tests and 36/36 fixtures; production-readiness was READY against `Clinical KB Database` read-only; `verify:pr-local` passed runtime, formatting, lint, typecheck, all 366 test files (3,254 passed/1 skipped), production build (1,677 pages), client-bundle secret scan and RAG fixture validation; the earlier local `verify:ui` passed 267/267. After the final UI fix, `verify:cheap` again passed all 20 non-test gates, lint, typecheck and 3,254 tests, with only tracked issue #067 timing out under machine load; its isolated retry also exceeded the same 30-second limit and was not repeated. Fresh exact-head hosted checks are required. No live RAG, OpenAI request, Supabase mutation, Railway action, production data operation or deployment ran. | | 2026-07-24 | `codex/safety-plan-no-patient-data-contract` | `b94987c94537f3114a3429848fa908bdecd1d80a` | Safety Plan Generator identifier, local-state, copy, print, privacy-notice and PIA contract | APPROVE. No P0-P2 finding. The patient name/initials field is removed; the builder now asks for identifier-free minimum content, retains working state only in the mounted React component, and makes clipboard/print/PDF export an explicit handling boundary. The PIA and product privacy copy distinguish this local-only tool from provider-backed questions. Highest residual risk is outside Clinical KB: users must handle exported copies under an approved clinical-record process, which the UI now states at the export controls. | Privacy/component DOM 3/3 plus updated privacy-copy 2/2; focused Chromium copy/print/no-fetch-or-XHR 1/1; `verify:cheap` passed all 21 gates, 366 files and 3,245 tests with 1 skip; production-readiness READY using the existing canonical environment without a provider call; production build and client-bundle secret scan passed; offline RAG fixture/manifest 36 cases/21 suites passed. `verify:pr-local` passed runtime, formatting, lint and typecheck, then stopped on the unrelated load-sensitive `reconciliation-preflight` 30-second timeout; that test passed 5/5 isolated and the preceding full suite passed, so the unchanged five-minute gate was not retried. No Supabase, OpenAI, Railway, live RAG, production data or deployment action ran. | | 2026-07-24 | PR #1114 / `codex/universal-ledger-main-followup-20260724` | `eedbe594a89eaa0aad8a517ee8808b089cb5ff57` + reviewed working diff | Streamlined-sidebar handoff in the merged universal ledger | APPROVE. PRs #1112, #1116, and #1117 landed the universal-ledger baseline, two preserved UI tasks, and credential/hook hardening; this follow-up preserves that current-main schema and adds only the still-missing sidebar delivery as collision-free `#066`. The queue remains gap-free and every recommended ID has one open row. Sidebar implementation stays recoverable from remote branch `origin/codex/sidebar-test-fix-20260723` at full commit `cd54e68fbf7b07b5dffe3220e36af2caa528da54` until its local/browser/build/hosted merge proof is complete. | Focused sidebar/favourites tests previously passed 18/18 on the recorded implementation commit; this ledger follow-up passed Prettier, remote-ref recovery, 36-row contiguous-order/46-open-ID/next-ID invariants, and `git diff --check`. The broader reconciled precursor passed `verify:cheap` (365 files, 3,242 passed/1 skipped); fresh exact-head hosted CI is required on PR #1114. No OpenAI, Supabase, Railway, deployment, live-app, credential, or production-data action ran. | From 181219fa070abac3470ea0658493c62797d469b6 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Fri, 24 Jul 2026 05:06:05 +0000 Subject: [PATCH 12/12] docs: expand comparison follow-up ledger HEAD SHA Use the full reviewed commit hash for the PR #1125 comparison clear-out row. --- docs/branch-review-ledger.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/branch-review-ledger.md b/docs/branch-review-ledger.md index 172bb3427..9f2bee673 100644 --- a/docs/branch-review-ledger.md +++ b/docs/branch-review-ledger.md @@ -20,7 +20,7 @@ Use this ledger to prevent repeated branch and PR reviews when the reviewed HEAD | Date | Branch or ref | Reviewed HEAD | Scope | Outcome | Checks | | ---------- | -------------------------------------------------------- | ---------------------------------------- | ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 2026-07-24 | PR #1125 / `codex/answer-relevance-fail-closed` | `8d9fb240` | Follow-up: clear comparison/`documentBreakdown` in untrusted clinical notes | APPROVE for the scoped P2. `trustGatedAnswerForClinicalNotes` now clears `documentBreakdown`, `comparisonMatrix`, and `comparisonEvaluationState` when relevance is not source-backed, so Clinical Notes → ClinicalOutputPanel cannot rebuild comparison-detail tables from raw `best_quote` values. Prior visual/section/quote gates remain. Residual risk is still deliberate low-trust rendering for legacy payloads without `isSourceBacked: true`. | Focused jsdom/policy regressions: `tests/visual-evidence-tabs.dom.test.tsx` 5/5 after hardening the comparison case (caption + matrix values absent). Thread disposition posted and resolved. No live RAG/OpenAI/Supabase mutation. | +| 2026-07-24 | PR #1125 / `codex/answer-relevance-fail-closed` | `8d9fb2408f13e305138749655214baa0020fcfd4` | Follow-up: clear comparison/`documentBreakdown` in untrusted clinical notes | APPROVE for the scoped P2. `trustGatedAnswerForClinicalNotes` now clears `documentBreakdown`, `comparisonMatrix`, and `comparisonEvaluationState` when relevance is not source-backed, so Clinical Notes → ClinicalOutputPanel cannot rebuild comparison-detail tables from raw `best_quote` values. Prior visual/section/quote gates remain. Residual risk is still deliberate low-trust rendering for legacy payloads without `isSourceBacked: true`. | Focused jsdom/policy regressions: `tests/visual-evidence-tabs.dom.test.tsx` 5/5 after hardening the comparison case (caption + matrix values absent). Thread disposition posted and resolved. No live RAG/OpenAI/Supabase mutation. | | 2026-07-24 | `codex/answer-relevance-fail-closed` | `ac0d4305478a0bc8fef03894b78ec5911912c08a` | Missing answer-relevance metadata across render policy and live dashboard grounding | APPROVE after resolving two review P2s. A shared `isAnswerSourceBacked` predicate now requires explicit `isSourceBacked: true`; missing or explicitly negative relevance cannot retain high render trust, a grounded dashboard state, visual/table evidence, or a clinical-notes table bypass. Explicitly source-backed answers preserve supported behavior. Retrieval, ranking, generation, source selection and stored data are unchanged. Highest residual risk is deliberate compatibility tightening for older answer payloads without relevance metadata; they render low-trust and expose review sources rather than richer evidence blocks. | Initial red policy proof failed with `high`; two later red proofs exposed retained visual evidence and the clinical-notes raw-table affordance, then passed after both render-model gates. Focused render/provenance/clinical-safety tests passed 37/37; the focused DOM/policy pair passed 31/31. Offline RAG passed 21 suites/308 tests and 36/36 fixtures; production-readiness was READY against `Clinical KB Database` read-only; `verify:pr-local` passed runtime, formatting, lint, typecheck, all 366 test files (3,254 passed/1 skipped), production build (1,677 pages), client-bundle secret scan and RAG fixture validation; the earlier local `verify:ui` passed 267/267. After the final UI fix, `verify:cheap` again passed all 20 non-test gates, lint, typecheck and 3,254 tests, with only tracked issue #067 timing out under machine load; its isolated retry also exceeded the same 30-second limit and was not repeated. Fresh exact-head hosted checks are required. No live RAG, OpenAI request, Supabase mutation, Railway action, production data operation or deployment ran. | | 2026-07-24 | `codex/safety-plan-no-patient-data-contract` | `b94987c94537f3114a3429848fa908bdecd1d80a` | Safety Plan Generator identifier, local-state, copy, print, privacy-notice and PIA contract | APPROVE. No P0-P2 finding. The patient name/initials field is removed; the builder now asks for identifier-free minimum content, retains working state only in the mounted React component, and makes clipboard/print/PDF export an explicit handling boundary. The PIA and product privacy copy distinguish this local-only tool from provider-backed questions. Highest residual risk is outside Clinical KB: users must handle exported copies under an approved clinical-record process, which the UI now states at the export controls. | Privacy/component DOM 3/3 plus updated privacy-copy 2/2; focused Chromium copy/print/no-fetch-or-XHR 1/1; `verify:cheap` passed all 21 gates, 366 files and 3,245 tests with 1 skip; production-readiness READY using the existing canonical environment without a provider call; production build and client-bundle secret scan passed; offline RAG fixture/manifest 36 cases/21 suites passed. `verify:pr-local` passed runtime, formatting, lint and typecheck, then stopped on the unrelated load-sensitive `reconciliation-preflight` 30-second timeout; that test passed 5/5 isolated and the preceding full suite passed, so the unchanged five-minute gate was not retried. No Supabase, OpenAI, Railway, live RAG, production data or deployment action ran. | | 2026-07-24 | PR #1114 / `codex/universal-ledger-main-followup-20260724` | `eedbe594a89eaa0aad8a517ee8808b089cb5ff57` + reviewed working diff | Streamlined-sidebar handoff in the merged universal ledger | APPROVE. PRs #1112, #1116, and #1117 landed the universal-ledger baseline, two preserved UI tasks, and credential/hook hardening; this follow-up preserves that current-main schema and adds only the still-missing sidebar delivery as collision-free `#066`. The queue remains gap-free and every recommended ID has one open row. Sidebar implementation stays recoverable from remote branch `origin/codex/sidebar-test-fix-20260723` at full commit `cd54e68fbf7b07b5dffe3220e36af2caa528da54` until its local/browser/build/hosted merge proof is complete. | Focused sidebar/favourites tests previously passed 18/18 on the recorded implementation commit; this ledger follow-up passed Prettier, remote-ref recovery, 36-row contiguous-order/46-open-ID/next-ID invariants, and `git diff --check`. The broader reconciled precursor passed `verify:cheap` (365 files, 3,242 passed/1 skipped); fresh exact-head hosted CI is required on PR #1114. No OpenAI, Supabase, Railway, deployment, live-app, credential, or production-data action ran. |