feat(rag): Option A escalation rescue + monitoring evidence gate parity

docs/branch-review-ledger.md

@@ -673,3 +673,8 @@ Use this ledger to prevent repeated branch and PR reviews when the reviewed HEAD
 | 2026-07-21 | claude/clinical-kb-pwa-review-asi3wb (PR: E-3c PR-B short-circuit) | f7e6cbb + hardening commit | E-3c PR-B: pre-generation validated-extractive short-circuit for gate-passed routine procedural "What...process/include/required" queries (marker validated_routine_extractive_first), generalizing the LAI + blocked-recovery precedents via new rag-extractive-first.ts (3 predicates moved byte-verbatim, machine-verified; rag.ts 5029→4908 vs 5030 budget). Kills the run-#57 6x wasted-generation class. REVIEWS (both pre-push): rag-retrieval-reviewer APPROVE-WITH-NITS — move fidelity brace-diff verified byte-identical; confidence-gate skip PROVEN safe (passed-gate is a no-op in applyConfidenceGate; markers pairwise mutually exclusive); comparison false-positives blocked by unchanged classifier precedence; offline/source-only idempotent; zero retrieval/ranking/selection/threshold change; P2 = eval-only assertions (intent_coverage/artifact_leaks/expected-file) unverifiable offline for flip candidates (quality-nocc-document-support, quality-form-required-documentation, quality-discharge-documentation, quality-duress-pathway + rag-set siblings) → pre-merge BRANCH canary recommended and ADOPTED (offline-green + review-approved proven insufficient for this surface, 2026-07-20). clinical-governance-reviewer APPROVE-WITH-NITS — full gate-stack trace: nothing bypassed (same finalizeRagAnswerQuality, same citation scoping, numeric verification not fail-open, ungrounded-finalize defense at rag.ts:3694); P2 = pre-existing bare-cross-reference-with-overlap gap, NOT materially widened (new trigger anti-correlates), hardening recommended → APPLIED this PR: !isBareCrossReferenceAnswer screen in hasValidatedExtractiveCandidate (closes all three short-circuit paths; discriminating test added; disclosed post-review delta, strictly narrows shipping). MERGE GATE: draft until the branch canary pair (baseline #57/#58 vs branch run with answer_case_limit=44 + answer_quality_eval=true, est $3-6 of authorized envelope) is green — zero per-case regressions, recalls 1.0, quality/targeting rates >= baseline. | Red-proof + 3 negative guards; focused 61/61 + fallback/offline/contract suites; full suite 3061 passed / 1 known container artifact (pre-hardening tree; hardening re-verified focused); typecheck+lint+prettier clean; maintainability budget passed (4908/5030) |
 | 2026-07-21 | claude/clinical-kb-pwa-review-asi3wb (pair verdict — no code change; main stays at 22b6a2e) | canary run 29794759627 (#59, branch head 7310cb3 = merged PR-B content) | E-3b + E-3c PR-B LIVE-VALIDATED (pair vs banked #57/#58): route_ceiling_failures 2→0 (E-3b proven: agitation timeout now fits inside budget; clozapine retrieval-exhausted ceiling honestly excused via the triple-condition cross-region carve-out — exactly one "retrieval-exhausted" audit cell in the report); p95 17.4s→15.47s (-11%); golden retrieval SUCCESS 36/36 (stop-ship criterion held); validated_routine_extractive_first fired on 4 of the 6 target cases (patient-safety-plan 3.3s, treatment-team-process 2.6s, ect-procedure 2.2s, illegal-substances 2.2s — all pure extractive, zero generation, was 6-9s each with a discarded attempt), the other 2 (community-home-visits, best-practice-prescribing) stayed on generation+fallback because their extractive candidates legitimately fail validation gates = the designed-conservative outcome; ZERO new failing cases (list 5→2, both known residuals: neuroleptic citation red = Option A territory, admission-comparison expected-doc = non-blocking labeling residual); grounded 1.0 + unsupported_correct 1.0 held; targeting 0.5909→0.619, fail_closed 0.9→0.9333, readability/artifact_leaks/intent_coverage unchanged; relevance 0.6→0.5667 = single-case wobble on n=30, WATCH in E-4, not a gate. Discarded-generation rate materially down (4 conversions; residual = the H2 strong-route slice named as E-3d candidate, per design's 20-33% expectation band). MERGE STANDS (user had armed auto-merge pre-verdict; revert drill not triggered). Spend +~$3-6 → Phase E total ~$6-12 of ≤$20. | Pair evidence: run #59 job log (Blocking failures = citation only; Answer Case Diagnostics markers; metric_rates + targeting blocks); dump artifact populated for PR-C diagnosis |
 | 2026-07-21 | claude/clinical-kb-pwa-review-asi3wb (PR: E-3c PR-C figure-aware selection) | 043b030 + P2-fix commit | E-3c PR-C: dose/monitoring extractive answers now carry the asked-for figure/schedule when the cited chunk verbatim supports it — lead-slot promotion (dose swaps last of 2 slots, monitoring appends 2nd sentence; no-op when a lead already carries a figure) guarded by the claim-support atom corpus (sourceEvidenceText exported, promotionAtomKey byte-identical to claim-support's atomKey), plus the dose/threshold generation-fallback preferring the safe figure-carrying candidate (safety gate unchanged, filter-order-stable). Fallback helpers extracted to rag-extractive-answer (cycle-check verified); rag.ts 4908→4901. Six discriminating tests each verified red-on-prior-code incl. proving the nuke-guard load-bearing by disabling it. REVIEWS (both pre-push on 043b030): rag-retrieval-reviewer APPROVE-WITH-NITS — no-op path byte-identical verified, atom-key identity verified, filter-vs-find proven side-effect-free, 2-sentence append gate-safe, intent double-gated, zero retrieval/ordering change, imputation contract green; P2 = zero-atom monitoring figures ("every 6 weeks" yields no value atom) pass the guard trivially and can be nuked by claim support if sourced only from adjacent context (fails SAFE — evidence gap, never a wrong figure). clinical-governance-reviewer APPROVE-WITH-NITS — all six clinical concerns CLEARED end-to-end (verbatim-support guarantee, citation binding preserved, conservative failure test-proven, unsafe candidates impossible, wrong-drug risk controlled by pre-existing entity/multi-drug guards, no PHI); same zero-atom finding as P3 + one comment-precision nit. P2 FIXED post-review (disclosed): zero-atom figures now require the matched figure substring verbatim in sourceEvidenceText (intentFigureMatchText); proven both directions by 3 new tests (promotes from content, refuses from adjacent-context-only); comment-precision nit folded in. | Focused post-fix: extractive-formatting 32/32 + fallback/eval-cases/offline/contract/extractive-first 98/98 incl. imputation contract; typecheck+prettier+budgets clean; full-suite 3068-passed baseline pre-P2-fix (fix re-verified focused). Live proof = E-4 pair next |
+| 2026-07-21 | claude/clinical-kb-pwa-review-asi3wb (E-4 pair verdict — no code change; #1039 merged as 9b655fa) | canary run 29800029819 (#60, main 9b655fa = E-3b+PR-B+PR-C) | PHASE E-3 WAVE CLOSED — E-4 VERDICT: ADOPT (no revert). 44-case: the ONLY blocking red is the KNOWN persisting neuroleptic citation case (0.0227, identical #57 signature: generation quality-failed → extractive fallback 1 citation — the pre-declared Option A carve-out, retrieval-side, untouched by answer waves); route_ceiling_failures 0 CONFIRMED ON MAIN (E-3b: agitation 23.1s < 25s after 20.3s provider timeout; clozapine 14.5s with exactly one retrieval-exhausted audit cell); grounded 1.0 / unsupported_correct 1.0 / numeric 0 / governance-danger 0 all held; expected_source_hit 0.6136→0.6364 (#1020 widen); generation attempts 19→9 across 44 cases (10+ cases short-circuit via validated_routine_extractive_first at 2-6s, zero generation spend — the absolute wasted-generation seconds collapse; residual 6 discarded attempts are the named E-3d H2 strong/comparison slice). Targeting vs #58 baseline: rate 0.5909→0.6667, dose 1/5→2/4 (sertraline + quetiapine still miss), document_lookup 5/5→6/6, contraindication 2/2, red_result 2/2, pathway 1/2; readability/artifact_leaks 1.0. NOT met: monitoring_schedule flat 1/5 — per-miss lens shows answers of 73-232 chars with NO schedule token available to promote (olanzapine-lai 79ch, metabolic 73ch = single-fact extractive answers; the PR-C promotion is a no-op when no figure-bearing fact is extracted) → root is fact-extraction/retrieval depth on monitoring shapes, queued as the Option-A-wave companion diagnosis (dump artifact 8483731630, 30d retention). WATCH escalated: relevance 0.6 (#58) → 0.5667 (#59) → 0.5333 (#60) — two single-case steps coinciding with more terse extractive answers; fail_closed 0.9 = exactly the #58 main baseline (#59's 0.9333 was the outlier), safety texture flat. Adoption per plan criteria: targeting ≥ baseline ✓, grounded/refusal 1.0 ✓, golden 36/36 ✓ (in-run), ceilings 0 ✓, citation red = carved known case ✓. CodeRabbit post-review follow-up landed pre-merge (02b5c78): interval-regex full-match reorder (atom path proven to intercept the claimed exploit; reorder = drift hardening), clinicalValueAtomKey exported (mirror deleted), guard tests made honestly discriminating + genuine zero-atom "annually" coverage both directions. Instrument note: cost rates live on the targeting step env but eval:quality still reports cost n/a (estimator not consuming them in the 44-case path) — minor tooling residual. Spend +~$2-4 → Phase E total ~$8-16 of ≤$20. | Evidence: run #60 job log read in full (Threshold Status: citation-only; Answer Metrics + 44-row diagnostics; targeting metric_rates + 7-miss list); artifact eval-canary-output 8483731630 sha256 d5c7006e… (download blocked in-session — GitHub App scope; log tee carried the targeting output) |
+| 2026-07-21 | claude/clinical-kb-pwa-review-asi3wb (Option A: title-supported escalation rescue) | 0abf3c9 (parent 1aebf02) | rag-retrieval-reviewer PROTECTED-surface review of the S3 document-lookup escalation rescue: shouldAttemptDocumentLookupFastPath exported + gains medication_dose_risk branch firing ONLY when analysis.intent==="escalation_risk" && documentTitleTerms.length>0; call site passes queryAnalysis; new tests/rag-document-lookup-escalation-rescue.test.ts. | APPROVE-WITH-NITS. No P0/P1/P2. Binding constraints held: released-search-order.ts + retrieval-selection.ts clamp + rag-candidate-sources.ts imputation all byte-identical to parent (git diff empty); imputation-contract test green; 0.66/0.055 floor (rag.ts:1765) unmodified and still gates rescued pools (S3 block sits after the 2513 fast-path return + re-runs decideTextFastPath at 2631). BLAST RADIUS empirically proven via 110-case offline probe (44 ragEvalCases + 30 answerQualityEvalCases + 36 golden): EXACTLY 1 fires (neuroleptic-side-effect-escalation, intent=escalation_risk tt=3); all 8 named dose cases non-firing (clozapine-monitoring general, paraphrase general, agitation-pharm general, im-po drug_dosing, typo-dosing drug_dosing, missed-dose-table drug_dosing, LAI general/tt0, prompt-injection-forge intent=protocol). intentFromSignals precedence (clinical-search.ts:575-585) returns drug_dosing before escalation_risk so pure-dose structurally cannot fire — confirmed by construction AND empirically. ADVERSARIAL: prompt-injection-forge intent=protocol => cannot fire (empirical); unsupported short-circuit (rag.ts:2371) precedes S3 block. DOWNSTREAM: buildRetrievalIntent for the escalation query yields EMPTY requiredTermSignals => demote/promote arms (retrieval-selection.ts:354-355/529/555) + wrong-medication cap (rag.ts:730-737, gated on clinical_subject) all inert; end-to-end test proves neuroleptic-doc rank#1 with >=2 citations, sibling retained, arrival-order invariant. golden vector-\* probes = broad_summary (already allowlisted) => predicate byte-identical => unaffected. TEST HONESTY: S3 fixture (0.92/0.34/0.94/synthetic_text) faithful to searchDocumentLookupFastPath (rag-candidate-sources.ts:570 caps alias documentScore at 0.34 => sim=min(0.92,0.58+0.34+bonus)=0.92, hybrid=min(0.94,0.94)); differently-relevant fixtures (not identical-content); red-proof structurally airtight (e2e gates its own fixture pool on the predicate). EFFORT: one S3 RPC (same call allowlisted classes issue), rescue-only in the non-forceEmbedding path (floor already rejected => query was headed to embedding anyway; successful rescue short-circuits at 2646 pre-embedding = net-neutral/positive). NITS (P3, non-blocking, no code change pre-canary): (1) redundant analysis?. on the documentTitleTerms clause (&& short-circuit already guarantees analysis defined there); (2) a queryMode forcing medication_dose_risk over an originally-escalation-shaped table_threshold could newly fire S3, but table_threshold already ran S3 so behavior-consistent, not a regression; (3) "forceEmbedding discards S3 merges" is imprecise — line 2649 merges S3 even under forceEmbedding; the accurate unaffected-reason for vector-\* probes is broad_summary allowlisting. TRUST GATE = the mandated live canary pair (correctly deferred by the commit). | Offline only, no provider calls: new test 10/10 + imputation-contract 2/2; protected-surface batch 31/31 (escalation-rescue + imputation-contract + fast-path-ordering + released-search-order + retrieval-selection); 110-case classifier probe (temp test, removed); npm run typecheck exit 0. Not run (provider-gated): eval:retrieval:quality, eval:rag, the live canary pair. |
+| 2026-07-21 | claude/clinical-kb-pwa-review-asi3wb (parity commit review) | 1aebf02 (fix landed a3b9a54) | rag-retrieval-reviewer on the monitoring evidence-gate parity commit: REQUEST-CHANGES (soft) — P2 reproduced: inflected monitoring kind tokens (monitor\w*/annual(?:ly)?/blood tests?/ecgs?/lfts?) steal sole-dose-value sentences from the dose arm; dose-intent answers then reject the monitoring-kind fact ("Quetiapine is monitored at a dose of 200 mg daily" flipped grounded true→false, source-gap — fails CLOSED, never a wrong dose). P3: monitoring figure escape lacked the dose escape's multi-drug bare-row guard. Clean: over-admission bounded (broad vocab lives in gate/filter only, promotion still corpus-guarded, claim-support unchanged); regex cost negligible; mismatched-unit test relaxation legitimate (synopsis is corpus-verbatim; weeks pin enforced by atom identity + adjacent_context exclusion from both gate and claim corpora). | BOTH FINDINGS FIXED in a3b9a54: kind arm classifies legacy tokens byte-identically and new-inflection-only sentences fall through to the dose arm when they carry a clinicalDoseValuePattern value (both repro sentences pinned as dose-intent tests); multi-drug bare-row guard extended to monitoring_schedule with a discriminating test — red-proven both directions. Reviewer checks: formatting 38/38, focused 644/644; targeting eval deferred to the wave's live canary pair. |
+| 2026-07-21 | claude/clinical-kb-pwa-review-asi3wb (parity commit review) | 1aebf02 (fix landed a3b9a54) | clinical-governance-reviewer on the same commit: APPROVE-WITH-NITS. P2 (independently converged with the retrieval reviewer's P3): monitoring figure-escape lacked the dose-path multi-drug cross-entity guard — a bare wrong-drug schedule/level row in a multi-drug chunk could be entity-prefixed for a named-drug monitoring query; downstream gates verify text-vs-source presence, never attribution (worked lithium/valproate LFT path traced through finalize). FIXED in a3b9a54 exactly as its smallest-fix prescribed (guard at the :872-881 site now fires for monitoring_schedule; negative multi-drug test added, red-proven). Clean: unsupported figures impossible (admission-only change; promotion corpus guard + numeric verification + claim support all byte-unchanged); conservative failure intact (figure-bearing-only escape, schedule-free refusal pinned); adjacent-context safety held (sourceEvidenceText excludes adjacent_context; weeks refusal confirmed by probe); no PHI/provider/ranking surface. P3s: RAG impact line (present in the PR body — behaviour-change form, correct since the PR also carries the Option A retrieval change); multi-drug negative test (landed in a3b9a54). | Offline guard-chain trace + targeted vitest probes (named-drug guard, bare-figure admission, conservative gap, adjacent refusal). Provider/release gates deferred per confirmation boundary. |
+| 2026-07-21 | claude/clinical-kb-pwa-review-asi3wb (Option A rescue review) | 0abf3c9 | clinical-governance-reviewer on the S3 escalation rescue: APPROVE-WITH-NITS, no P0/P1. P2 = the mandated live canary pair itself (process gate, declared in the PR body; offline-green + review-approved proven insufficient for this surface 2026-07-20). P3s: multi-drug escalation-query recall edge (titled drug + untitled drug — fast-path return can skip the vector leg; recall limitation, not misattribution, mirrors the pre-existing allowlisted-class tradeoff); reviewer probe files must stay uncommitted (relocated to scratchpad). All six clinical concerns verified safe: wrong-document impossible (alias phrases must appear in the query; per-document alias groups, no cross-drug conflation), conservative availability (S3 purely additive via keyed-union merge; sibling retention test-pinned), live expansion acceptable (title-named correct-entity SOP in every firing shape), fail-closed double layer (adversarial short-circuit precedes the predicate; injection-forge case intent=protocol cannot fire — executed), governance metadata unbypassed (same attachDocumentRankingMetadata + status=indexed + access-scope filters), no PHI/provider/schema surface. | Reviewer checks: escalation-rescue suite 8/8, injection-forge intent derivation executed, static trace of the full S3 chain. Live canary pair = the trust gate, dispatched post-merge. |