feat(rag): pre-generation validated-extractive short-circuit for routine procedural queries (E-3c PR-B)

docs/branch-review-ledger.md

@@ -669,3 +669,4 @@ Use this ledger to prevent repeated branch and PR reviews when the reviewed HEAD
 | 2026-07-21 | claude/clinical-kb-pwa-review-asi3wb (PR: E-3b budget-aware generation deadlines) | 314d03f | Clinical-governance review of E-3b diff `git diff origin/main...HEAD` (commits 1078264 + 314d03f): answer-side generation timing/gating + telemetry — reserve-aware generation timeout (`generationRequestTimeoutMs`), truncation self-heal budget gate (`deadlineAllowsGenerationRetry`), `route_budget_exhausted_by_retrieval` telemetry, and the cross-region eval carve-out in scripts/eval-quality.ts. | APPROVE-WITH-NITS. No P0/P1/P2. (a) Conservative failure preserved: reserve-aware timeout fires ~2s early producing the SAME error type — an internal SDK timeout is mapped by mapOpenAIError to PublicApiError(openai_timeout) (openai.ts:525-529), NOT a bare DOMException, so the rag.ts:4636 re-throw guard is not tripped and the existing source-backed fallback/extractive recovery is reached; truncation-skip falls through to the terminal throw (rag.ts:4309-4313) into the same catch. No new answer-producing path. (b) Safety gates intact: all recovery answers finalize through finalizeAnswer→finalizeRagAnswerQuality and isSafeExtractiveFallbackCandidate (grounded/confidence/quality/numeric) — none touched. (c) Eval carve-out env-gated: crossRegionRunner && budgetExhaustedByRetrieval && generationMs===0; EVAL_LATENCY_CONTEXT set only in eval-canary.yml:164, prod caller (eval-quality.ts:1095) passes no options → inert in release/local; generationMs===0 requirement means a generation-side failure (generationMs>0) is never suppressed. (d)/(e) Telemetry additions non-PHI (boolean + mechanical retry-reason strings); no privacy/query-privacy/cross-border/verification source files touched; no new provider call. Nits (P3, non-blocking): carve-out also excuses fast/strong routes when generationMs===0 (sound — provably no generation ran); `requestTimeoutMs` now prod-dead (test-only); report label "retrieval-exhausted" (routeDeadlineExceeded && flag) is broader than actual gate suppression (audit label only, gate stays strict). | 88 offline unit tests PASS (tests/rag-route-budget.test.ts, tests/eval-quality.test.ts, tests/rag-offline-answer.test.ts, tests/rag-answer-fallback.test.ts); no provider/Supabase/OpenAI calls; no files mutated except this ledger row |
 | 2026-07-21 | claude/clinical-kb-pwa-review-asi3wb (PR: E-3b budget-aware generation deadlines) | 314d03f (+eb08ec5 review row) | ADDENDUM 5 wave E-3b implemented per the design-agent plan: generation attempts clamped to route budget minus a measured 2s recovery reserve (single call-site, all four attempt kinds); truncation self-heal gated on retry viability (reserve+5s floor) with observable truncation_retry_skipped_budget_reserve marker; additive route_budget_exhausted_by_retrieval runtime flag; eval route-ceiling gains the triple-condition cross-region carve-out (context + runtime flag + zero generation) with retrieval-exhausted audit cells — local/release gates provably strict. Fixes I3 (54ms budget overrun after 22.6s provider timeout), I5 (82s truncation waste class), resolves I2 (clozapine 13.3s retrieval vs 12s runtime budget = geography, now suppressed ONLY in the sanctioned cross-region context with full auditability). Reviewer verdicts: rag-retrieval-reviewer APPROVE-WITH-NITS (2 P3: prod-dead requestTimeoutMs retained for symmetry; report-cell coupling cosmetic; cached-replay invariant PROVEN — budget-exhausted answers never cached, carve-out unreachable via replay; marker isolation proven — SLO counters key on fallback_reason not answer_retry_reasons); clinical-governance-reviewer APPROVE-WITH-NITS (prior row) — internal-timeout→PublicApiError→existing-fallback path verified, all safety gates still applied to recovery answers. ALSO BANKED — E-2 targeting baseline (canary run #58, 29788404357, all-green incl. first execution of the !cancelled()-fixed instrument, ~$1-2): metric_rates relevance 0.6 / readability 1.0 / artifact_leaks 1.0 / intent_coverage 0.9333 / fail_closed 0.9; targeting_rate 0.5909 (13/22); by intent: document_lookup 5/5, red_result_action 3/3, contraindication 2/2, dose 1/5, monitoring_schedule 1/5, pathway_referral 1/2; all 9 misses = missing dose figure/schedule-interval (answer lengths 73-232 chars) → E-3c co-primary target alongside the wasted-generation class. Phase E spend ≈$3-6 of ≤$20. | Red-proofs: reserve pinned 3 independent ways (exact 23000ms grant, deadline flag clear, total under budget); self-heal skip pins exact marker + single provider call; offline flag pinned true/false. Focused: route-budget 9/9, eval-quality 27/27, fallback+offline 52/52, parser/abort regressions 15/15. Full suite 3043 passed / 1 known container pdf artifact. typecheck+lint+prettier clean. No provider calls; live proof = E-4 paired run |
 | 2026-07-21 | claude/clinical-kb-pwa-review-asi3wb (restarted from e02ba3d; PR: E-3c PR-A instrument) | see PR head | ADDENDUM 5 wave E-3c design accepted (Plan agent, full report in session record) and PR-A delivered: eval-answer-quality gains --dump-answers <path> (per-case answer TEXT + sections + targeting verdict JSON for the canary artifact — answers are not retained at rest by privacy design, so miss diagnosis needs eval-time capture), parseArgs/buildAnswerDumpRecord exported behind an import.meta main-guard (eval-quality precedent), canary targeting step wired with the flag. Design highlights for the record: DEFECT 1 root confirmed as fast-attempt-doomed-then-discarded on strong_routine_retrieval procedural shapes — PR-B generalizes the EXISTING validated-extractive short-circuit pattern (LAI + blocked-recovery precedents, hasValidatedExtractiveCandidate) to the measured shape via a new rag-extractive-first.ts module (net ~-110 rag.ts lines against the 5030 budget); DEFECT 2 rank-1 root = extractive lead-slot selection prefers shortest sentence and admits figure-less leads (sort at rag-extractive-answer.ts:917, 1-slot monitoring leads) — PR-C adds intent-figure-aware lead promotion with a claim-support atom-corpus nuke-proofing guard + dose/threshold fallback candidate preference (find(safe && figure) ?? find(safe)); answer-verification CLEARED as direct cause (whole-answer gap or unbold only, never per-figure deletion). H2 (strong-route comparison/complex residual) explicitly deferred as the named E-3d candidate. E-4 metrics set: discarded-generation <20% from ~47%, dose ≥3/5, monitoring ≥3/5, no intent below #58, recalls pinned 1.0, relevance/fail_closed/readability/artifact_leaks ≥ #58. | PR-A gates: new tests/eval-answer-quality.test.ts 4/4; typecheck, check:github-actions, check:ci-scope, eval-canary-workflow test, prettier all clean; no provider calls |
+| 2026-07-21 | claude/clinical-kb-pwa-review-asi3wb (PR: E-3c PR-B short-circuit) | f7e6cbb + hardening commit | E-3c PR-B: pre-generation validated-extractive short-circuit for gate-passed routine procedural "What...process/include/required" queries (marker validated_routine_extractive_first), generalizing the LAI + blocked-recovery precedents via new rag-extractive-first.ts (3 predicates moved byte-verbatim, machine-verified; rag.ts 5029→4908 vs 5030 budget). Kills the run-#57 6x wasted-generation class. REVIEWS (both pre-push): rag-retrieval-reviewer APPROVE-WITH-NITS — move fidelity brace-diff verified byte-identical; confidence-gate skip PROVEN safe (passed-gate is a no-op in applyConfidenceGate; markers pairwise mutually exclusive); comparison false-positives blocked by unchanged classifier precedence; offline/source-only idempotent; zero retrieval/ranking/selection/threshold change; P2 = eval-only assertions (intent_coverage/artifact_leaks/expected-file) unverifiable offline for flip candidates (quality-nocc-document-support, quality-form-required-documentation, quality-discharge-documentation, quality-duress-pathway + rag-set siblings) → pre-merge BRANCH canary recommended and ADOPTED (offline-green + review-approved proven insufficient for this surface, 2026-07-20). clinical-governance-reviewer APPROVE-WITH-NITS — full gate-stack trace: nothing bypassed (same finalizeRagAnswerQuality, same citation scoping, numeric verification not fail-open, ungrounded-finalize defense at rag.ts:3694); P2 = pre-existing bare-cross-reference-with-overlap gap, NOT materially widened (new trigger anti-correlates), hardening recommended → APPLIED this PR: !isBareCrossReferenceAnswer screen in hasValidatedExtractiveCandidate (closes all three short-circuit paths; discriminating test added; disclosed post-review delta, strictly narrows shipping). MERGE GATE: draft until the branch canary pair (baseline #57/#58 vs branch run with answer_case_limit=44 + answer_quality_eval=true, est $3-6 of authorized envelope) is green — zero per-case regressions, recalls 1.0, quality/targeting rates >= baseline. | Red-proof + 3 negative guards; focused 61/61 + fallback/offline/contract suites; full suite 3061 passed / 1 known container artifact (pre-hardening tree; hardening re-verified focused); typecheck+lint+prettier clean; maintainability budget passed (4908/5030) |

src/lib/rag/rag-extractive-first.ts

@@ -0,0 +1,218 @@
+import {
+  buildExtractiveAnswer,
+  finalizeRagAnswerQuality,
+  isBareCrossReferenceAnswer,
+} from "@/lib/rag/rag-extractive-answer";
+import type { RagQueryClass, RetrievalConfidenceGateStatus, SearchResult } from "@/lib/types";
+
+/**
+ * Pre-generation validated-extractive short-circuit.
+ *
+ * Each predicate here targets a measured wasted-generation shape: a query whose
+ * deterministic extractive answer independently passes every final quality and
+ * grounding gate, so the paid model call adds latency and cost without adding
+ * trust. A predicate may flip the answer route to `extractive` only when its
+ * validated candidate has already cleared those gates; retrieval, ranking, and
+ * selection behaviour are never touched.
+ */
+
+type ShortCircuitRoute = { mode: "unsupported" | "extractive" | "fast" | "strong"; reason: string };
+
+/**
+ * Allow a score-blocked routine document-content query to use the deterministic
+ * answer only when that answer independently passes the final safety gates.
+ *
+ * This is deliberately narrower than the normal extractive router: it cannot
+ * recover medication, threshold, comparison, broad-summary, complex, or weakly
+ * related queries. The retrieval diagnostic remains blocked so the UI still
+ * presents the recovered answer with low-trust guidance.
+ */
+export function hasValidatedExtractiveCandidate(args: {
+  query: string;
+  queryClass: RagQueryClass;
+  results: SearchResult[];
+  routeReason: string;
+}) {
+  const candidate = finalizeRagAnswerQuality(
+    buildExtractiveAnswer({
+      query: args.query,
+      queryClass: args.queryClass,
+      results: args.results,
+      quoteCards: [],
+      documentBreakdown: [],
+      evidenceSummary: undefined,
+      sourceCoverage: undefined,
+      conflictsOrGaps: [],
+      visualEvidence: [],
+      bestSource: null,
+      smartPanel: undefined,
+      relatedDocuments: [],
+      routeReason: args.routeReason,
+      timings: undefined,
+    }),
+    args.query,
+    args.queryClass,
+  );
+
+  return (
+    candidate.grounded &&
+    candidate.confidence !== "unsupported" &&
+    candidate.citations.length > 0 &&
+    candidate.responseMode !== "evidence_gap" &&
+    !/final_quality_gate:/.test(candidate.routingReason ?? "") &&
+    // Governance-review hardening (PR-B P2): a cross-reference lead that shares query terms
+    // passes the overlap gate yet answers nothing ("Refer to the X procedure for..."). Screen
+    // it here so no short-circuit path can ship a redirect-only answer; the query then stays
+    // on model synthesis (or the existing fallback chain) instead.
+    !isBareCrossReferenceAnswer(candidate.answer ?? "")
+  );
+}
+
+/** Recover only routine, source-backed document lookups whose deterministic answer passes every final gate. */
+export function hasValidatedRoutineExtractiveRecovery(args: {
+  query: string;
+  queryClass: RagQueryClass;
+  results: SearchResult[];
+  route: { mode: "unsupported" | "extractive" | "fast" | "strong"; reason: string };
+  sourceBacked: boolean;
+}) {
+  if (
+    args.queryClass !== "document_lookup" ||
+    args.route.mode !== "fast" ||
+    args.route.reason !== "strong_routine_retrieval" ||
+    !args.sourceBacked
+  ) {
+    return false;
+  }
+
+  return hasValidatedExtractiveCandidate({
+    query: args.query,
+    queryClass: args.queryClass,
+    results: args.results,
+    routeReason: `${args.route.reason}; validated_routine_extractive_recovery`,
+  });
+}
+
+/**
+ * Generic LAI-management questions repeatedly time out in generation despite strong direct
+ * source support. Skip that paid tail only when the question asks no specific clinical detail
+ * and the deterministic answer independently passes the same final quality and grounding gates.
+ */
+export function hasValidatedGenericLaiManagementExtractiveAnswer(args: {
+  query: string;
+  queryClass: RagQueryClass;
+  results: SearchResult[];
+  route: { mode: "unsupported" | "extractive" | "fast" | "strong"; reason: string };
+  sourceBacked: boolean;
+}) {
+  const genericLaiManagementQuery = /^\s*how (?:are|should) long[- ]acting injectables? (?:be )?managed\??\s*$/i.test(
+    args.query,
+  );
+
+  if (
+    !genericLaiManagementQuery ||
+    args.queryClass !== "medication_dose_risk" ||
+    args.route.mode !== "fast" ||
+    args.route.reason !== "clinical_fast_grounded_synthesis" ||
+    !args.sourceBacked
+  ) {
+    return false;
+  }
+
+  return hasValidatedExtractiveCandidate({
+    query: args.query,
+    queryClass: args.queryClass,
+    results: args.results,
+    routeReason: `${args.route.reason}; validated_generic_lai_management_extractive_answer`,
+  });
+}
+
+const routineProceduralLeadPattern = /^\s*what\b/i;
+const routineProceduralKeywordPattern =
+  /\b(?:process|procedure|steps?|includes?|include|required?|requires?|requirements?|documentation)\b/i;
+
+/**
+ * Routine procedural document-content shape: a "What ..." question asking what a
+ * process/procedure includes or requires. Deliberately EXCLUDES "How is X
+ * handled/managed?" shapes — those stay on model synthesis (see the gate-passed
+ * routine document-content contract in tests/rag-answer-fallback.test.ts).
+ */
+export const routineProceduralContentPattern = {
+  test(query: string) {
+    return routineProceduralLeadPattern.test(query) && routineProceduralKeywordPattern.test(query);
+  },
+};
+
+/**
+ * Gate-passed routine procedural document-content questions repeatedly pay for model
+ * synthesis that the deterministic extractive answer already covers. Skip that paid tail
+ * only for the strong routine fast route, only for routine lookup/general classes (never
+ * dose, threshold, comparison, or broad-summary classes), and only when the deterministic
+ * answer independently passes every final quality and grounding gate.
+ */
+export function hasValidatedRoutineProceduralExtractiveAnswer(args: {
+  query: string;
+  queryClass: RagQueryClass;
+  results: SearchResult[];
+  route: ShortCircuitRoute;
+  sourceBacked: boolean;
+  gateStatus: RetrievalConfidenceGateStatus;
+}) {
+  if (
+    args.route.mode !== "fast" ||
+    args.route.reason !== "strong_routine_retrieval" ||
+    args.gateStatus !== "passed" ||
+    !args.sourceBacked ||
+    (args.queryClass !== "document_lookup" && args.queryClass !== "unsupported_or_general") ||
+    args.results.length === 0 ||
+    !routineProceduralContentPattern.test(args.query)
+  ) {
+    return false;
+  }
+
+  return hasValidatedExtractiveCandidate({
+    query: args.query,
+    queryClass: args.queryClass,
+    results: args.results,
+    routeReason: `${args.route.reason}; validated_routine_extractive_first`,
+  });
+}
+
+/**
+ * Choose the first applicable validated-extractive short-circuit for the current route.
+ *
+ * Precedence is fixed: the generic LAI-management skip (gate passed), then the
+ * score-blocked routine recovery (gate blocked), then the gate-passed routine
+ * procedural short-circuit. Returns the routing-reason marker to append, or null
+ * when generation should proceed unchanged.
+ */
+export function chooseValidatedExtractiveShortCircuit(args: {
+  query: string;
+  queryClass: RagQueryClass;
+  results: SearchResult[];
+  route: ShortCircuitRoute;
+  sourceBacked: boolean;
+  gateStatus: RetrievalConfidenceGateStatus;
+}): { reasonMarker: string } | null {
+  const predicateArgs = {
+    query: args.query,
+    queryClass: args.queryClass,
+    results: args.results,
+    route: args.route,
+    sourceBacked: args.sourceBacked,
+  };
+
+  if (args.gateStatus === "passed" && hasValidatedGenericLaiManagementExtractiveAnswer(predicateArgs)) {
+    return { reasonMarker: "validated_generic_lai_management_extractive_answer" };
+  }
+
+  if (args.gateStatus === "blocked" && hasValidatedRoutineExtractiveRecovery(predicateArgs)) {
+    return { reasonMarker: "validated_routine_extractive_recovery" };
+  }
+
+  if (hasValidatedRoutineProceduralExtractiveAnswer(args)) {
+    return { reasonMarker: "validated_routine_extractive_first" };
+  }
+
+  return null;
+}