diff --git a/docs/branch-review-ledger.md b/docs/branch-review-ledger.md index 6f0a2c433..ae4096166 100644 --- a/docs/branch-review-ledger.md +++ b/docs/branch-review-ledger.md @@ -653,3 +653,5 @@ Use this ledger to prevent repeated branch and PR reviews when the reviewed HEAD | 2026-07-20 | claude/clinical-kb-pwa-review-asi3wb (restarted; PR: eval measurement floor, ADDENDUM 4 A-PR-1) | 81ab9696da4b330ca0b2e5519891a9942f90421b | Measurement floor for evidence-gated ranking tuning: canary artifact emission, alias-aware snapshot builder, snapshot provenance/freshness — no ranking behavior change | Closes the three gaps blocking safe tuning (Phase B): (1) eval-canary's golden step now writes the per-case JSON artifact (--json-out decoupled from --json so the tee'd log keeps the human-readable lines the failure-issue analyzer parses) and uploads .local/eval-canary/ via pinned upload-artifact (30-day retention, include-hidden-files for the dot-dir, contents = same class as the already-public step logs: titles/telemetry/220-char previews of the all-public corpus) — snapshot regeneration stops costing a paid dispatch; (2) clinicalDocumentAliases/clinicalContentAliases moved verbatim to shared scripts/lib/clinical-aliases.ts and the snapshot builder grades documentMatch/contentMatch through them (discriminating tests: EMHS agitation title and spelled-out "absolute neutrophil count" grade as hits only via aliases — raw labelMatches pinned false), ending tuner ground truth disagreeing with the live gates; (3) snapshots carry generatedAt + optional sourceRunId, validator accepts them, exactly-36 relaxed to at-least-36 (floor still rejects truncated artifacts; sourceCaseCount + per-case candidate minimums unchanged), and a 30-day freshness test (activates on first regeneration) blocks silent corpus drift. Builder smoke-verified end-to-end on a synthetic 36-case artifact (alias grading + provenance stamped + validator green). Static hyphen audit of all 36 cases' terms: no currently-blocked term (canary #52 = 36/36); residual risk classes documented for the A-PR-2 artifact-grounded pass — punctuation-joined tokens (IM/PO, schizo-affective, post-natal) and inert stem entries (obsess/compuls/hyperactiv/impuls can never match whole-token) that currently ride on whole-word OR-alternates. | Targeted vitest 52/52 (ranking-tuning + eval-retrieval + eval-quality); npm run test 3019 passed / 1 known container-only pdf-budget artifact; lint + typecheck clean; check:github-actions + check:ci-scope PASS; prettier clean; check:production-readiness expected missing-secret FAILs only (demo-mode container); no provider calls — live validation = tonight's scheduled canary emits the first artifact at $0 | | 2026-07-20 | claude/clinical-kb-pwa-review-asi3wb (restarted; PR #1001: A-PR-2 measurement-floor completion) | 3d8f798 + 5b664f8 + e58c827 | First provenance-stamped snapshot regeneration from a live canary artifact + alias tiering docs + fixture-length pin + lithium doc-gate — Phase A of ADDENDUM 4 functionally complete | Artifact chain: user-authorized paid dispatch (canary #53, run 29763761133, 36/36 green, doc_recall 1.0 / content_recall 1.0 — first perfect content recall, ciwa alias confirmed live; mrr@10 0.8644, irrelevant@10 0.1083) emitted the first eval-canary-output artifact (51787 bytes, sha256 5af5b802… verified byte-identical after user transfer into the sandbox — this container cannot download run artifacts). Work: (1) two-tier alias documentation — investigation of the governance-review P3 showed src/lib/eval-document-matching.ts is a deliberately WIDER captured-case tier (e.g. "Clozapine GP Shared Care"); bulk-merge would loosen golden ground truth, so both files now carry cross-referencing do-not-merge headers instead; (2) snapshot case count pinned to live golden fixture length (regeneration instructions in failure message) — closes the coarse-floor P3; (3) snapshot regenerated via the alias-aware builder with --source-run-id provenance: agitation-im-po-options 0→5 graded positives (EMHS alias working on real data), flowchart-next-step confirmed sole zero-positive case; generatedAt promoted to validator-REQUIRED (closes the hand-edit P3); two stale data pins updated (missing-positives 2→1; broad_summary defaults-equality pin dropped — defaults' provenance was the retired snapshot, fresh recommendations are Phase B input); (4) artifact-grounded punctuation audit: 7 joined-token occurrences in top-5 previews — 3 ciwa-ar (alias-covered, incl. line-broken "ciwa- ar"), 4 ORDINARY-PROSE punctuation ("treatment," / "mood," / "(opioid" / "ptsd.[35]") — matcher word-boundary change proposed as its OWN reviewed follow-up per plan (systemic class, not bundled); (5) lithium-therapy-monitoring was the ONLY ungated case (rr@10 hardcoded 0.00 = measurement noise): expectedDocumentSubstrings ["Lithium"] added from live evidence (deliberately broad across the corpus's multiple legitimate lithium guidelines), snapshot rebuilt in lockstep from the same artifact, measured mrr@10 +~0.028 from de-noising. Deferred with reasons: NEW-query fixture cases (saturated-tie shapes, captured rag_query_misses) need live validation before they may gate — unlocked by Phase D-1 branch-eval dispatch or a dedicated validation dispatch; real ordering headroom for Phase B = flowchart 0.20, alcohol-ciwa 0.25, patient-safety 0.33, opioid 0.33, all text_fast_path. | Targeted vitest 76/76 ×3 (after each stage); npm run test 3019 passed / 1 known container-only pdf-budget artifact; prettier clean; freshness gate ACTIVE and green; no provider calls beyond the user-authorized dispatch (~$1-2, ADDENDUM 4 spend now ~$1-2 of ≤$10) | | 2026-07-20 | claude/clinical-kb-pwa-review-asi3wb (restarted; PR: B-PR-1 canary dispatch inputs) | see PR head | eval-canary.yml workflow_dispatch gains rag_ranking_config (staged Phase B weight evals, loud malformed-JSON validation to prevent silent fallback corrupting pair comparisons) + ref (Phase D-1 pre-merge branch evals) — schedule runs unaffected (empty inputs = today's behavior) | Tuner (offline, $0) on the A-PR-2 regenerated snapshot recommends 3 constrained per-class improvements (document_lookup titleSectionRelevance→0.9; table_threshold clinicalEvidence→0.95; comparison hybridRelevance→0.95 with proxy mrr 0.833→1.0), all with recall non-regression + zero high-risk hard-negative failures; medication_dose_risk + broad_summary stay neutral. Staged config JSON banked for the live pair; baseline = canary #53 re-gated (mrr@10 ≈0.8922 after lithium de-noising). Targets per approved plan: mrr@10 ≥0.90, irrelevant@10 ≤0.08, zero case regressions. Budget: user raised cap to ≤$20 (spent ≈$1-2). Security note: ref input runs branch code with eval secrets — dispatch requires repo write access (sole trusted collaborator), documented in PR risk. | check:github-actions PASS; check:ci-scope PASS; prettier clean; no provider calls (workflow change only) | +| 2026-07-20 | claude/clinical-kb-pwa-review-asi3wb (Phase B live pair — no code change; recorded for eval-history completeness) | canary runs 29763761133 (#53 baseline) + 29769050798 (#54 tuned) | ADDENDUM 4 Phase B verdict: staged 3-class tuner weights produced ZERO live movement — NOT adopted per the measured-gain rule | Offline tuner recommendations (document_lookup titleSection→0.9, table_threshold clinicalEvidence→0.95, comparison hybridRelevance→0.95; proxy comparison-mrr 0.833→1.0) staged via the B-PR-1 rag_ranking_config input; #54 log proves the override active (validation step echo). Result vs re-gated baseline: mrr@10 0.8921 vs 0.8922, irrelevant@10 0.1083 vs 0.1083, recalls 1.0/1.0 both, 36/36 both, all four headroom cases byte-identical rr (flowchart 0.20, ciwa-threshold 0.25, patient-safety 0.33, opioid 0.33). Learning: the 5-candidate linear proxy saturates; real headroom lives in fast-path saturated-tie structure (Phase C trigger condition mrr < 0.90 formally met; user authorized C). Nothing to roll back (per-run override). Spend ≈$2-4 of user-raised ≤$20 cap. | Both canary runs green (36/36 + answer-quality); override-active proof in #54 job log; no adoption = no code diff | +| 2026-07-20 | claude/clinical-kb-pwa-review-asi3wb (restarted; PR: Phase C saturation-tail primaries) | 7572c7f | ADDENDUM 4 Phase C (user-authorized): per-candidate discriminative primaries for saturated fast-path ties — design-agent planned (consumer map + dead-band envelope proof), red-proven, tie-conservation guarded | Mechanism: min(text_rank,1) collapses all tr≥1 candidates to byte-identical imputed primaries; ordering fell to chunk id at release. Fix: saturationTailUnit (pure, monotone, SET-INDEPENDENT — rejected per-query min-max + rank-tier designs for set-dependence/#118 authority risk; rejected full-range log rescale for moving sub-knee values across the 0.62-0.82 gate ladder) scales the excess into DEAD cap bands only: S2 table-fact similarity (0.92, 0.94) with hybrid byte-identical (gates/triggers/selection provably unchanged; similarity = the release tie-break key), S1 lexical-chunk hybrid (0.48, 0.5) behind the truthful-contract signature (sub-0.5 bars hold). Sub-knee byte-identical (fixtures now DERIVE from the helper; 0.45→0.755/0.795 pinned). Discriminating test verified RED on old formulas (2 fail: discriminating + envelope) → green with tail; equal-tr tie-conservation pins the #987 coverage comparator; second-stage-engaged pools documented out of scope (position-derived releaseRankScore sorts first there) — matches live evidence that non-engaged pools (patient-safety, opioid, flowchart) are where id-order decided. S3/S4 = C-PR-2 candidates, evidence-gated on the post-merge canary vs #54 baseline (doc/content recall MUST stay 1.0, zero per-case regressions; success signal = rr lift on the headroom cases). Rollback: single revert (helpers + 2 expression sites + 1 map call; no schema/config/cache surface). | Targeted vitest 121/121 (fast-path 11/11 incl. 6 new, retrieval-selection, rag-routing, rag-answer-fallback, ranking-tuning, second-stage); npm run test 3025 passed / 1 known container pdf-budget artifact; lint + typecheck + prettier clean; red-proof executed and recorded; live validation = post-merge canary dispatch (~$1-2) | diff --git a/docs/rag-hybrid-findings-and-todo.md b/docs/rag-hybrid-findings-and-todo.md index 1a780aaf0..4d93f91ff 100644 --- a/docs/rag-hybrid-findings-and-todo.md +++ b/docs/rag-hybrid-findings-and-todo.md @@ -297,10 +297,15 @@ denied to set parameter`)** — the RC11 blocker. The only method hosted allows audit on `claude/retrieval-correctness`: the headline `least(0.95, 0.56 + text_rank*0.39)` proxy NO LONGER EXISTS — `match_document_chunks_text` already returns `similarity = 0` with hybrid capped at 0.5 and the lexical signal isolated in `lexical_score` (codified in - schema.sql with the "do not fabricate" comment). What remains synthetic are three app-side - fabricators, all tagged `similarity_origin: "synthetic_text"`: the document-lookup fast path - (0.58 + documentScore, hybrid ≤ 0.94), memory-card chunk loader (0.58 + confidence·0.28, - hybrid ≤ 0.89), and table-fact signal matches. Their consumers: + schema.sql with the "do not fabricate" comment). Since ADDENDUM-4 Phase C, rows whose + text_rank saturates the SQL clamp additionally get a per-candidate hybrid spread inside the + dead (0.48, 0.5) band (`liftSaturatedLexicalChunkHybrid`) so exact ties resolve by raw text + relevance instead of chunk-id order — the sub-0.5 contract holds. What remains synthetic are + three app-side fabricators, all tagged `similarity_origin: "synthetic_text"`: the + document-lookup fast path (0.58 + documentScore, hybrid ≤ 0.94), memory-card chunk loader + (0.58 + confidence·0.28, hybrid ≤ 0.89), and table-fact signal matches (whose imputed + similarity likewise carries a Phase C tail into its dead (0.92, 0.94) band via + `imputedTableFactPrimaries`; hybrid unchanged). Their consumers: `evaluateEvidenceCoverageGate` / `shouldReturnTextFastPath` / `chooseAnswerRoute` / `shouldUseExtractiveAnswer` (thresholds 0.32–0.76 — the fabricated 0.58 floor is deliberately load-bearing there, always paired with structural checks like diff --git a/src/lib/rag/rag-candidate-sources.ts b/src/lib/rag/rag-candidate-sources.ts index d5208f98b..7def62ac9 100644 --- a/src/lib/rag/rag-candidate-sources.ts +++ b/src/lib/rag/rag-candidate-sources.ts @@ -211,7 +211,12 @@ export async function searchTextChunkCandidates(args: { // most-terminal lexical layer surfaces in hybrid_rpc_errors telemetry // instead of silently degrading to zero candidates. Return value unchanged. if (error) recordHybridRpcError(args.telemetry, "match_document_chunks_text", error); - return error || !data?.length ? ([] as SearchResult[]) : (data as SearchResult[]); + // Phase C: single choke point for every lexical-chunk variant (strict, corrected, + // OR-relaxed, legacy-merged) so saturated rows gain the discriminative hybrid lift + // before mergeSearchResults sees them. + return error || !data?.length + ? ([] as SearchResult[]) + : (data as SearchResult[]).map(liftSaturatedLexicalChunkHybrid); }; const variants = args.queryVariants.slice(0, maxTextRpcQueryVariants); @@ -314,6 +319,60 @@ type DocumentLookupChunkRow = { text_rank?: number | null; }; +/** + * ADDENDUM-4 Phase C: per-candidate discrimination for saturated imputed primaries. + * + * The embedding-free text fast path imputes primaries from text_rank through min(text_rank, 1), + * so every candidate with text_rank >= 1 collapses to byte-identical values and unrelated + * documents matching the same terms tie exactly — ordering then falls to chunk-id, which is + * arbitrary (live evidence: canary #53/#54 identical rr@10 on the four fast-path headroom cases). + * saturationTailUnit maps the raw rank's excess above the knee into [0, 1) — pure, strictly + * monotone, set-independent (one candidate's value never depends on the rest of the pool) — and + * the callers scale it into DEAD score bands above the pre-existing caps' functional range + * (table-fact similarity (0.92, 0.94); lexical-chunk hybrid (0.48, 0.5)), where no retrieval + * gate, routing threshold, or confidence bar exists. Below the knee the imputation is + * byte-identical to the historical formulas, and genuinely equal text_ranks still tie, so the + * coverage tie-break contract (retrieval-selection) is preserved. See the Phase C design in the + * PR that introduced this for the full consumer map and envelope proof. + */ +export function saturationTailUnit(textRank: number): number { + if (!Number.isFinite(textRank) || textRank <= 1) return 0; + return (textRank - 1) / (textRank - 1 + 0.5); +} + +/** + * Imputed table-fact primaries (S2). similarity carries the discriminative tail into its dead + * cap band; hybridScore stays byte-identical to the historical formula so selection scores, + * fast-path/coverage gates, and the second-stage trigger are provably unchanged — similarity is + * exactly the tie-break key after hybrid/releaseRankScore in the released-order comparators. + */ +export function imputedTableFactPrimaries(textRank: number): { similarity: number; hybridScore: number } { + return { + similarity: Math.min(0.94, 0.62 + Math.min(textRank, 1) * 0.3 + 0.02 * saturationTailUnit(textRank)), + hybridScore: Math.min(0.97, 0.66 + Math.min(textRank, 1) * 0.3), + }; +} + +/** + * Lexical text-chunk lift (S1). Chunk rows keep similarity === 0 (the no-fabricated-cosine + * contract), so their saturation spread goes into hybrid's dead (0.48, 0.5) band instead — + * still strictly below the 0.5 answer-confidence and 0.64 strong-evidence bars. Guarded by the + * truthful-contract signature, never a float-equality test on the saturated value. + */ +export function liftSaturatedLexicalChunkHybrid(row: SearchResult): SearchResult { + const textRank = typeof row.text_rank === "number" ? row.text_rank : Number(row.text_rank ?? 0); + if ( + row.similarity !== 0 || + typeof row.lexical_score !== "number" || + !Number.isFinite(textRank) || + textRank <= 1 || + typeof row.hybrid_score !== "number" + ) { + return row; + } + return { ...row, hybrid_score: Math.min(0.4999, 0.18 + 0.3 + 0.02 * saturationTailUnit(textRank)) }; +} + export type ChunkSignalMatch = { chunkId: string; similarity: number; @@ -991,11 +1050,12 @@ export async function searchTableFactCandidates(args: { const textRank = Number(fact.text_rank ?? 0); const current = grouped.get(fact.source_chunk_id); const tableFact = { ...fact, text_rank: textRank, match_reason: fact.match_reason ?? "table_row" }; + const imputed = imputedTableFactPrimaries(textRank); const next: ChunkSignalMatch = { chunkId: fact.source_chunk_id, - similarity: Math.min(0.94, 0.62 + Math.min(textRank, 1) * 0.3), + similarity: imputed.similarity, textRank, - hybridScore: Math.min(0.97, 0.66 + Math.min(textRank, 1) * 0.3), + hybridScore: imputed.hybridScore, reason: fact.match_reason ?? "table_row", tableFacts: [...(current?.tableFacts ?? []), tableFact].slice(0, 5), }; diff --git a/src/lib/rag/rag.ts b/src/lib/rag/rag.ts index 6425d2439..335df7cc4 100644 --- a/src/lib/rag/rag.ts +++ b/src/lib/rag/rag.ts @@ -831,9 +831,11 @@ function buildRetrievalDiagnostics(args: { }) { // Lexical-only retrieval rows carry a truthful score contract since migration // 20260713062107_restore_text_fallback_lexical_score: similarity is 0 (no vector - // ran) and hybrid_score is deliberately capped at 0.48 so a keyword hit can never - // masquerade as a moderate/strong cosine match downstream. The honest lexical - // signal lives in lexical_score (0.4..0.99). This gate must therefore read + // ran) and hybrid_score stays strictly below 0.5 — 0.48 from SQL, plus a sub-0.5 + // per-candidate saturation spread (liftSaturatedLexicalChunkHybrid, Phase C) that + // only disambiguates rows whose text_rank saturated the SQL clamp — so a keyword + // hit can never masquerade as a moderate/strong cosine match downstream. The honest + // lexical signal lives in lexical_score (0.4..0.99). This gate must therefore read // max(scoreValue, lexical_score) — reading the capped hybrid_score alone makes // topScore < 0.5 unconditional for every text-fast-path answer, refusing // well-supported documentation lookups whose expected document is at rank 1. diff --git a/tests/rag-fast-path-ordering.test.ts b/tests/rag-fast-path-ordering.test.ts index f1ed951db..654f2c735 100644 --- a/tests/rag-fast-path-ordering.test.ts +++ b/tests/rag-fast-path-ordering.test.ts @@ -1,5 +1,10 @@ import { describe, expect, it } from "vitest"; import { applySecondStageRerankIfNeeded } from "../src/lib/rag/rag"; +import { + imputedTableFactPrimaries, + liftSaturatedLexicalChunkHybrid, + saturationTailUnit, +} from "../src/lib/rag/rag-candidate-sources"; import { classifyRagQuery, rankClinicalResults } from "../src/lib/clinical-search"; import { selectRetrievalEvidence } from "../src/lib/retrieval-selection"; import { resultsHaveReleaseRankScore, stabilizeReleasedSearchOrder } from "../src/lib/released-search-order"; @@ -20,18 +25,25 @@ import type { SearchResult } from "../src/lib/types"; type QueryClass = ReturnType["queryClass"]; +// Fixtures derive their primaries from the production imputation helper so they stay +// byte-accurate to whatever the fast path actually emits. At the historical default +// text_rank 0.45 (sub-knee) this yields exactly the former hardcoded pair +// similarity 0.755 / hybrid 0.795, keeping every pre-Phase-C guard's +// "byte-identical primaries" premise intact. function imputedResult( overrides: Partial & Pick, ): SearchResult { + const textRank = typeof overrides.text_rank === "number" ? overrides.text_rank : 0.45; + const imputed = imputedTableFactPrimaries(textRank); return { page_number: 5, chunk_index: 0, section_heading: null, image_ids: [], images: [], - similarity: 0.755, - text_rank: 0.45, - hybrid_score: 0.795, + similarity: imputed.similarity, + text_rank: textRank, + hybrid_score: imputed.hybridScore, ...overrides, }; } @@ -383,3 +395,183 @@ describe("text-fast-path ordering under imputed identical primaries", () => { expect(released.map((result) => result.document_id)).toContain("archived-doc"); }); }); + +describe("Phase C saturation-tail primaries", () => { + it("keeps sub-knee imputation byte-identical to the historical formulas", () => { + for (const textRank of [0, 0.1, 0.25, 0.45, 0.7, 0.999, 1]) { + const { similarity, hybridScore } = imputedTableFactPrimaries(textRank); + expect(similarity).toBe(Math.min(0.94, 0.62 + Math.min(textRank, 1) * 0.3)); + expect(hybridScore).toBe(Math.min(0.97, 0.66 + Math.min(textRank, 1) * 0.3)); + expect(saturationTailUnit(textRank)).toBe(0); + } + // The historical hardcoded fixture pair reproduces exactly at the default text_rank. + expect(imputedTableFactPrimaries(0.45)).toEqual({ similarity: 0.755, hybridScore: 0.795 }); + }); + + it("confines the saturated similarity tail to the dead (0.92, 0.94) band and leaves hybrid untouched", () => { + let previous = 0.92; + for (const textRank of [1.0001, 1.05, 1.5, 3, 50]) { + const { similarity, hybridScore } = imputedTableFactPrimaries(textRank); + expect(similarity).toBeGreaterThan(0.92); + expect(similarity).toBeLessThan(0.94); + // hybrid must stay byte-identical so gates, triggers, and selection scores cannot move. + expect(hybridScore).toBe(0.96); + // strict monotonicity in raw text relevance + expect(similarity).toBeGreaterThan(previous); + previous = similarity; + expect(saturationTailUnit(textRank)).toBeGreaterThan(0); + expect(saturationTailUnit(textRank)).toBeLessThan(1); + } + }); + + it("lifts saturated lexical chunks only under the truthful-contract signature and stays below 0.5", () => { + const base = imputedResult({ + id: "chunk-a", + document_id: "doc-a", + title: "Reference Compendium", + file_name: "ReferenceCompendium.pdf", + content: "content", + }); + const lexicalRow: SearchResult = { ...base, similarity: 0, lexical_score: 0.99, hybrid_score: 0.48 }; + + const lifted = liftSaturatedLexicalChunkHybrid({ ...lexicalRow, text_rank: 1.9 }); + expect(lifted.hybrid_score).toBeGreaterThan(0.48); + expect(lifted.hybrid_score).toBeLessThan(0.5); + const liftedLower = liftSaturatedLexicalChunkHybrid({ ...lexicalRow, text_rank: 1.1 }); + expect(liftedLower.hybrid_score).toBeGreaterThan(0.48); + expect(lifted.hybrid_score!).toBeGreaterThan(liftedLower.hybrid_score!); + + // Non-matching signatures pass through unmodified: real-vector rows, sub-knee rows, + // rows without the lexical contract. + expect(liftSaturatedLexicalChunkHybrid({ ...lexicalRow, similarity: 0.5, text_rank: 1.9 }).hybrid_score).toBe(0.48); + expect(liftSaturatedLexicalChunkHybrid({ ...lexicalRow, text_rank: 0.9 }).hybrid_score).toBe(0.48); + expect( + liftSaturatedLexicalChunkHybrid({ ...lexicalRow, lexical_score: undefined, text_rank: 1.9 }).hybrid_score, + ).toBe(0.48); + }); + + it("resolves a saturated tie by raw text_rank instead of chunk-id order (the discriminating case)", () => { + // Both candidates saturate the historical clamp (text_rank >= 1) and tie on every + // pre-Phase-C key: identical content (identical coverage and boost signals), no table + // facts or images, query-token-free titles, no governance metadata — and ids chosen so + // the chunk-id fallback would seat the DISTRACTOR first. Under the pre-Phase-C + // byte-identical primaries this ordering was arbitrary (id order); the saturation tail + // makes the strictly more text-relevant document win. Verified red on the old formulas. + // + // Scope note (matches the live headroom evidence): the tail resolves ties at the + // release comparator, which is consulted when the second-stage rerank does NOT engage — + // exactly the pools where canary #53/#54 show id-order deciding (patient-safety, opioid, + // flowchart). A table_threshold pool with tied top scores always engages the second + // stage, whose position-derived releaseRankScore sorts first, so this case deliberately + // uses a visual-evidence-free document_lookup pool and pins the non-engaged premise. + const query = "What should a patient safety plan include?"; + const { queryClass } = classifyRagQuery(query); + expect(queryClass).toBe("document_lookup"); + const sharedContent = + "A patient safety plan should include warning signs, coping strategies, and emergency contacts."; + + const distractor = imputedResult({ + id: "aa-distractor", + document_id: "aa-generic-doc", + title: "Service Reference Compendium", + file_name: "ServiceReferenceCompendium (NMHS).pdf", + content: sharedContent, + text_rank: 1.05, + }); + const correct = imputedResult({ + id: "zz-correct", + document_id: "zz-relevant-doc", + title: "Ward Reference Compendium", + file_name: "WardReferenceCompendium (SMHS).pdf", + content: sharedContent, + text_rank: 1.8, + }); + + const released = runTextFastPathOrdering({ + query, + queryClass, + candidates: [distractor, correct], + topK: 8, + }); + // Non-engaged premise: no visual evidence in the pool, so release order falls through + // hybrid (tied) to the Phase C similarity spread. + expect(resultsHaveReleaseRankScore(released)).toBe(false); + expect(released[0]?.document_id).toBe("zz-relevant-doc"); + expect(released.map((result) => result.document_id)).toContain("aa-generic-doc"); + + // Arrival order must not matter either. + const releasedReversed = runTextFastPathOrdering({ + query, + queryClass, + candidates: [correct, distractor], + topK: 8, + }); + expect(releasedReversed[0]?.document_id).toBe("zz-relevant-doc"); + }); + + it("still ties genuinely equal text_ranks so the coverage comparator keeps deciding (#987 contract)", () => { + // Identical text_rank -> identical primaries under the tail too; the coverage-richer + // candidate must win regardless of adversarial id order, proving the tail cannot + // starve the content-aware tie-break. + const query = "What CIWA-Ar score threshold requires drug treatment in alcohol withdrawal?"; + const { queryClass } = classifyRagQuery(query); + + const coveragePoor = imputedResult({ + id: "aa-poor-coverage", + document_id: "aa-poor-doc", + title: "Withdrawal Care Overview", + file_name: "WithdrawalCareOverview (NMHS).pdf", + content: "General supportive care principles for patients in alcohol withdrawal.", + text_rank: 1.6, + }); + const coverageRich = imputedResult({ + id: "zz-rich-coverage", + document_id: "zz-rich-doc", + title: "Withdrawal Scale Reference", + file_name: "WithdrawalScaleReference (SMHS).pdf", + content: "A CIWA-Ar score of 10 or more is the threshold that requires drug treatment in alcohol withdrawal.", + text_rank: 1.6, + }); + + expect(imputedTableFactPrimaries(1.6)).toEqual(imputedTableFactPrimaries(1.6)); + const released = runTextFastPathOrdering({ + query, + queryClass, + candidates: [coveragePoor, coverageRich], + topK: 8, + }); + expect(released[0]?.document_id).toBe("zz-rich-doc"); + }); + + it("orders saturated lexical text chunks by raw text_rank through the release pipeline", () => { + const query = "What ANC threshold requires review?"; + const { queryClass } = classifyRagQuery(query); + const sharedContent = "The ANC threshold requires review whenever results fall in the flagged range."; + const lexicalChunk = (id: string, documentId: string, title: string, textRank: number): SearchResult => + liftSaturatedLexicalChunkHybrid({ + ...imputedResult({ + id, + document_id: documentId, + title, + file_name: `${title.replace(/\s+/g, "")}.pdf`, + content: sharedContent, + text_rank: textRank, + }), + similarity: 0, + lexical_score: 0.99, + hybrid_score: 0.48, + }); + + const released = runTextFastPathOrdering({ + query, + queryClass, + candidates: [ + lexicalChunk("aa-low", "aa-low-doc", "Service Reference Compendium", 1.1), + lexicalChunk("zz-high", "zz-high-doc", "Ward Reference Compendium", 1.9), + ], + topK: 8, + }); + expect(released[0]?.document_id).toBe("zz-high-doc"); + expect(released.map((result) => result.document_id)).toContain("aa-low-doc"); + }); +});