fix(rag): exempt deterministic document-list/table answers from prose sanitizer

[BigSimmo]

Completes item #1 from the answer-quality triage — the direct-document-lookup-nocc failure.

Root cause (traced end-to-end)

buildDocumentSupportListAnswer / buildTableOrVisualSourceLookupAnswer build deterministic, template answers from source metadata — not model prose. But finalizeRagAnswerQualityCore ran them through sanitizeAnswerText, a clinical-prose extractor. It stripped their document names (facility-code suffixes like (NOCC)(AKG) and semicolon-separated entries read as non-prose), turning a valid answer:

"I found 5 indexed documents…: National Outcomes and Casemix Collection (NOCC)(AKG); …; and Mental Health Medically Cleared For Discharge Procedure (EMHS Policy)."

into garble the quality gate then correctly refused — so the case had 5 citations but grounded=false. (Verified with a QGATE_DEBUG RAW-vs-CLEAN dump: polishClinicalAnswerProse alone is clean; sanitizeStructuredText does the mangling.)

Fix

A preformatted flag on RagAnswer, set by the two deterministic builders and propagated by buildExtractiveAnswer. finalizeRagAnswerQualityCore returns preformatted grounded answers untouched — they're well-formed by construction and carry no free-text clinical claims, so the prose sanitizer/gate is neither needed nor safe for them. The shared sanitizer and quality gate are unchanged for model prose (the "don't touch the load-bearing sanitizer" constraint).

Scope (deliberately narrow)

Fixes the document-list case only. agitation-arousal-table-lookup is queryClass: table_threshold, which routes to fact synthesis (not a structured builder) and produces genuinely incoherent table-cell text the gate correctly refuses — a separate synthesis-quality issue, intentionally left out.

Verification (full golden RAG eval, live Supabase + OpenAI)

• NOCC now grounded (extractive, 5 citations, no failures).
• grounded_supported_rate 0.933 → 0.967.
• No regression from this change. The one unsupported flip in the eval run (unsupported-close-title-noise) is pre-existing LLM variance, not this fix: it is preformatted: None (this fix never touches its path) and refuses 3/3 on re-run.
• New unit test asserts facility-code suffixes survive intact; tests/rag-answer-fallback.test.ts 26/26; typecheck / lint / prettier clean.

🤖 Generated with Claude Code

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bb4da7d9fe

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Preserve preformatted answers through the client renderer

When this path returns a document-list answer intact, the browser still re-sanitizes answer.answer: I checked ClinicalDashboard.tsx, where safeAnswerText is always built with sanitizeAnswerDisplayText(answer?.answer ?? "") and passed as safeAnswerText || answer.answer before NaturalLanguageAnswer calls primaryAnswerDisplayText, which also uses the display/prose sanitizers. For the fixed NOCC-style query the API answer now keeps (NOCC) (AKG), but the rendered answer can still strip or mangle that suffix before display, so the user-facing regression remains unless the renderer honors preformatted or skips display sanitization for it.

Useful? React with 👍 / 👎.