feat(rag): ship D4/D5 governance levers behind default-off gates

[BigSimmo]

Audit 2026-07-13 deferred items D4 + D5, completed as zero-behavior-change plumbing per the #118 lesson (no governance weighting without golden-eval proof) — the levers ship OFF and are activated by the operator only behind a green golden eval once OpenAI quota recovers.

• D4: secondStage.unknownCurrentnessPenalty in ranking-config (default 0, clamped non-negative) applied in the second-stage rerank when document_status is 'unknown'. Activation = RAG_RANKING_CONFIG JSON override, the module's designed eval-gated tuning path.
• D5: deriveTrust broadening behind NEXT_PUBLIC_RAG_TRUST_CAP_ALL_CLAIMS (unset = false): when on, unverified-authority evidence caps trust to medium for ALL supported claims, not just high-risk ones. NEXT_PUBLIC because answer-render-policy renders client-side.
• Tests lock both the zero-change defaults and the flag-on behavior.

Verified: verify:cheap green (2,310 tests), eval:rag:offline green.

Summary

Verification

• npm run verify:pr-local

During development, use npm run verify:cheap as the faster iteration gate before the final PR-local preflight.

• npm run verify:ui when UI, routing, styling, browser behavior, reduced-motion, or forced-colors behavior changed
• npm run verify:release before release or handoff confidence claims

For retrieval, ranking, selection, chunking, source/citation rendering, or answer-contract changes, verify:pr-local runs eval:rag:offline automatically. Run the offline command directly during iteration before spending a live eval.

• npm run eval:retrieval:quality (must stay 36/36) when retrieval, ranking, selection, chunking, or scoring behavior changed — CI cannot run it (needs live keys), so run it locally and paste the summary. A metadata/governance-weighting change once buried correct docs (recall 1.0→0.76) and only this eval caught it.
• npm run eval:rag -- --limit 15 + npm run eval:quality -- --rag-only when answer generation, the synthesis prompt, or answer post-processing changed (grounded-supported must not drop; citation-failure 0)
• npm run check:production-readiness when clinical workflow, privacy, environment, Supabase, source governance, or deployment behavior changed
• npm run check:deployment-readiness when deployment startup, hosting, or rollout behavior changed

Clinical Governance Preflight

Complete this section when the change touches ingestion, answer generation, search/ranking, source rendering, document access, privacy, production env, or clinical output.

• Source-backed claims still require linked source verification before clinical use
• No patient-identifiable document workflow was introduced or expanded without explicit governance approval
• Supabase target remains Clinical KB Database (sjrfecxgysukkwxsowpy)
• Service-role keys and private document access remain server-only
• Demo/synthetic content remains clearly separated from real clinical sources
• Source metadata, review status, and outdated/unknown-source behavior remain conservative
• Deployment classification/TGA SaMD impact was checked when clinical decision-support behavior changed

Notes

Summary by CodeRabbit

• New Features

  • Added configurable trust limits for claims supported by unverified evidence.
  • Added ranking support for penalizing results with unknown document freshness.

• Bug Fixes

  • Improved result ranking by accounting for documents with unknown currentness.
  • Ensured ranking penalties cannot use negative values.

• Tests

  • Added coverage for configurable trust behavior and unknown-document ranking penalties.

[supabase]

This pull request has been ignored for the connected project sjrfecxgysukkwxsowpy because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.

---

Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: c3bfa010-d9db-4e3c-96e3-12b64ad42e70

📥 Commits

Reviewing files that changed from the base of the PR and between 550e058 and bba80a9.

📒 Files selected for processing (5)

• src/lib/answer-render-policy.ts
• src/lib/rag.ts
• src/lib/ranking-config.ts
• tests/answer-render-policy.test.ts
• tests/ranking-config.test.ts

---

📝 Walkthrough

Walkthrough

Trust derivation now supports environment-controlled gating for all claims, while second-stage ranking supports a configurable penalty for unknown document status. Tests cover the new trust flag and ranking configuration behavior.

Changes

Trust gating

Layer / File(s)	Summary
Authority evidence trust cap src/lib/answer-render-policy.ts, tests/answer-render-policy.test.ts	deriveTrust gates either all supported claims or only high-risk claims based on NEXT_PUBLIC_RAG_TRUST_CAP_ALL_CLAIMS; tests cover both modes and clean up the stubbed environment.

Ranking currentness

Layer / File(s)	Summary
Unknown currentness penalty src/lib/ranking-config.ts, src/lib/rag.ts, tests/ranking-config.test.ts	Second-stage ranking subtracts the configured penalty for unknown document status; configuration defaults, JSON overrides, and negative-value clamping are tested.

Estimated code review effort: 3 (Moderate) | ~20 minutes

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch claude/d4d5-eval-gated-flags-2026-07-14

---

_{Comment @coderabbitai help to get the list of available commands.}