Revert "feat(rag): per-candidate saturation-tail primaries" pending retrieval review (#1004)

[BigSimmo]

Summary

User-requested revert of #1004 (f96217c): auto-merge fired before the adversarial rag-retrieval-reviewer pass on the diff had reported, so the retrieval path returns to its last fully-reviewed state until that verdict is in. This is the exact single-commit rollback documented in #1004's risk section — pure helpers + two expression sites + one map call; no schema, config, cache, or workflow surface.

• 82426c7 — clean git revert of the squash
• b5c4eb9 — retains the append-only ledger history rows from feat(rag): per-candidate saturation-tail primaries on the text fast path (ADDENDUM 4 Phase C) #1004 (record, not behavior) and adds the revert's own ledger row

Standing evidence, unaffected by this revert: the offline proof of the reverted change (red-proof, 121/121 targeted, envelope tests) remains valid, and the 19:44 UTC post-merge canary dispatched on f96217c completes regardless — it stands as Phase C's live validation datapoint. Re-land decision = reviewer verdict + that canary pair, together.

Verification

• Clean revert (no conflicts); worktree returns to pre-feat(rag): per-candidate saturation-tail primaries on the text fast path (ADDENDUM 4 Phase C) #1004 content plus ledger rows
• Hosted pr-required on this PR re-proves the pre-C baseline (the 5-test fast-path suite shape)
• Verification not run: local vitest re-run — the revert restores exactly the tree CI validated on main before feat(rag): per-candidate saturation-tail primaries on the text fast path (ADDENDUM 4 Phase C) #1004; hosted CI is the gate

Risk and rollout

• Risk: minimal — restores the previously-validated main state. The only asymmetry is the retained ledger rows (docs-only).
• Rollback: revert this PR (which re-lands Phase C) — the intended path if the reviewer approves and the canary pair is green.
• Provider or production effects: None.

Clinical Governance Preflight

• Source-backed claims still require linked source verification before clinical use — restored to the reviewed state
• No patient-identifiable document workflow was introduced or expanded
• Supabase target remains Clinical KB Database (sjrfecxgysukkwxsowpy)
• Service-role keys and private document access remain server-only
• Demo/synthetic content remains clearly separated from real clinical sources
• Source metadata, review status, and outdated/unknown-source behavior remain conservative — the revert restores the previously-reviewed ordering behavior in full
• Deployment classification/TGA SaMD impact was checked: reverting to the prior reviewed retrieval behavior; no new clinical decision-support behavior

Notes

• Re-land path on approval: revert this PR (re-lands Phase C with both proofs in hand).

🤖 Generated with Claude Code

https://claude.ai/code/session_01EXsJcLrbZUXwnBeG91cVo9

Summary by CodeRabbit

• Bug Fixes

  • Rolled back the Phase C retrieval scoring change to restore the last reviewed retrieval behavior.
  • Lexical search results now retain their returned scoring without additional score adjustments.
  • Table-fact similarity scoring no longer adds saturation-tail boosts.
  • Synthetic-only evidence remains capped at a medium confidence level.

• Documentation

  • Updated retrieval review records and findings to reflect the rollback and current scoring behavior.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

[supabase]

This pull request has been ignored for the connected project sjrfecxgysukkwxsowpy because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.

---

Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

[coderabbitai]

📝 Walkthrough

Walkthrough

This change reverts Phase C saturation-tail retrieval scoring, removes its exported helpers and tests, updates table-fact scoring and lexical diagnostics documentation, and records the rollback in review documentation.

Changes

Retrieval scoring rollback

Layer / File(s)	Summary
Remove saturation-tail candidate scoring src/lib/rag/rag-candidate-sources.ts, src/lib/rag/rag.ts	Lexical chunk results no longer receive saturation lifting; saturation helpers are removed; table-fact similarity and hybrid scores are computed without the saturation-tail term.
Update retrieval ordering validation tests/rag-fast-path-ordering.test.ts	Fixtures use fixed historical primaries, and the Phase C saturation-tail test suite is removed.
Record rollback and score semantics docs/branch-review-ledger.md, docs/rag-hybrid-findings-and-todo.md	Documentation records the rollback and clarifies lexical and synthetic similarity sources.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

• BigSimmo/Database#1004: Introduced the Phase C saturation-tail scoring reverted here.
• BigSimmo/Database#641: Added the candidate-source retrieval functions modified by this rollback.

Suggested labels: codex

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly states this is a revert of the saturation-tail primaries change pending retrieval review, matching the main change.
Description check	✅ Passed	The description follows the template with Summary, Verification, Risk, Governance, and Notes, and includes concrete rollback and verification details.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch claude/clinical-kb-pwa-review-asi3wb

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

src/lib/rag/rag.ts (1)

832-840: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Use 0.5 here, not 0.48. match_document_chunks_text caps hybrid_score at 0.5, so this comment should match the RPC value to avoid confusing the topScore < 0.5 gate.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/rag/rag.ts` around lines 832 - 840, Update the explanatory comment
near the topScore gate in the text-only retrieval path to state that
match_document_chunks_text caps hybrid_score at 0.5, replacing the outdated 0.48
reference while preserving the rest of the score-contract explanation.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@src/lib/rag/rag.ts`:
- Around line 832-840: Update the explanatory comment near the topScore gate in
the text-only retrieval path to state that match_document_chunks_text caps
hybrid_score at 0.5, replacing the outdated 0.48 reference while preserving the
rest of the score-contract explanation.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: acaffb45-a3c8-4dd6-b848-806e79b64e5e

📥 Commits

Reviewing files that changed from the base of the PR and between f96217c and b5c4eb9.

📒 Files selected for processing (5)

• docs/branch-review-ledger.md
• docs/rag-hybrid-findings-and-todo.md
• src/lib/rag/rag-candidate-sources.ts
• src/lib/rag/rag.ts
• tests/rag-fast-path-ordering.test.ts