fix(rag): retain live hybrid fallback

[BigSimmo]

Final release verification

• Exact-head hosted CI passed on 9152ca62d7200fdb6e048d2b76546faee4f76326: https://github.com/BigSimmo/Database/actions/runs/29683207386
• Repository automation merged the PR as 39378863a5d713bfdeb617377a90319ae75810d4.
• Exact-main live canary passed: https://github.com/BigSimmo/Database/actions/runs/29683303597
  • Retrieval: 36/36; document/content recall@5 1.0; hit rate 1.0; MRR@10 0.8505; content MRR@10 0.9159; nDCG@10 0.917; irrelevant-source rate@10 0.1083; required-signal coverage@10 1.0; zero failed cases.
  • Live answer subset: 8/8; grounded-supported and expected-source hit rates 1.0; citation/numeric-grounding/danger/route-ceiling failure rates 0; no threshold failures.
• Supabase project Clinical KB Database (sjrfecxgysukkwxsowpy) remains ACTIVE_HEALTHY; deployed migrations are recorded and both edge functions are active.

Summary

• Preserve clinical order only for the late-review scenario where second-stage release ordering was requested but the final result set has no valid bounded scores.
• Restore the live-proven raw-hybrid fallback for ordinary result sets without release scores.
• Keep strongest-copy deduplication, valid bounded release ordering, and finite-score validation unchanged.

Verification

• Prettier applied to both changed files
• git diff --check
• Regression coverage distinguishes sticky second-stage fallback from ordinary raw-hybrid fallback
• Focused local Vitest (currently queued behind another legitimate Database Playwright lock)
• Required hosted CI
• Exact-head live retrieval and answer-quality canary

Live evidence:

• fix(rag): preserve selected order without release scores #920 merge canary: https://github.com/BigSimmo/Database/actions/runs/29682430068 — 35/36; only patient-safety-plan-include failed because the expected PtSafetyPlan document fell from top five after the broad preserve-order change.
• Pre-fix(rag): preserve selected order without release scores #920 exact-main canary: https://github.com/BigSimmo/Database/actions/runs/29681465785 — 36/36 retrieval and 8/8 live answer cases.

Risk and rollout

• Risk: low and isolated to choosing the final ordering comparator when no release score exists.
• Rollback: revert this commit to preserve all no-score result sets in their incoming order.
• Provider or production effects: no production data mutation. The authorized canary reads the live Clinical KB corpus and calls configured evaluation providers.

Clinical Governance Preflight

• Source-backed claims still require linked source verification before clinical use
• No patient-identifiable document workflow was introduced or expanded without explicit governance approval
• Supabase target remains Clinical KB Database (sjrfecxgysukkwxsowpy)
• Service-role keys and private document access remain server-only
• Demo/synthetic content remains clearly separated from real clinical sources
• Source metadata, review status, and outdated/unknown-source behavior remain conservative
• Deployment classification/TGA SaMD impact was checked when clinical decision-support behavior changed

Summary by CodeRabbit

• Bug Fixes
  • Improved deduplicated search result ordering with smarter fallback when second-stage release ranking signals (release scores) are missing.
  • Added optional behavior to preserve the incoming semantic rerank order when requested, even if release scores aren’t available.
  • Updated second-stage stabilization to consistently use the best available ordering signal (bounded release score when available; hybrid strength fallback otherwise).
• Tests
  • Refreshed and expanded ordering test coverage for mixed and missing release-score scenarios, including preservation of semantic rerank order.

[supabase]

This pull request has been ignored for the connected project sjrfecxgysukkwxsowpy because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.

---

Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: c12a143b-e1f7-4526-9226-d1bd264e8cf6

📥 Commits

Reviewing files that changed from the base of the PR and between 14149d2 and 9152ca6.

📒 Files selected for processing (1)

• src/lib/rag.ts

🚧 Files skipped from review as they are similar to previous changes (1)

• src/lib/rag.ts

---

📝 Walkthrough

Walkthrough

stabilizeReleasedSearchOrder now supports preserving incoming order, gates second-stage release sorting on usable scores, and falls back to hybrid-strength ordering when appropriate. Telemetry-driven semantic rerank preservation and corresponding tests were added.

Changes

Released search ordering

Layer / File(s)	Summary
Ordering decision logic src/lib/rag.ts	Adds incoming-order preservation, gates release-score comparison on active second-stage ordering, applies hybrid-strength fallback sorting, and passes semantic rerank preservation state from telemetry.
Ordering behavior validation tests/rag-second-stage-ranking.test.ts	Verifies preserved second-stage order, hybrid fallback ordering without release scores, and explicit preservation of semantic rerank order.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

• BigSimmo/Database#901: Introduces the semantic reranking telemetry used by this ordering preservation path.
• BigSimmo/Database#919: Modifies the same second-stage release ordering and missing-score behavior.
• BigSimmo/Database#920: Adjusts stabilizer sorting and tests for missing or invalid release scores.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is concise and accurately summarizes the hybrid fallback change.
Description check	✅ Passed	The description follows the template closely and covers summary, verification, risk, and governance; only the optional Notes section is missing.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/restore-hybrid-fallback

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

CI triage

CI failed on this PR. Automated classification of the 2 failed job(s):

• Unit coverage — needs investigation: inspect the failing step and uploaded diagnostics; rerun only after classifying the cause.
• PR required — needs investigation: inspect the failing step and uploaded diagnostics; rerun only after classifying the cause.

Compared with main CI run #3559 (success).

Classification is evidence routing, not permission to ignore a failure. Exact quarantined Playwright identities remain governed by the flake ledger.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ba812925e2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[BigSimmo]

@codex resolve actionable Codex review findings for this pull request and current head using the repository instructions. This is the pull request's single automatic repair pass: do not perform a fresh review, create new standalone findings, or request another review. Work only the existing unresolved Codex threads on the current head. Always fix P0 and P1 findings. For P2 and lower findings, fix only clear, scoped, low-risk issues; otherwise disposition them with a concise reason. After fixing or dispositioning a thread, reply in that thread with as the first line, followed by a concise summary; that marker authorizes the workflow to close that exact thread. If human input or new authorization is required, do not use the marker and leave the thread open with the blocker. Finish only after every actionable thread is fixed or dispositioned and closed, or explicitly left open for a human decision. Do not update the branch from main, address unrelated reviews, broaden scope, or create more than one scoped fix commit. Do not use external APIs, paid services, credentials, dependency changes, or broad refactors unless explicitly authorized. Add targeted tests where behavior changes and run the narrowest relevant validation.

[BigSimmo]

@cursoragent Please can you resolve the failing CI checks for me. Please can you also fix all of the comments and mark them resolved once solved.

[cursor]

Taking a look!

  

[BigSimmo]

@cursoragent Please can you resolve the failing CI checks for me. Please can you also fix all of the comments and mark them resolved once solved.

[cursor]

Summary

CI is green on PR #921, and both Codex review threads are fixed and resolved.

Root cause (P1 — failing CI)

When preferSecondStageScore was false, stabilizeReleasedSearchOrder still sorted with compareReleasedSearchOrder, which prefers releaseRankScore over raw hybrid score. That broke the test expecting hybrid-leading (0.84) to rank above release-ranked (0.79 hybrid / 0.95 release).

Fixes applied

1. P1 — raw hybrid fallback restored
   When release ordering is inactive, results now sort by compareReleasedHybridStrength instead of compareReleasedSearchOrder.

2. P2 — semantic rerank order preserved
   recordSearchScoreTelemetry passes preserveIncomingOrder=true when semantic_rerank_outcome === "reordered" and there are no bounded release scores, so semantic rerank order survives deduplication.

3. Formatting
   Prettier formatting on src/lib/rag.ts (already on the branch as 9152ca62).

Review comments

Both Codex threads were replied to with <!-- codex-thread-disposition:resolved --> and marked resolved:

• P1: Ignore release scores in raw fallback
• P2: Preserve semantic rerank order before stabilizing

CI status

All required checks pass, including:

• Static PR checks
• Unit coverage
• PR required

Verification run locally

• tests/rag-second-stage-ranking.test.ts — 9/9 passed
• npm run format:check — passed