RAG routing hardening, registry corpus embedding, and review docs by BigSimmo · Pull Request #443 · BigSimmo/Database

BigSimmo · 2026-07-09T11:02:34Z

Summary

Harden RAG answer routing for clozapine blood-count/withhold threshold queries: require blood-count and withhold/stop language in the same sentence before treating retrieval as explicit action evidence, and block table_threshold extractive shortcuts when retrieval gaps/conflicts are actionable.
Extract answer-quality helpers into src/lib/rag-extractive-answer.ts and restore default OpenAI SDK retry budget for answer generation (remove maxRetries: 0).
Add registry corpus embedding tooling (scripts/embed-registry-records.ts, src/lib/registry-corpus.ts) and document hybrid-RAG/registry findings in docs/rag-hybrid-findings-and-todo.md.
Add scheduled/manual live-drift.yml workflow with service-role key scoped only to drift-check steps (not npm ci).
Add JSDoc coverage across PR-touched RAG/registry modules to satisfy governance docstring threshold.

Verification

npm run verify:cheap — lint, typecheck, and 1412 unit tests passed locally (2026-07-09)
npm run verify:ui when UI, routing, styling, browser behavior, reduced-motion, or forced-colors behavior changed
npm run verify:release before release or handoff confidence claims
npm run format:check — passed after Prettier on touched source files
npm run eval:retrieval:quality (must stay 36/36) when retrieval, ranking, selection, chunking, or scoring behavior changed — ran locally as npm run eval:retrieval:quality -- --force-embedding: passed with document_recall_at_5=1, content_recall_at_5=1, top_k_hit_rate=1, force_embedding_failure_count=0, failed_cases=[] (documented in docs/rag-hybrid-findings-and-todo.md)
npm run eval:rag -- --limit 15 + npm run eval:quality -- --rag-only when answer generation, the synthesis prompt, or answer post-processing changed (grounded-supported must not drop; citation-failure 0)
npm run check:production-readiness when clinical workflow, privacy, environment, Supabase, source governance, or deployment behavior changed
npm run check:deployment-readiness when deployment startup, hosting, or rollout behavior changed

Clinical Governance Preflight

Complete this section when the change touches ingestion, answer generation, search/ranking, source rendering, document access, privacy, production env, or clinical output.

Source-backed claims still require linked source verification before clinical use
No patient-identifiable document workflow was introduced or expanded without explicit governance approval
Supabase target remains Clinical KB Database (sjrfecxgysukkwxsowpy)
Service-role keys and private document access remain server-only
Demo/synthetic content remains clearly separated from real clinical sources
Source metadata, review status, and outdated/unknown-source behavior remain conservative
Deployment classification/TGA SaMD impact was checked when clinical decision-support behavior changed

Notes

Post-registry embed retrieval eval command recorded explicitly: npm run eval:retrieval:quality -- --force-embedding (36/36, recall metrics 1.0).
Registry corpus write path still uses separate documents + document_chunks upserts with client-side rollback on chunk failure; atomic Postgres RPC remains a follow-up (CodeRabbit data-integrity comment deferred to avoid scope creep in this docs/routing PR).
Remaining product blockers from registry embed workstream: citation failure rate 0.0227 and RAG latency thresholds — not introduced by this routing/docs pass.

…ft manifest

…ithout GHAS

… claude/llm-pipeline-review

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

…20260708160001

- Defer embedding until after text/document fast paths and coverage gate - Rehydrate cached document metadata in attachDocumentRankingMetadata - Load env-dependent script imports after loadEnvConfig in seed/reindex scripts - Harden registry corpus: shared identity, medication tags, rollback, detail hrefs - Route registry citations to detail pages; handle registry rows in signed-url API - Prioritize safety warnings over registry info; preserve stale registry labels - Guard OCR repair against dropping isolated single-letter clinical tokens - Update skill docs, changelog dedup, CI Supabase setup-cli@v3 Co-authored-by: BigSimmo <BigSimmo@users.noreply.github.com>

- Backfill NULL document_images.index_generation_id during re-stamp - Count globally forced embedding eval cases in retrieval summaries Co-authored-by: BigSimmo <BigSimmo@users.noreply.github.com>

Treat NULL document_images.index_generation_id as stale even when JSON metadata already matches the committed generation, and propagate global --force-embedding into eval reporting/validation so index-unit-vector runs count forced cases correctly. Co-authored-by: BigSimmo <BigSimmo@users.noreply.github.com>

… codex/pr433-fix # Conflicts: # .cursor/skills/supabase-postgres-best-practices/CHANGELOG.md # .cursor/skills/supabase-postgres-best-practices/references/security-rls-performance.md # scripts/reindex-image-generation-metadata.ts # src/lib/differential-seed.ts # src/lib/document-index-units.ts # src/lib/medication-seed.ts # src/lib/rag.ts # src/lib/registry-corpus.ts # src/lib/registry-seed.ts # src/lib/source-governance.ts # src/lib/source-metadata.ts # supabase/migrations/20260708160000_retrieval_owner_matches_fail_closed.sql # supabase/migrations/20260708160001_retrieval_owner_matches_fail_closed.sql # supabase/migrations/20260708161000_retrieval_owner_matches_fail_closed.sql # tests/document-index-units.test.ts # tests/registry-corpus.test.ts # tests/source-governance.test.ts

… codex/pr433-fix

PR #433 preview branch recorded 20260708160000 before the file was removed, causing remote migration versions not found in local migrations directory. Keep a no-op stub for history sync; transactional index DDL stays on 170000. Co-authored-by: Cursor <cursoragent@cursor.com>

… codex/pr433-fix

* fix(rag): finish registry re-embed and quality routing * fix: resolve merge conflicts with claude/llm-pipeline-review --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>

… claude/llm-pipeline-review # Conflicts: # tests/rag-routing.test.ts

coderabbitai · 2026-07-09T11:02:43Z

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

@coderabbitai resume to resume automatic reviews.
@coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

▶️ Resume reviews
🔍 Trigger review

📝 Walkthrough

Walkthrough

This PR updates RAG routing and extractive gating, adds registry re-embed tooling and CLI owner listing, and introduces drift-check, schema permission, and review-process documentation changes.

Changes

Clinical RAG retrieval and routing behavior

Layer / File(s)	Summary
Community/admission token expansion `src/lib/clinical-search.ts`, `src/lib/rag-retrieval-variants.ts`, `tests/clinical-search.test.ts`, `tests/retrieval-query-variants.test.ts`	Token normalization and retrieval variant generation add the "patients" token and new admission/community and clozapine amber/red range variants, validated by updated tests.
Extractive evidence gating `src/lib/rag-extractive-answer.ts`	New helpers detect blood-count and withhold-action evidence requirements, rejecting red_result_action matches at result and sentence level lacking that evidence.
Routing heuristics for titles and thresholds `src/lib/rag-routing.ts`, `tests/rag-routing.test.ts`	New patterns and helpers refine `chooseAnswerRoute` for document_lookup, table_threshold, medication_dose_risk, and multi-document cases, with matching test updates favoring extractive routing.
Fast-path gating and failure recovery `src/lib/rag.ts`, `tests/rag-answer-fallback.test.ts`	`decideTextFastPath` and evidence coverage checks add admission/community title gating, generation adds extractive recovery for selected failures, and fallback test coverage is updated.

Registry corpus embedding and re-embed-on-edit tooling

Layer / File(s)	Summary
Batched embedding and re-embed API `src/lib/registry-corpus.ts`, `tests/registry-corpus.test.ts`	Embedding writes are batched with rollback on failure, and new re-embed-by-slug helpers plus routing/best-effort wrappers are added and tested.
Embedding script owner-listing mode `scripts/embed-registry-records.ts`, `docs/rag-hybrid-findings-and-todo.md`	Adds `--list-owners` CLI reporting, owner-count aggregation, and dry-run messaging, alongside updates that mark registry embedding status and follow-up items.

Drift CI workflow, schema grants, and review protocol docs

Layer / File(s)	Summary
Live drift CI, schema grants, manifest `.github/workflows/live-drift.yml`, `supabase/schema.sql`, `supabase/drift-manifest.json`	Adds a scheduled/manual Supabase drift-check workflow, restricts execute permissions on two ingestion-job functions to service_role, and updates drift manifest metadata.
Review protocol and playbook docs `docs/branch-review-ledger.md`, `docs/codex-review-protocol.md`, `docs/codex-prompt-playbook.md`	Adds documentation for a branch review ledger, a Codex review protocol with severity rubric, and new operational prompt playbook sections.

Estimated code review effort: 4 (Complex) | ~60 minutes

🚥 Pre-merge checks | ✅ 5 | ❌ 6

❌ Failed checks (6 inconclusive)

Check name	Status	Explanation	Resolution
Generated And Sensitive Files	❓ Inconclusive	Repository clone failed, so this custom check could not run with code access.	Retry the review run. If this persists, inspect pre-merge custom-check logs for infrastructure or agent runtime failures.
Verification Claims	❓ Inconclusive	Repository clone failed, so this custom check could not run with code access.	Retry the review run. If this persists, inspect pre-merge custom-check logs for infrastructure or agent runtime failures.
Risky Git Or Deployment Actions	❓ Inconclusive	Repository clone failed, so this custom check could not run with code access.	Retry the review run. If this persists, inspect pre-merge custom-check logs for infrastructure or agent runtime failures.
Supabase Project And Schema Safety	❓ Inconclusive	Repository clone failed, so this custom check could not run with code access.	Retry the review run. If this persists, inspect pre-merge custom-check logs for infrastructure or agent runtime failures.
Runtime And Package Manager Integrity	❓ Inconclusive	Repository clone failed, so this custom check could not run with code access.	Retry the review run. If this persists, inspect pre-merge custom-check logs for infrastructure or agent runtime failures.
Api Route Failure Handling	❓ Inconclusive	Repository clone failed, so this custom check could not run with code access.	Retry the review run. If this persists, inspect pre-merge custom-check logs for infrastructure or agent runtime failures.

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	Docstring coverage is 99.34% which is sufficient. The required threshold is 70.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title is concise and accurately reflects the main changes: RAG routing hardening, registry corpus embedding, and docs updates.
Description check	✅ Passed	The description follows the required template with Summary, Verification, Clinical Governance Preflight, and Notes sections, and is mostly complete.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch claude/llm-pipeline-review

✨ Simplify code

Create PR with simplified code
Commit simplified code in branch claude/llm-pipeline-review

_{Comment @coderabbitai help to get the list of available commands.}

BigSimmo · 2026-07-09T11:02:47Z

@copilot resolve the merge conflicts in this pull request

supabase · 2026-07-09T11:03:04Z

Updates to Preview Branch (claude/llm-pipeline-review) ↗︎

Deployments	Status	Updated
Database	✅	Thu, 09 Jul 2026 13:20:35 UTC
Services	✅	Thu, 09 Jul 2026 13:20:35 UTC
APIs	✅	Thu, 09 Jul 2026 13:20:35 UTC

Tasks are run on every commit but only new migration files are pushed.
Close and reopen this PR if you want to apply changes from existing seed or migration files.

Tasks	Status	Updated
Configurations	✅	Thu, 09 Jul 2026 13:20:38 UTC
Migrations	✅	Thu, 09 Jul 2026 13:20:40 UTC
Seeding	✅	Thu, 09 Jul 2026 13:20:41 UTC
Edge Functions	✅	Thu, 09 Jul 2026 13:20:42 UTC

View logs for this Workflow Run ↗︎.
Learn more about Supabase for Git ↗︎.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 632e90477b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 632e90477b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

BigSimmo and others added 24 commits July 9, 2026 00:26

feat: add image generation metadata re-stamp script

f86ae20

chore: reconcile ingestion RPC execute privileges, schema.sql and dri…

bac1752

…ft manifest

ci: add db-reset-verify and dependency-review workflows

9216187

ci: remove dependency-review workflow because repository is private w…

83b437e

…ithout GHAS

Merge origin/main into claude/llm-pipeline-review

1eee7d8

Merge remote-tracking branch 'origin/claude/llm-pipeline-review' into…

50adfd5

… claude/llm-pipeline-review

📝 CodeRabbit Chat: Simplify code implementation (#434)

6b7e581

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

fix(db): rename duplicate migration version 20260708160000 to unique …

7efedc2

…20260708160001

fix: run prettier on 19 files to fix CI format:check failure (#436)

24314db

fix: address follow-up PR 433 review comments

255ca67

- Backfill NULL document_images.index_generation_id during re-stamp - Count globally forced embedding eval cases in retrieval summaries Co-authored-by: BigSimmo <BigSimmo@users.noreply.github.com>

fix: address PR 433 review blockers

e811cc1

Merge origin/main into claude/llm-pipeline-review

d831252

Merge remote-tracking branch 'origin/claude/llm-pipeline-review' into…

a76fc39

… codex/pr433-fix

ci: keep supabase cache save non-blocking

e1b5407

Merge remote-tracking branch 'origin/claude/llm-pipeline-review' into…

6c06307

… codex/pr433-fix

ci: run migration verification before docker image cache save

93c70e3

fix: address registry corpus review comments

c424845

Finish RAG registry re-embed and quality routing (#438)

75ad728

* fix(rag): finish registry re-embed and quality routing * fix: resolve merge conflicts with claude/llm-pipeline-review --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>

docs: codify review and branch cleanup workflow

3e64189

Merge remote-tracking branch 'origin/claude/llm-pipeline-review' into…

009e85c

… claude/llm-pipeline-review # Conflicts: # tests/rag-routing.test.ts

Copilot started work on behalf of BigSimmo July 9, 2026 11:03 View session

chore: initial plan — resolving merge conflicts with main

c4ce648

This comment was marked as resolved.

Sign in to view

BigSimmo marked this pull request as ready for review July 9, 2026 13:08

BigSimmo enabled auto-merge (squash) July 9, 2026 13:08

chatgpt-codex-connector Bot reviewed Jul 9, 2026

View reviewed changes

Comment thread src/lib/rag-routing.ts Outdated

chatgpt-codex-connector Bot reviewed Jul 9, 2026

View reviewed changes

Comment thread src/lib/rag-routing.ts

BigSimmo and others added 3 commits July 9, 2026 21:15

Potential fix for pull request finding

c97f79f

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Potential fix for pull request finding

a7d1d2d

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Potential fix for pull request finding

4958168

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

coderabbitai Bot previously approved these changes Jul 9, 2026

View reviewed changes

BigSimmo dismissed coderabbitai[bot]’s stale review via 4958168 July 9, 2026 13:15

coderabbitai Bot previously approved these changes Jul 9, 2026

View reviewed changes

Copilot started work on behalf of BigSimmo July 9, 2026 13:17 View session

fix: format docs/branch-review-ledger.md to pass Prettier check

6186b89

auto-merge was automatically disabled July 9, 2026 13:18
Head branch was pushed to by a user without write access

Copilot AI dismissed coderabbitai[bot]’s stale review via 6186b89 July 9, 2026 13:18

Copilot finished work on behalf of BigSimmo July 9, 2026 13:19

Merge branch 'main' into claude/llm-pipeline-review

76516e3

coderabbitai Bot approved these changes Jul 9, 2026

View reviewed changes

BigSimmo enabled auto-merge (squash) July 9, 2026 13:25

BigSimmo merged commit 3140d69 into main Jul 9, 2026
10 checks passed

BigSimmo deleted the claude/llm-pipeline-review branch July 9, 2026 14:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

RAG routing hardening, registry corpus embedding, and review docs#443

RAG routing hardening, registry corpus embedding, and review docs#443
BigSimmo merged 42 commits into
mainfrom
claude/llm-pipeline-review

BigSimmo commented Jul 9, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Jul 9, 2026 •

edited

Loading

Reviews paused

Walkthrough

Changes

❌ Failed checks (6 inconclusive)

Uh oh!

BigSimmo commented Jul 9, 2026

Uh oh!

supabase Bot commented Jul 9, 2026 •

edited

Loading

Uh oh!

This comment was marked as resolved.

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

BigSimmo commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Verification

Clinical Governance Preflight

Notes

Uh oh!

coderabbitai Bot commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviews paused

Walkthrough

Changes

❌ Failed checks (6 inconclusive)

Uh oh!

BigSimmo commented Jul 9, 2026

Uh oh!

supabase Bot commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as resolved.

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

BigSimmo commented Jul 9, 2026 •

edited

Loading

coderabbitai Bot commented Jul 9, 2026 •

edited

Loading

supabase Bot commented Jul 9, 2026 •

edited

Loading