refactor(rag): extract candidate-source searchers into rag-candidate-sources.ts

[BigSimmo]

Summary

Deferred audit item D3 — rag.ts decomposition phase 1 (2026-07-13 audit, finding 11). Zero behavior change: 1,014 lines moved verbatim, nothing rewritten.

What moved (rag.ts 5,779 → ~4,790 lines)

New module src/lib/rag-candidate-sources.ts — the candidate retrieval surfaces for the search waterfall:

• Versioned-RPC plumbing: callVersionedRetrievalRpc, the legacy owner/public row merge, RPC-error and variant-fanout telemetry recorders, SupabaseRpcError.
• The candidate searchers: searchTextChunkCandidates (with trigram-correction + OR-relaxation fallbacks), searchDocumentLookupFastPath (+ title-alias and best-chunk helpers), searchTableFactCandidates, searchEmbeddingFieldCandidates, searchIndexUnitCandidates.
• The chunk hydrators loadChunksForMemoryCards / loadChunksForSignalMatches and the withMemoryBoostedCandidates merge step, plus mergeSearchResults / memoryCardChunkScore.

How the move was made safe

• Script-extracted with boundary guards and per-symbol promotion assertions; the module generated from the pre-fix(rag): harden GPT-5.6 answer generation #626 base and the post-fix(rag): harden GPT-5.6 answer generation #626 base is byte-identical (the answer-hardening PR touched no moved range), confirming nothing was hand-retyped.
• rag.ts re-exports callVersionedRetrievalRpc, loadChunksForMemoryCards, loadChunksForSignalMatches — existing test imports (tests/retrieval-access-scope, tests/retrieval-hydration-scope) are untouched.
• The extracted module imports only non-rag lib modules (owner-scope, clinical-search, deep-memory, rag-retrieval-variants, reindex-pipeline, …) — no import cycle.

Verification

• typecheck + lint clean (14 now-unused rag.ts imports removed)
• eval:rag:offline green — 279 offline RAG production-contract tests (the zero-behavior-change guard)
• verify:cheap green (2,301 tests)
• verify:pr-local green (production build + client-bundle secret scan)

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features

  • Improved search across document text, aliases, table facts, embeddings, and indexed content.
  • Added smarter query fallback and clinical-term correction for broader matching.
  • Added memory-based result boosting and improved source explanations.
  • Improved handling of legacy and current retrieval paths.

• Bug Fixes

  • Added safeguards for inconsistent indexed content and unavailable retrieval services.
  • Improved result ranking, merging, and filtering for more relevant answers.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

[supabase]

This pull request has been ignored for the connected project sjrfecxgysukkwxsowpy because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.

---

Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 3efee427-87c5-4c9e-9a64-20a997b549d8

📥 Commits

Reviewing files that changed from the base of the PR and between e60c18b and 2cdcda3.

📒 Files selected for processing (2)

• src/lib/rag-candidate-sources.ts
• src/lib/rag.ts

---

📝 Walkthrough

Walkthrough

The PR adds src/lib/rag-candidate-sources.ts for RPC compatibility, lexical and structured candidate retrieval, chunk hydration, telemetry, scoring, and memory boosting. src/lib/rag.ts is refactored to consume and re-export these helpers.

Changes

RAG candidate retrieval

Layer / File(s)	Summary
Retrieval surfaces and RPC compatibility src/lib/rag-candidate-sources.ts	Adds versioned/legacy RPC fallback, deterministic result merging, lexical query fallbacks, document alias lookup, chunk ranking, and retrieval telemetry.
Candidate hydration and match explanations src/lib/rag-candidate-sources.ts	Hydrates memory-card and signal matches into scoped SearchResult objects with index-generation checks, explanations, index units, and promotable table facts.
Specialized candidate searches src/lib/rag-candidate-sources.ts	Adds table-fact, embedding-field, and index-unit searches backed by hybrid RPCs and chunk hydration.
Memory boosting and RAG wiring src/lib/rag-candidate-sources.ts, src/lib/rag.ts	Memoizes memory-card fetches, merges boosted candidates, and moves retrieval imports, exports, and deep-memory wiring into the new module.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant RAG
  participant CandidateSources
  participant Supabase
  RAG->>CandidateSources: request candidate search
  CandidateSources->>Supabase: execute retrieval RPC or fallback query
  Supabase-->>CandidateSources: candidate rows and metadata
  CandidateSources->>Supabase: hydrate authorized chunks
  Supabase-->>CandidateSources: chunk records
  CandidateSources-->>RAG: ranked and boosted SearchResult candidates

Loading

Possibly related PRs

• BigSimmo/Database#514: Shares hybrid RPC retrieval telemetry and document/table-fact retrieval paths.
• BigSimmo/Database#553: Shares versioned/legacy RPC compatibility and access-scoped chunk loading.

Suggested reviewers: claude, copilot

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch claude/rag-candidate-sources-2026-07-14

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

ESLint install failed. For unrecoverable errors, disable the tool in CodeRabbit configuration.

---

_{Comment @coderabbitai help to get the list of available commands.}