rag: add retrieval phase latency telemetry

[BigSimmo]

Summary

• Record total search latency and named retrieval-phase latency telemetry.
• Resolve query classification, alias loading, and cache indexing version concurrently.
• Reuse one request-start indexing version across answer and nested search cache reads while preserving current retrieval deadlines and adversarial-request refusal.

Verification

• npm run test — 313 files, 2,840 tests passed.
• Focused latency/eval regression tests — 3 files, 28 tests passed.
• npm run eval:rag:offline — 36 golden cases, 21 suites, 294 tests passed.
• npm run typecheck
• Changed-file ESLint
• npm run check:production-readiness:ci — READY; only expected missing-local-env warnings.
• npm run verify:pr-local

Verification not run: npm run verify:pr-local was not repeated after its constituent full unit, typecheck, lint, offline RAG, and readiness checks passed; hosted CI remains responsible for the clean production build.

Live verification not run: no provider-backed retrieval or answer evaluation was run. No provider calls were made.

UI verification not run: no UI, routing, styling, or browser behavior changed.

Risk and rollout

• Risk: cache-version orchestration and phase concurrency change request timing, but not ranking or answer semantics; abort signals and current retrieval deadlines remain enforced.
• Rollback: revert the single commit.
• Provider or production effects: None. No live OpenAI, Supabase, deployment, or production workflow was invoked.

Clinical Governance Preflight

• Source-backed claims still require linked source verification before clinical use
• No patient-identifiable document workflow was introduced or expanded without explicit governance approval
• Supabase target remains Clinical KB Database (sjrfecxgysukkwxsowpy)
• Service-role keys and private document access remain server-only
• Demo/synthetic content remains clearly separated from real clinical sources
• Source metadata, review status, and outdated/unknown-source behavior remain conservative
• Deployment classification/TGA SaMD impact was checked when clinical decision-support behavior changed
  • This PR adds observability and latency orchestration without changing the clinical-use boundary.

Notes

• The full suite confirmed the cache-disabled path skips the obsolete indexing-version preflight and preserves lexical retrieval behavior.

Summary by CodeRabbit

Summary by CodeRabbit

• New Features

  • Added total search latency and per-phase retrieval latency telemetry to search responses and golden retrieval evaluations.
  • Improved cache consistency by reusing request-start indexing information across local and shared cache operations, with consistent telemetry stored in cache.
  • Enabled concurrent search preparation steps to reduce unnecessary waiting.

• Bug Fixes

  • Corrected latency computation to prefer precomputed total search latency when available.
  • Improved cache validation when indexing versions change.

• Tests

  • Added latency telemetry and cache invariant coverage, plus updates to early-exit retrieval assertions.

[supabase]

This pull request has been ignored for the connected project sjrfecxgysukkwxsowpy because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.

---

Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 5566ae76-42af-44ec-8f0a-fe6837790377

📥 Commits

Reviewing files that changed from the base of the PR and between 9b2ac55 and 4b823c6.

📒 Files selected for processing (2)

• src/lib/rag.ts
• tests/rag-tail-latency.test.ts

🚧 Files skipped from review as they are similar to previous changes (2)

• tests/rag-tail-latency.test.ts
• src/lib/rag.ts

---

📝 Walkthrough

Walkthrough

The retrieval pipeline now records total and per-phase latency telemetry, exposes it through the search API and evaluation results, and shares request-start indexing versions across local, shared, and nested cache operations.

Changes

RAG cache consistency and request context

Layer / File(s)	Summary
Request-start cache version propagation src/lib/rag-contracts.ts, src/lib/rag-cache.ts, src/lib/rag.ts, tests/rag-tail-latency.test.ts	Cache reads and writes accept shared indexing versions, nested answer-to-search reads reuse request-start state, and shared cache rows use the computed version consistently.

Retrieval timing and telemetry

Layer / File(s)	Summary
Retrieval phase timing and return paths src/lib/rag.ts, tests/rag-tail-latency.test.ts	Bootstrap, cache, hydration, reranking, fallback, and early-return paths are timed and finalized through shared telemetry helpers.
Telemetry propagation and evaluation src/app/api/search/route.ts, scripts/eval-retrieval.ts, tests/eval-retrieval.test.ts, tests/rag-variant-early-exit.test.ts	Total and per-phase latency fields flow through API responses and evaluation results, with corresponding assertions and an updated early-exit expectation.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant SearchClient
  participant SearchRoute
  participant searchChunksWithTelemetry
  participant rag-cache
  SearchClient->>SearchRoute: request search
  SearchRoute->>searchChunksWithTelemetry: execute retrieval
  searchChunksWithTelemetry->>rag-cache: timed cache lookup
  rag-cache-->>searchChunksWithTelemetry: cached or uncached result
  searchChunksWithTelemetry-->>SearchRoute: results and latency telemetry
  SearchRoute-->>SearchClient: response telemetry

Loading

Possibly related PRs

• BigSimmo/Database#457: Modifies the RAG retrieval contracts and cache-related telemetry surfaces.
• BigSimmo/Database#632: Also changes the shared-search caching path in src/lib/rag-cache.ts.
• BigSimmo/Database#641: Refactors retrieval orchestration and telemetry-aware candidate-source handling.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly matches the main change: adding retrieval-phase latency telemetry.
Description check	✅ Passed	The description follows the template and includes summary, verification, risk/rollout, clinical governance, and notes.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/rag-tail-latency-main-20260719

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/rag.ts`:
- Around line 2467-2469: Update the fetchEnabledRagAliases call within the
ragAliasesPromise initialization to pass args.signal through to the concurrent
alias-loading operation, preserving the existing timing wrapper and arguments.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 65f0c704-e90e-4ac6-9b07-5d6a874dc593

📥 Commits

Reviewing files that changed from the base of the PR and between 68841ae and 9b2ac55.

📒 Files selected for processing (8)

• scripts/eval-retrieval.ts
• src/app/api/search/route.ts
• src/lib/rag-cache.ts
• src/lib/rag-contracts.ts
• src/lib/rag.ts
• tests/eval-retrieval.test.ts
• tests/rag-tail-latency.test.ts
• tests/rag-variant-early-exit.test.ts