feat(rag): budget-aware generation deadlines + retrieval-exhausted telemetry (Phase E-3b)

[BigSimmo]

Summary

Phase E-3b of the approved plan — fixes the three measured latency/waste defects from baseline run #57 and live telemetry (issues I3, I5, I2):

• Reserve-aware generation timeouts: every generation attempt (all four attempt kinds share one call-site) is clamped to route budget − 2,000ms recovery reserve. The reserve is measured, not guessed (recovery costs 26–100ms locally, ≤1s cross-region). Replaying the run-#57 agitation failure: the attempt now times out with ~2.4s to spare and the source-backed recovery lands inside budget instead of 54ms over.
• Truncation self-heal viability gate: when under 7s of budget remains, the ~20s+ retry is skipped as guaranteed-futile — recorded as an observable truncation_retry_skipped_budget_reserve marker in answer_retry_reasons — and the existing source-backed recovery delivers. Kills the live 82s-wasted max_output_tokens class.
• Honest telemetry instead of a weakened gate for the clozapine shape (13.3s retrieval vs the runtime's 12s extractive budget on a US runner): the runtime records an additive route_budget_exhausted_by_retrieval flag, and the eval suppresses the route-ceiling only when cross-region context AND the runtime flag AND zero generation time all hold. Local/release gates provably unchanged (pinned by tests); suppressed cases stay auditable via a retrieval-exhausted report cell.

Both reviewer subagents ran pre-push, zero P0/P1/P2:

• rag-retrieval-reviewer: APPROVE-WITH-NITS — confirmed zero retrieval behaviour change (deadline created after selection freezes), timeout arithmetic correct across degenerate cases, the skip branch cannot dead-end or double-count, marker isolation proven (SLO counters key on fallback_reason, never answer_retry_reasons), and the cached-replay invariant proven: budget-exhausted answers are never cached, so the carve-out is unreachable via replay. 2 P3 nits (prod-dead requestTimeoutMs retained for symmetry; cosmetic report-cell coupling).
• clinical-governance-reviewer: APPROVE-WITH-NITS — both clinician-visible spots shift only toward the already-approved conservative surface via existing reviewed paths (internal timeout → PublicApiError → existing fallback; never the abort re-throw); claim-support, numeric verification, final quality gate, and source-only fallback all still applied to recovery answers; telemetry additions carry no PHI.

Also banked in the ledger: the run #58 targeting baseline (relevance 0.6; targeting 0.59 with dose 1/5 and monitoring 1/5 — the E-3c targets).

RAG impact: no retrieval behaviour change — answer-side timing/gating and telemetry only; retrieval, ranking, selection, and comparators untouched (adversarially verified by the retrieval reviewer).

Verification

• Red-proofs that fail on revert: the 2s reserve is pinned three independent ways (exact 23,000ms grant on a fast route, deadline flag stays clear, total under budget); the self-heal skip pins the exact marker sequence + single provider call + no retry-count increment; the offline harness pins the exhaustion flag both true (13s retrieval on a 12s budget, provider never called, grounded extractive answer) and false
• Focused vitest: route-budget 9/9, eval-quality 27/27 (carve-out matrix incl. strict-local proof), fallback+offline 52/52, telemetry-parser/abort regressions 15/15
• npm run test — 3043 passed / 1 known container-only pdf-budget artifact / 3 skipped
• npm run typecheck, npm run lint, prettier — clean
• No provider calls in this PR; live proof arrives with the E-4 paired run (~$2–4 of the approved envelope)

Risk and rollout

• Risk: low-medium — two narrow latency-band behavior shifts (documented above), both toward the existing conservative surface; bounded at 8% of the fast budget and observable live via the generation_fallback:provider_timeout rate and the new skip marker. No new failure modes: every new branch terminates on a pre-existing path.
• Rollback: single revert. No env knobs, no schema change, no cache-version change; the new latencyTimings field is optional so mixed-version telemetry interoperates both directions.
• Provider or production effects: none from this diff; behavior changes take effect on deploy and are measured by the E-4 paired canary before being trusted.

Clinical Governance Preflight

• Source-backed claims still require linked source verification before clinical use — recovery answers pass the unchanged claim-support + numeric-verification + final-quality gates (reviewer-verified)
• No patient-identifiable document workflow was introduced or expanded
• Supabase target remains Clinical KB Database (sjrfecxgysukkwxsowpy) — no schema/config/data changes
• Service-role keys and private document access remain server-only
• Demo/synthetic content remains clearly separated from real clinical sources
• Source metadata, review status, and outdated/unknown-source behavior remain conservative
• Deployment classification/TGA SaMD impact was checked: answer TEXT can shift in a narrow latency band from late model prose to the already-approved source-backed fallback copy — conservative-direction, governance-reviewed, measured by the E-4 pair

🤖 Generated with Claude Code

https://claude.ai/code/session_01EXsJcLrbZUXwnBeG91cVo9

---

Generated by Claude Code

Summary by CodeRabbit

• Bug Fixes

  • Improved response generation under tight time limits by reserving recovery time for source-backed answers.
  • Skipped escalation/self-heal retries when remaining route budget can’t support them.
  • Preserved grounded extractive answers when retrieval consumes the route budget.
  • Refined deadline reporting when budget is exhausted before generation begins, including cross-region behavior.

• Diagnostics

  • Added retrieval-related budget exhaustion telemetry (non-PII) and clarified the “ceiling exceeded” state.

• Tests

  • Expanded coverage for budget-aware deadline math, retry skipping, offline exhaustion scenarios, and cross-region diagnostics.

[supabase]

This pull request has been ignored for the connected project sjrfecxgysukkwxsowpy because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.

---

Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: c06f312f-72f4-4cfa-9517-172d36bf522e

📥 Commits

Reviewing files that changed from the base of the PR and between 56c5d2c and 500a66e.

📒 Files selected for processing (1)

• src/lib/rag/rag.ts

---

📝 Walkthrough

Walkthrough

The RAG route budget reserves recovery time for generation fallback, gates truncation retries, records retrieval-attributed exhaustion, and updates cross-region evaluation diagnostics with a specific deadline carve-out and status.

Changes

Budget-aware RAG generation

Layer / File(s)	Summary
Route budget deadline contract src/lib/rag/rag-route-budget.ts, tests/rag-route-budget.test.ts	Adds recovery and retry-floor constants, reserve-aware generation timeouts, retry eligibility checks, and corresponding deadline tests.
Generation fallback and route telemetry src/lib/rag/rag.ts, src/lib/types.ts, tests/rag-answer-fallback.test.ts, tests/rag-offline-answer.test.ts	Caps generation requests, skips truncation retries when the reserve is insufficient, records retrieval budget exhaustion, and validates fallback behavior and telemetry.
Evaluation deadline attribution scripts/eval-quality.ts, tests/eval-quality.test.ts, docs/branch-review-ledger.md	Adds the cross-region retrieval-exhausted carve-out, preserves detailed timing attribution, renders retrieval-exhausted, and records the E-3b review entries.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant RAG
  participant RouteDeadline
  participant OpenAI
  participant ExtractiveFallback
  RAG->>RouteDeadline: calculate reserve-aware timeout
  RouteDeadline-->>RAG: return generation timeout
  RAG->>OpenAI: request structured answer
  OpenAI-->>RAG: return truncated answer
  RAG->>RouteDeadline: check retry budget
  RouteDeadline-->>RAG: allow or deny retry
  RAG->>ExtractiveFallback: use source-backed recovery when denied

Loading

Suggested labels: codex

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is concise and accurately reflects the main budget-aware RAG deadline and telemetry changes.
Description check	✅ Passed	The required sections are present and detailed; verification and governance notes are mostly complete.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch claude/clinical-kb-pwa-review-asi3wb

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

🧹 Nitpick comments (1)

tests/rag-answer-fallback.test.ts (1)

2979-3118: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Duplicate waSource fixture helper.

The waSource builder inside lithiumTruncatedGenerationAnswer (lines 2988-3007) duplicates the existing top-level waSource helper (~line 1785) verbatim. Consider hoisting one shared helper to avoid the two copies drifting apart over time.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/rag-answer-fallback.test.ts` around lines 2979 - 3118, Remove the
locally defined waSource helper from lithiumTruncatedGenerationAnswer and reuse
the existing top-level waSource fixture helper near the other lithium tests.
Preserve the current source construction and publisher-specific metadata
behavior while eliminating the duplicate definition.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tests/rag-answer-fallback.test.ts`:
- Around line 2979-3118: Remove the locally defined waSource helper from
lithiumTruncatedGenerationAnswer and reuse the existing top-level waSource
fixture helper near the other lithium tests. Preserve the current source
construction and publisher-specific metadata behavior while eliminating the
duplicate definition.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: ad83614d-7ba1-4d42-8f4d-0bc086a36dad

📥 Commits

Reviewing files that changed from the base of the PR and between 5bd7e8d and 56c5d2c.

📒 Files selected for processing (9)

• docs/branch-review-ledger.md
• scripts/eval-quality.ts
• src/lib/rag/rag-route-budget.ts
• src/lib/rag/rag.ts
• src/lib/types.ts
• tests/eval-quality.test.ts
• tests/rag-answer-fallback.test.ts
• tests/rag-offline-answer.test.ts
• tests/rag-route-budget.test.ts