Reduce cold RAG round trips and reconcile retention controls#632
Conversation
Correct degraded-answer telemetry, consolidate bounded cache retention, parallelize independent hydration reads, and reconcile privacy and launch documentation with the verified live topology.
|
Updates to Preview Branch (codex/rag-performance-followups) ↗︎
Tasks are run on every commit but only new migration files are pushed.
View logs for this Workflow Run ↗︎. |
|
Warning Review limit reached
Next review available in: 6 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (4)
📝 WalkthroughWalkthroughThe change updates provider-generation degradation measurement, concurrent RAG enrichment, shared-cache handling, response-cache retention, privacy disclosures, cross-border governance records, launch sequencing, and performance findings. Tests cover the revised SLO filters, concurrency, cache behavior, privacy copy, and migration SQL. ChangesRAG platform and operations
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant RAGAnswerRoutes
participant logRagQuery
participant answerSloSnapshot
participant Supabase
RAGAnswerRoutes->>logRagQuery: persist provider_generation_degraded metadata
answerSloSnapshot->>Supabase: count provider_generation_degraded = "true"
Supabase-->>answerSloSnapshot: return degraded count
answerSloSnapshot-->>RAGAnswerRoutes: expose degradedQueries and degradedRate
Possibly related PRs
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: aa264e92c4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex resolve actionable Codex review findings for this pull request and current head using the repository instructions. This is the pull request's single automatic repair pass: do not perform a fresh review, create new standalone findings, or request another review. Work only the existing unresolved Codex threads on the current head. Always fix P0 and P1 findings. For P2 and lower findings, fix only clear, scoped, low-risk issues; otherwise disposition them with a concise reason. After fixing or dispositioning a thread, reply in that thread with as the first line, followed by a concise summary; that marker authorizes the workflow to close that exact thread. If human input or new authorization is required, do not use the marker and leave the thread open with the blocker. Finish only after every actionable thread is fixed or dispositioned and closed, or explicitly left open for a human decision. Do not update the branch from main, address unrelated reviews, broaden scope, or create more than one scoped fix commit. Do not use external APIs, paid services, credentials, dependency changes, or broad refactors unless explicitly authorized. Add targeted tests where behavior changes and run the narrowest relevant validation. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
docs/openai-cross-border-basis.md (1)
152-154: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick winTrack Railway’s pending DPA in the checklist/status table.
The paragraph correctly says the Railway contract is pending, but the numbered checklist and status table track only OpenAI. Because PIA-1 covers both Singapore and US processing, add a Railway checkbox and evidence/status field so completing the visible checklist cannot imply that the whole cross-border record is closed.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/openai-cross-border-basis.md` around lines 152 - 154, Update the numbered checklist and status table in the cross-border processing documentation to include Railway alongside OpenAI, adding a checkbox and corresponding evidence/status field for Railway’s pending DPA and Singapore processor/sub-processor basis. Ensure the visible checklist cannot indicate PIA-1 is complete while Railway’s requirements remain unfinished.src/lib/rag-cache.ts (1)
270-271: 🗄️ Data Integrity & Integration | 🔵 TrivialLGTM on the correctness of the fold — the selector's existing filters (
indexing_version,dependency_version,expires_at) do cover what the removed probe distinguished, and the new single-read path is verified by tests/rag-shared-cache.test.ts.One thing worth a quick check: collapsing TTL-expired/indexing-mismatch/dependency-mismatch into one
unknown_filter_missreason loses the granularity any existing cache-miss dashboards or triage runbooks may have relied on for those specific causes.Also applies to: 360-379
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/lib/rag-cache.ts` around lines 270 - 271, Update SharedCacheMissReason and the shared-cache miss classification flow to preserve distinct reasons for TTL expiration, indexing-version mismatch, and dependency-version mismatch instead of mapping them to unknown_filter_miss. Trace the selector/result handling around the single-read path and retain compatible reason names for existing dashboards and triage runbooks, while keeping cache_lookup_error, cache_lookup_exception, and cache_payload_invalid unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/openai-cross-border-basis.md`:
- Around line 104-106: Update the merge-status statement in the APP 5/1 section
to describe the privacy page and composer notice as a shipped draft with
governance approval pending, rather than marking APP 5/1 complete. Preserve the
existing PR and commit references, and ensure PIA-5 remains explicitly listed as
open alongside APP 8.
---
Nitpick comments:
In `@docs/openai-cross-border-basis.md`:
- Around line 152-154: Update the numbered checklist and status table in the
cross-border processing documentation to include Railway alongside OpenAI,
adding a checkbox and corresponding evidence/status field for Railway’s pending
DPA and Singapore processor/sub-processor basis. Ensure the visible checklist
cannot indicate PIA-1 is complete while Railway’s requirements remain
unfinished.
In `@src/lib/rag-cache.ts`:
- Around line 270-271: Update SharedCacheMissReason and the shared-cache miss
classification flow to preserve distinct reasons for TTL expiration,
indexing-version mismatch, and dependency-version mismatch instead of mapping
them to unknown_filter_miss. Trace the selector/result handling around the
single-read path and retain compatible reason names for existing dashboards and
triage runbooks, while keeping cache_lookup_error, cache_lookup_exception, and
cache_payload_invalid unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: db64555d-9063-42a7-a6c9-ef1033405241
📒 Files selected for processing (19)
docs/branch-review-ledger.mddocs/deployment-architecture.mddocs/launch-operator-runbook.mddocs/observability-slos.mddocs/openai-cross-border-basis.mddocs/privacy-impact-assessment.mddocs/rag-hybrid-findings-and-todo.mdsrc/app/privacy/page.tsxsrc/lib/health-response.tssrc/lib/observability/answer-slo.tssrc/lib/rag-cache.tssrc/lib/rag.tssupabase/migrations/20260713201542_consolidate_rag_response_cache_retention.sqltests/answer-slo.test.tstests/privacy-ui.test.tstests/rag-answer-fallback.test.tstests/rag-query-concurrency.test.tstests/rag-shared-cache.test.tstests/supabase-schema.test.ts
Track generation fallbacks separately from intentional extractive answers so healthy source-only traffic does not page the provider-health SLO.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 07cac8f4e0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a2eb6db0ef
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
1 participant
Summary
Why
Cold RAG answers were paying avoidable serial database round trips, the degraded-answer SLO lacked a provider-specific signal, and governance/runbook state no longer matched the deployed Railway Singapore and Supabase Sydney topology.
Areas changed
src/lib/rag-cache.ts,src/lib/rag.ts, answer SLO/health telemetryVerification
npm run check:runtime— passed (Node 24.18.0 / npm 11.17.0)npm run format:check— passednpm run lint— passednpm run typecheck— passed, including after review fixnpm run test— 2,213 passed / 1 skipped on the initial PR headnpm run build— passed; 636 pages generated and client-bundle secret scan passednpm run eval:rag:offline— 36 fixtures and 277/277 contract tests passed before and after review fixcheck:supabase-project— correct Clinical KB Database refcheck:production-readiness— READY, 6 passes / 2 expected local-file warningsgit diff --check— passedThe combined
verify:pr-localwrapper exceeded the local shell's 15-minute ceiling; every selected component above was rerun individually and passed.Review follow-up
Codex identified that the broad source-only
degradedstate also includes healthy extractive answers. Commit07cac8f4addsprovider_generation_degraded, derived only fromgeneration_fallback, and points the provider-health SLO at that narrower flag. The thread is fixed and resolved.Production and governance notes
The bounded retention migration was already applied and verified live: query-miss job 13 and response-cache job 16 are active, and the obsolete unbounded job is absent. This PR records that state and supplies the matching migration artifact.
No deployment, read replica, model change, dependency change, or legal agreement acceptance is included. Model-routing experiments remain blocked by exhausted OpenAI quota. Railway/OpenAI contractual execution remains with the authorised account/legal owner.
Checks not run
Summary by CodeRabbit