Expose privacy-safe in-process answer coalescing metrics on deep health probe

[BigSimmo]

Motivation

• Operators lacked a privacy-safe, per-process signal showing whether the app process was absorbing duplicate answer work, making replica dilution and cache/key regressions hard to diagnose.
• The absence of that signal increases the risk of amplified retrieval fan-out, higher Postgres CPU pressure, and unnecessary OpenAI requests when replicas dilute in-memory coalescing.
• Add a lightweight, non-sensitive operational metric to make coalescing effectiveness visible without exposing queries, owners, documents, or cache keys.

Description

• Add a new process-local metrics module src/lib/observability/answer-coalescing-metrics.ts that records originations, coalesced waiters, an active-origination gauge, and a derived coalescing rate.
• Instrument the in-flight answer coalescer in src/lib/rag.ts to increment the new counters for coalescible requests and to avoid counting cache-bypassing requests or emitting any request-specific content.
• Surface the metrics on the authorized deep health probe via src/lib/health-response.ts as an opt-in coalescing block while keeping /api/health/ready free of diagnostic counters via src/app/api/health/ready/route.ts.
• Add unit tests for the new metrics invariants in tests/answer-coalescing-metrics.test.ts, extend tests/health-route.test.ts to assert privacy/authorization behavior, and update docs (docs/observability-slos.md, docs/capacity-review.md) describing operator interpretation and recommended usage.

Testing

• Pre-commit checks (git diff --cached --check) passed for the staged changes so the patch is syntactically clean.
• The PR-local dry-run (npm run verify:pr-local -- --dry-run) successfully produced the verification plan including the expected gates for check:runtime, lint/typecheck, tests and build; it did not execute hosted/provider checks (dry run only).
• Full automated Node/npm gates could not complete in this environment because the worktree lacked required runtime/dependencies: npm run test failed due to missing vitest runtime artifact and the environment provided Node v20 while the repo requires Node 24, and verify:cheap / verify:release:offline blocked on a missing tsx/cli resolver; these are environmental, not code failures.
• No hosted, provider-backed, or deployment checks were run per the instruction not to perform hosted/staging actions.

---

Codex Task

Summary by CodeRabbit

• New Features

  • Authorized deep health probes now report answer coalescing metrics, including originations, coalesced waiters, active originations, and coalescing rate.
  • Metrics help identify duplicate-heavy periods and potential capacity or cost regressions.

• Bug Fixes

  • Readiness health responses continue to exclude diagnostic coalescing details.

• Documentation

  • Updated observability and capacity guidance with metric definitions, examples, privacy considerations, and operational interpretation.

[supabase]

This pull request has been ignored for the connected project sjrfecxgysukkwxsowpy because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.

---

Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: e75a7e7f-a198-4c09-a814-4dd1e17a4fe6

📥 Commits

Reviewing files that changed from the base of the PR and between 1ab6119 and f201567.

📒 Files selected for processing (8)

• docs/capacity-review.md
• docs/observability-slos.md
• src/app/api/health/ready/route.ts
• src/lib/health-response.ts
• src/lib/observability/answer-coalescing-metrics.ts
• src/lib/rag.ts
• tests/answer-coalescing-metrics.test.ts
• tests/health-route.test.ts

---

📝 Walkthrough

Walkthrough

The change adds process-local answer coalescing metrics, records them in the RAG in-flight request path, exposes them only through authorized deep health probes, excludes them from readiness responses, and documents the new diagnostic block and capacity guidance.

Changes

Answer Coalescing Observability

Layer / File(s)	Summary
Metrics instrumentation and validation src/lib/observability/answer-coalescing-metrics.ts, src/lib/rag.ts, tests/answer-coalescing-metrics.test.ts	Adds origination, waiter, active-originations, and derived rate metrics; records them across the in-flight answer lifecycle; and tests zero-state, rate, lifecycle, and non-negative gauge behavior.
Health probe response wiring src/lib/health-response.ts, src/app/api/health/ready/route.ts, tests/health-route.test.ts	Adds conditional coalescing data to authorized deep probes, disables it for readiness responses, and verifies authorized, unauthorized, and readiness payload behavior.
Probe and capacity documentation docs/observability-slos.md, docs/capacity-review.md	Documents the third deep-probe counter block, its fields, rate calculation, visibility, and capacity-review guidance using coalescing and cache counter deltas.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant AnswerRequest
  participant RAG
  participant Metrics
  participant DeepHealthProbe
  participant HealthResponse
  AnswerRequest->>RAG: request answer
  RAG->>Metrics: record origination or coalesced waiter
  RAG->>Metrics: record origination completion
  DeepHealthProbe->>HealthResponse: request authorized deep probe
  HealthResponse->>Metrics: read coalescing snapshot
  Metrics-->>HealthResponse: return counters and rate
  HealthResponse-->>DeepHealthProbe: return coalescing diagnostic block

Loading

Possibly related PRs

• BigSimmo/Database#510: Modifies the same in-flight answer coalescing logic in src/lib/rag.ts.
• BigSimmo/Database#511: Modifies shared deep-health and readiness response plumbing.
• BigSimmo/Database#536: Modifies deep health payload composition, tests, and documentation.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The PR description omits required template sections like Summary, Risk and rollout, Notes, and the verification checklist.	Add the missing template sections, complete the verification checklist, and include risk/rollback plus governance notes for the privacy/API change.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly names the main change: exposing privacy-safe answer coalescing metrics on the deep health probe.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/improve-database-production-readiness

---

_{Comment @coderabbitai help to get the list of available commands.}