fix(eval): cost readout counts zero-usage cases as $0 + follow-up ledger capture

docs/outstanding-issues.md

@@ -27,7 +27,7 @@ Durable, cross-session memory of everything still outstanding for this repo: ope
 - Resolving an item moves its row to **Resolved / archive** with the date and a one-line outcome —
   rows are archived, not deleted, so the history stays auditable.
 
-<!-- issues:next-id=018 -->
+<!-- issues:next-id=024 -->
 
 ## Open items
 
@@ -47,6 +47,12 @@ Durable, cross-session memory of everything still outstanding for this repo: ope
 | #015 | P3  | task  | Content-first fallback regression tests                 | CodeRabbit nitpick on #1024: add focused tests for the medication + registry (services/forms) content-first transitions — fallback content during loading, live record swap-in, error/empty states, and that **no authoritative verification badge shows before live governance reconciles** (the neutralized fixture flag).                                                                                                                                                                                                                                                                   | #1024 CodeRabbit review; `src/components/clinical-dashboard/medication-record-page.tsx`; `src/components/registry-record-loader.tsx`    | 2026-07-21 |
 | #016 | P3  | rec   | "Big but not easy" structural + motion perf             | Deferred larger levers: (a) nonce-CSP forces every product route to `ƒ Dynamic` (zero static generation) — evaluate Partial Prerendering / static shells for the static clinical catalogues (DSM/differentials/therapy/specifiers/formulation); (b) sidebar expand/collapse animates `grid-template-columns` (biggest smoothness cost, motion-gated — needs a transform-overlay rethink); (c) Therapy Compass fetches 692 KB / 2.5 MB JSON client-side (defer until interaction + confirm brotli); (d) settings/setup/admin dialogs static-imported into the home chunk (`next/dynamic` them). | session 2026-07-21 (build route table + design audit)                                                                                   | 2026-07-21 |
 | #017 | P3  | task  | Field Web-Vitals baseline via live Lighthouse           | In-sandbox runtime vitals were blocked (prod server hard-requires Supabase secrets; dev-mode CLS measured excellent at 0.00–0.04, content-first pages 0.000). Run Lighthouse against `psychiatry.tools` for real LCP/INP/CLS to prioritize #012–#016 by measured impact rather than reasoning.                                                                                                                                                                                                                                                                                                 | session 2026-07-21 (measurement pass)                                                                                                   | 2026-07-21 |
+| #018 | P2  | task  | Monitoring retrieval-depth trio misses schedules        | quality-lithium-monitoring-range, quality-antipsychotic-metabolic-monitoring, quality-adhd-medication-monitoring still fail targeting ("no schedule/interval") after the #1040 gate parity — their live chunk sets apparently surface no admissible schedule sentence. Diagnose via read-only chunk queries + run #61 dump artifact (30d retention) to split corpus-content vs retrieval-depth vs fact-extraction; fix shape TBD after diagnosis. A retrieval-side fix is a protected surface (canary pair, fresh ~$2-4 authorization).                                                        | run #61 targeting misses; branch-review-ledger 2026-07-21                                                                               | 2026-07-21 |
+| #019 | P2  | task  | Comparison-class coverage: admission doc never in top-5 | admission-discharge-comparison misses its expected admission-side doc on every canary (#57-#61) — top-5 carries discharge-planning + unrelated docs only. Same "why doesn't retrieval surface X" species as #018; diagnose jointly. Ground-truth already widened (#1020); the residual is genuine comparison-class retrieval coverage.                                                                                                                                                                                                                                                         | runs #57-#61; PR #1020 replay verdict                                                                                                   | 2026-07-21 |
+| #020 | P3  | task  | Validate eval:quality cost readout post-fix             | The estimator fix landed in PR #1050 (zero-usage cases count $0 with rates configured; provider-attempted cases with missing usage stay n/a). Remaining action: confirm the next canary run prints a real dollar figure in Answer Metrics, then archive this row.                                                                                                                                                                                                                                                                                                                              | PR #1050; run #60/#61 Answer Metrics tables                                                                                             | 2026-07-21 |
+| #021 | P3  | rec   | E-3d H2 residual: strong/comparison generation discards | approx. 6 generation attempts per full 44-case run still fail the final quality gate and fall to extractive on strong-route comparison/complex shapes (the designed-conservative outcome). PARKED: weakest cost/benefit on the queue — a wave (approx. $2-4 pair + reviewer cycle) to shave seconds off a few hard cases. Revisit only if latency/waste complaints or a cheaper lever appears.                                                                                                                                                                                                 | E-3c design record; runs #59-#61 diagnostics                                                                                            | 2026-07-21 |
+| #022 | P2  | task  | Source-governance metadata refresh (operator)           | Governance warning rate ~0.84 across canaries: stale/review-required/unknown source metadata on most top results. Operator work (document review-status attestation in the app), not a code defect. Next action: generate the prioritized refresh worklist ($0 read-only) and schedule a metadata pass.                                                                                                                                                                                                                                                                                        | runs #57-#61 Source Governance tables; docs/observability-slos.md                                                                       | 2026-07-21 |
+| #023 | P3  | task  | Read Sunday 2026-07-26 scheduled-run artifacts          | The 18:00 UTC scheduled runs deliver three free datapoints at once: first full-44 weekly canary (validates the #1044 ANSWER_CASE_LIMIT raise), browser-matrix flake second datapoint (webkit ui-route-coverage:312, firefox ui-formulation:91 — diagnose only if reproduced), and the irrelevant@10 labeling-audit artifact (§3.1 human-decision class). Read all three, then disposition.                                                                                                                                                                                                     | sessions 2026-07-20/21; branch-review-ledger convergence notes                                                                          | 2026-07-21 |
 
 ## Resolved / archive
 

scripts/eval-quality.ts

@@ -12,6 +12,7 @@ import {
   type GoldenRetrievalResult,
 } from "./eval-retrieval";
 import {
+  configuredCostRates,
   estimateCostUsd,
   loadAdminClient,
   percentile,
@@ -1137,7 +1138,19 @@ async function runRagQualityCases(args: {
       routingReason: answer.routingReason,
       timings,
       routeCeilingExceeded,
-      estimatedCostUsd: hasOpenAIUsage ? estimateCostUsd(openAIUsage) : null,
+      // A case that never touched the provider costs exactly $0 when rates
+      // are configured; null stays reserved for "cannot estimate" — either
+      // rates unconfigured, or a provider call was attempted (request id,
+      // model, or generation time recorded) without usage metadata (e.g. a
+      // provider timeout), where real-but-unmeasured tokens must not be
+      // reported as $0 (issue #014 + review findings on PR #1050).
+      estimatedCostUsd: hasOpenAIUsage
+        ? estimateCostUsd(openAIUsage)
+        : openAIRequestIds.length > 0 || answer.modelUsed || generationLatencyMs > 0
+          ? null
+          : configuredCostRates()
+            ? 0
+            : null,
       openAIRequestIds: openAIRequestIds.length > 0 ? openAIRequestIds : undefined,
       openAIUsage: hasOpenAIUsage ? openAIUsage : undefined,
     });

tests/eval-quality.test.ts

@@ -658,6 +658,21 @@ describe("eval quality reporting", () => {
     expect(report.blocking_threshold_failures).toContain("RAG route_ceiling_failure_count 1 above 0");
   });
 
+  it("sums estimated cost across priced and zero-cost cases, going n/a only when a case cannot be estimated", () => {
+    // Issue #014: extractive/unsupported cases make no provider call and cost
+    // exactly $0 when rates are configured — they must not null the run total.
+    // null stays reserved for "rates unconfigured" (cannot estimate).
+    const priced = ragResult({ estimatedCostUsd: 0.002 });
+    const zeroCost = ragResult({ estimatedCostUsd: 0 });
+    const unpriced = ragResult({ estimatedCostUsd: null });
+
+    const summed = buildEvalQualityReport({ retrievalResults: [], ragResults: [priced, zeroCost] });
+    expect(summed.rag.summary.estimated_cost_usd).toBe(0.002);
+
+    const unknown = buildEvalQualityReport({ retrievalResults: [], ragResults: [priced, zeroCost, unpriced] });
+    expect(unknown.rag.summary.estimated_cost_usd).toBeNull();
+  });
+
   it("accepts an offline source-only result only when expected sources are retrieved and cited", () => {
     const sourceOnly = ragResult({
       grounded: false,