feat: harden public API limits, server-only env boundaries, and registry corpus sync

.prettierignore

@@ -4,6 +4,7 @@ out/
 build/
 coverage/
 test-results/
+test-results-codex-remediation/
 playwright-report/
 sample-documents/
 dev-server*.log

AGENTS.md

@@ -164,7 +164,7 @@ When a branch or PR review completes, record the reviewed branch/ref, HEAD SHA,
 
 # Process hardening phases
 
-- For non-trivial source/config/test changes, prefer `npm run verify:cheap` as the first broad gate.
+- For non-trivial source/config/test changes, prefer `npm run verify:cheap` as the first broad gate and `npm run verify:pr-local` before PR handoff when the change is ready. The PR-local gate runs format plus `verify:cheap`, then conditionally adds the production build/client-bundle scan and code-backed offline RAG tests. Browser, dependency-audit, Docker/Supabase replay, and provider-backed checks remain separate gates. Use `npm run verify:pr-local -- --dry-run --files <comma-separated paths>` to inspect selection without running commands. The broader `--extended` plan is dry-run only unless explicit approval is reflected by `ALLOW_EXTENDED_PR_LOCAL=true`.
 - For UI, frontend, browser, routing, styling, reduced-motion, or forced-colors changes, run `npm run ensure` before browser work and use `npm run verify:ui` as the Chromium UI gate.
 - For release or handoff confidence, use `npm run verify:release`; this includes the full Playwright project set.
 - For clinical ingestion, answer generation, source governance, privacy, production-readiness, or environment changes, run the smallest relevant domain check plus `npm run check:production-readiness`.
@@ -414,4 +414,4 @@ Durable notes for Cloud Agents. Standard commands live in `README.md` and `packa
 - Live-mode caveat: `RAG_PROVIDER_MODE=auto` attempts OpenAI (fast → strong route); if generation fails the built-in quality gates it silently degrades to a deterministic "Source-only" answer that still cites real documents — this is expected, not a failure. The header sign-in UI exposes magic-link + OAuth only (no password field), but the `/api/answer` + retrieval flow works server-side without a browser session.
 - What still won't run in this VM even with secrets: `npm run worker` also needs the Python OCR stack (`worker/python/requirements.txt`) and heavy parsing deps; Supabase edge functions need Deno v2.x + deployment. `verify:release` additionally runs governance/eval gates. Treat missing-secret failures of `check:supabase-project`/`verify:release` in demo mode as expected, not regressions.
 - Dev server: `npm run dev` selects a stable per-project localhost port (e.g. `4461`), binds `0.0.0.0`, and prints the exact URL. Never assume port 3000/3001/3002. `npm run ensure` starts/verifies it in the background.
-- Verification without secrets: `npm run lint`, `npm run typecheck`, and `npm run test` (vitest) all pass offline. `npm run verify:cheap` also runs `check:runtime` + `sitemap:check` and is safe offline.
+- Verification without secrets: `npm run lint`, `npm run typecheck`, and `npm run test` (vitest) all pass offline. `npm run verify:cheap` also runs runtime, GitHub Actions pin, CI-scope, and sitemap checks. `npm run verify:pr-local` adds format, conditional build/client-bundle scanning, and code-backed offline RAG tests; browser, Docker/Supabase, audit, and provider checks remain separate.

docs/branch-review-ledger.md

@@ -42,3 +42,4 @@ Use this ledger to prevent repeated branch and PR reviews when the reviewed HEAD
 | 2026-07-11 | PR #466 / claude/search-timeout-failure-s6aiuj      | 54d52292eeb9e1c7856b3dad89d1b72e0d49fd53 | open-PR review, unresolved comments, and CI | P2 fixed: SSE progress/token/error emission now tolerates a client cancellation racing an enqueue, so the catch path cannot throw while reporting the original stream error. No additional high-confidence defect was found in the six-file diff.                                                                                                                                                                                                                                                                                                                                                                                                                  | Focused SSE and search utility Vitest (13/13); TypeScript; focused Prettier. Hosted advisory browser failure was shared stale assertion drift and is rerun after this push.                                                                                                                                                        |
 | 2026-07-11 | PR #483 / claude/differentials-page-review-a3daaf   | 36cca1bf7c13718dcc60a61b75272c7c4fa5cd44 | open-PR review, unresolved comments, and CI | P2 fixed: authenticated diagnosis detail responses now derive related links, overlap links, and comparison presentation from the owner's current diagnosis and presentation rows rather than the bundled snapshot. Added an owner-only catalog regression test. No additional high-confidence defect was found in the changed scope.                                                                                                                                                                                                                                                                                                                               | Focused differentials route/catalog Vitest (26/26); TypeScript; focused Prettier; `git diff --check`. Production readiness ran fail-closed with provider variables cleared and reported only expected missing provider configuration.                                                                                              |
 | 2026-07-11 | PR #489 / claude/document-viewer-redesign-55b68b    | 9130c8b15a22dbbc965464a247ae930c04f2da62 | open-PR review, unresolved comments, and CI | P2 fixed: document deep links now expand the mobile indexed-text details and scroll the branch-specific visible mobile or desktop chunk instead of the first duplicated DOM match. Added focused desktop/mobile assertions. No additional high-confidence defect was found in the three-file diff.                                                                                                                                                                                                                                                                                                                                                                 | Focused Prettier; TypeScript; `git diff --check`. Browser proof delegated to hosted CI because Turbopack rejects the isolated worktree's external node_modules junction.                                                                                                                                                           |
+| 2026-07-11 | PR #488 / claude/code-review-42a2c3                 | 7a8ea145013444f7cc29631499f48a8b0454937a | open-PR review, unresolved comments, and CI | Confirmed the remaining public error-code finding was already fixed at the reviewed head. Added the two focused advisory UI assertion stabilizations required by the hosted failure logs; no additional high-confidence defect was found in the changed scope.                                                                                                                                                                                                                                                                                                                                                                                                     | `tests/http-error-response.test.ts` (3/3); Prettier check on affected files; `git diff --check`; hosted required CI passed before the test-only fix. Browser rerun deferred to hosted CI because Turbopack rejects the isolated worktree's external node_modules junction.                                                         |

docs/database-drift-detection.md

@@ -1,6 +1,6 @@
 # Database drift detection (`npm run check:drift`)
 
-Last updated: 2026-07-07
+Last updated: 2026-07-10
 
 This repo's worst operational incidents were live-vs-repo schema drift: hybrid
 retrieval RPCs silently broken on live for an unknown period, and migrations
@@ -99,6 +99,13 @@ live project need explicit operator approval.
 > found live had diverged _forward_ from its retrieval bodies (item 0, new).
 > Live is under active concurrent multi-session editing, so the allowlist is a
 > point-in-time snapshot needing periodic regeneration.
+>
+> **2026-07-10 update:** local `check:drift` surfaced five repo-ahead live debts
+> for the July 8 hardening batch: fail-closed `retrieval_owner_matches`, R17's
+> one-open-ingestion-job index, and R5's document metadata deep-merge helpers /
+> `commit_document_index_generation` body. These are now allowlisted as known
+> pending live-apply work. Applying them to live remains an explicit
+> operator-approved migration action.
 
 0. **NEW — forward-codify the live-ahead retrieval RPCs** (was the "apply
    20260705210000" item, inverted). Live carries newer raw-SQL retrieval bodies
@@ -124,6 +131,13 @@ live project need explicit operator approval.
 2. ✅ **DONE 2026-07-08** — `20260703030000`'s effects (recorded-but-absent on
    live) re-applied via `20260708000000_reapply_storage_cleanup_jobs_indexes`;
    live storage_cleanup_jobs indexes now match schema.sql.
+   2a. **Apply the remaining July 8 hardening batch to live**: run the approved
+   migration workflow for `20260708160001_retrieval_owner_matches_fail_closed`,
+   `20260708170000_ingestion_jobs_one_open_per_document`, and
+   `20260708310000_r5_document_metadata_merge`. Remove the matching allowlist
+   entries for `retrieval_owner_matches`, `ingestion_jobs_one_open_per_document_uidx`,
+   `jsonb_merge_deep`, `apply_document_metadata_patch`, and
+   `commit_document_index_generation` after `check:drift` verifies live parity.
 3. **Codify the remaining live-only functions**: `get_visual_evidence_cards`,
    `repair_enrichment_quality_batch`, `run_all_visual_eval_cases`,
    `run_visual_eval_case` (same pattern as `20260707000000`).

docs/deployment-architecture.md

@@ -69,9 +69,12 @@ them only after the shared `rag_response_cache` hit rate is confirmed healthy.
 - `node:24-bookworm-slim` in all stages — respects `engines`/`engine-strict`
   and the `preinstall` engine guard.
 - The build stage runs the repo's own `npm run build`
-  (`guard-next-build.mjs` + `next build --webpack`) — **the image build fails
-  exactly where a local build would**. The build allocates an 8 GiB heap; give
-  the Docker builder ≥ 10 GiB memory.
+  (`guard-next-build.mjs` + `next build --webpack` + the client-bundle secret
+  scan) — **the image build fails exactly where a local build would**. The
+  `--webpack` flag is deliberate: `next.config.ts` carries a webpack-specific
+  WasmHash workaround and the CSP-nonce work was validated against webpack
+  prod chunks, so switching bundlers needs its own verified change. The build
+  allocates an 8 GiB heap; give the Docker builder ≥ 10 GiB memory.
 - `NEXT_PUBLIC_SUPABASE_URL` and `NEXT_PUBLIC_SUPABASE_PUBLISHABLE_KEY` are
   build args (they inline into the client bundle). The publishable key is
   public by design; the placeholder default exists so CI can build without

docs/operator-apply-july8-batch.md

@@ -23,15 +23,15 @@ is live — `worker/main.ts` already passes `p_worker_id` to completion RPCs.
 
 All steps below are safe through a single `supabase db push` when the ingestion
 queue is quiet. R17 uses its **own migration version** (`20260708170000`, not
-`20260708160000`) so history/repair cannot collide with the fail-closed tenancy
-migration at `20260708160000`.
+`20260708160001`) so history/repair cannot collide with the fail-closed tenancy
+migration at `20260708160001`.
 
 | Step | Migration                                                 | What                                         | How                                            |
 | ---- | --------------------------------------------------------- | -------------------------------------------- | ---------------------------------------------- |
 | 1    | `20260708140000_drop_ingestion_job_stages_job_id_fk.sql`  | R24e — drop phantom FK from fresh-env schema | Normal `supabase db push` (no-op on live)      |
 | 2    | `20260708130000_ingestion_concurrency_rpc_hardening.sql`  | R1/R2 lease fences, R7/R9/R23 RPC hardening  | Normal push — **apply before worker redeploy** |
 | 3    | `20260708150000_ensure_retrieval_owner_matches.sql`       | Ensure helper exists before fail-closed      | Normal push                                    |
-| 4    | `20260708160000_retrieval_owner_matches_fail_closed.sql`  | Tenancy fail-closed (#409)                   | Normal push                                    |
+| 4    | `20260708160001_retrieval_owner_matches_fail_closed.sql`  | Tenancy fail-closed (#409)                   | Normal push                                    |
 | 5    | `20260708310000_r5_document_metadata_merge.sql`           | R5 metadata deep-merge (#408)                | Normal push (safe before worker)               |
 | 6    | `20260708170000_ingestion_jobs_one_open_per_document.sql` | R17 one-open-job index (#405)                | Normal push when queue quiet — see below       |
 
@@ -48,7 +48,7 @@ create unique index concurrently if not exists ingestion_jobs_one_open_per_docum
   where status in ('pending', 'processing');
 ```
 
-Then mark only the R17 version as applied (never `20260708160000`):
+Then mark only the R17 version as applied (never `20260708160001`):
 
 ```bash
 supabase migration repair --linked --status applied 20260708170000

docs/rag-hybrid-findings-and-todo.md

@@ -303,25 +303,28 @@ denied to set parameter`)** — the RC11 blocker. The only method hosted allows
     answer-confidence label — "high" requires a genuine-cosine citation; synthetic-origin
     evidence caps at "medium" (strictly tightening, ordering/routing untouched, unit-tested in
     tests/rag-score.test.ts).
-22. 🔶 **Registry-to-corpus embedding (universal search Phase 5) — implementation restored and live
-    owner embedded (2026-07-09).** Medications/services/forms/differentials are federated into
+22. ✅ **Registry-to-corpus embedding (universal search Phase 5) — implemented behind the
+    default-off flag, live owner embedded, and blocking gates clean (2026-07-09).** Medications/services/forms/differentials are federated into
     `/api/search/universal` but were not retrieval-corpus entities, so Answer mode could not cite them.
     Implemented pieces: `RAG_REGISTRY_CORPUS_EMBEDDING` default-off flag,
     `scripts/embed-registry-records.ts` dry-run/write/list-owner tool, synthetic document/chunk mapping
     with `metadata.source_kind = 'registry_record'`, source-governance labelling, comparator tooling,
+    seed/reseed embedding paths, `reembedRegistryRecordAfterEdit` / `bestEffortReembedRegistryRecordAfterEdit`,
     and registry corpus tests. The sentinel-owner dry-run
     (`00000000-0000-0000-0000-000000000000`) correctly found zero rows. `--list-owners` found the real
     registry owner `4f1b3c19-3c39-4597-b9df-168c8e6007ff` with 739 eligible rows; guarded write with
     `RAG_REGISTRY_CORPUS_EMBEDDING=true --write --confirm` upserted 739 synthetic registry corpus
     chunks. Post-write retrieval eval passed with `document_recall_at_5=1`, `content_recall_at_5=1`,
     `top_k_hit_rate=1`, `force_embedding_failure_count=0`, `failed_cases=[]`. Post-write
     `eval:quality -- --rag-only` completed under budget; invented-term controls still refused and
-    numeric grounding failure rate was `0`. Remaining blockers are not registry regressions:
-    citation failure rate `0.0227` and RAG latency thresholds (`p95=44847ms`; route p95 extractive
-    `46745ms`, fast `27131ms`, strong `63613ms`). Remaining implementation before treating this as
-    fully productized: re-embed-on-edit hooks for registry updates and a dedicated answer-mode UX check
-    that registry-backed citations are labelled as curated registry records rather than primary source
-    documents.
+    numeric grounding failure rate was `0`. The post-routing RAG-only rerun then cleared the former
+    blocker metrics: `citation_failure_rate=0`, `numeric_grounding_failure_rate=0`, no blocking threshold
+    failures, and `p95_latency_ms=20385`. Two individual cases still exceeded the 20-second latency target;
+    that non-blocking performance debt remains tracked in item 25. Current productization boundary: there are no mutating registry
+    edit routes today, so the re-embed-on-edit helper is present but not wired to a route; any future
+    registry `POST`/`PATCH`/`PUT` path must call `bestEffortReembedRegistryRecordAfterEdit` after the
+    write commits. Remaining non-blocking UX follow-up: an answer-mode check that registry-backed
+    citations render as curated registry records rather than primary source documents.
 23. ⏳ **Finding #11 full fix (RAG optimisation Phase 2)** — the classifier-verdict memo (shipped
     2026-07-06) makes zero-result behaviour deterministic per query but does not close the gap:
     the deterministic analyzer still cannot tell in-corpus topics from out-of-corpus ones.
@@ -333,9 +336,15 @@ denied to set parameter`)** — the RC11 blocker. The only method hosted allows
     (compare table-cell tokens against the document's own clean chunk text — "p ycho ocial"
     aligns to "psychosocial" within the same page's raw text) rather than heuristic detection at
     query time. Scope to `worker/` table extraction; requires the Python OCR stack to test.
-25. ⏳ **Retrieval latency p90 ~8.6s (local)** — remaining sequential layers after the 2026-07-01
-    parallelisation. Cheapest next step (measure first): overlap `embedTextWithTelemetry` with
-    the text fast path unconditionally (today preload only fires when `shouldPreloadEmbedding`),
-    and collapse the repeated `attachDocumentRankingMetadata` calls to one batched fetch per
-    request. Both are perf-only; gate with the golden eval unchanged + p90 from
-    `rag_retrieval_logs` before/after.
+25. ⏳ **Retrieval/RAG latency remains a performance backlog, not a current gate blocker.**
+    Post-registry retrieval stayed quality-clean (`top_k_hit_rate=1`, `document_recall_at_5=1`,
+    `content_recall_at_5=1`, `force_embedding_failure_count=0`) with local `p90_latency_ms=13145`.
+    Post-routing RAG-only passed with `p95_latency_ms=20385` and no blocking threshold failures, but
+    local-machine→remote-DB latency and remaining sequential layers still justify a dedicated perf pass.
+    Next step (measure first): instrument the existing per-request `documentRankingMetadataCache` to
+    quantify incremental cache misses/fetches before changing enrichment, then test starting
+    `embedTextWithTelemetry` concurrently only when `forceEmbedding` is set or routing has already ruled
+    out an accepted lexical/document fast path. Do not preload embeddings unconditionally: preserve
+    source-only and lexical-only behavior, and record provider-call count plus whether each embedding
+    result was consumed so any p90 gain is weighed against cost. Gate with the golden eval unchanged,
+    provider usage unchanged or explicitly accepted, and p90 from `rag_retrieval_logs` before/after.