fix(ingestion): phase-3 concurrency fixes — R11, R15/R16, R22, R24d

[BigSimmo]

Summary

Phase 3 of the ingestion-concurrency/scale review: implements the smallest safe fix for four confirmed state-machine violations from docs/ingestion-state-machine.md §6 (analysis merged to main; phase-2 scale review in docs/scale-readiness-review.md). One commit per finding, each with a regression test.

Commit	Finding	Class	Fix
5e60952	R11	DATA-LOSS, DET	DELETE route clears the storage_cleanup_jobs ledger's storage paths on every abort path, so the janitor can't destroy a live document's PDF/images after a transient delete failure.
95b1aac	R15/R16	DATA-LOSS, DET	Retry route no longer demotes an already-indexed document to queued (which forced the worker's destructive non-atomic reset of the live index); completed jobs are rejected (no zombie re-ingest).
3ca2870	R22	OPS-CHURN, DET	Queue recovery no longer supersedes (cancels) — or destructively retries — a plain pending reindex of an already-indexed document.
c0808b9	R24d	DATA-LOSS/SILENT-CORRUPTION	Reindex mode:'enrichment' refuses to run while a live indexing-v3 agent pass holds the document (prevents the interleaved-delete "completed/good doc with zero artifacts" end state).

Per-finding detail

• R11 — the ledger row is created with the live doc's paths before the point of no return; every abort path only flipped status to failed and left paths populated, and the janitor (scripts/cleanup-storage.ts) removes ('pending','failed') rows without checking the doc still exists. Now cleared on abort; genuine post-delete storage failures keep their paths so the janitor can finish. New pure helper buildStorageCleanupJobUpdate.
• R15/R16 — retryDocumentQueueUpdate keeps indexed docs on the atomic reindex path (status untouched → old generation stays live until the new commit); ingestionJobRetryRejectionReason rejects completed jobs with 409.
• R22 — a pending job on an indexed doc is a legitimately-queued reindex, not an abandoned leftover; it is now left for the worker's atomic path. Failed/stale-processing jobs on indexed docs are still superseded.
• R24d — hasActiveAgentEnrichmentJob (+ pure isActiveAgentEnrichmentJob); a stale/abandoned agent lease does not block (no heartbeat), steady-state pending/completed never block.

Scope notes / deliberate deferrals

• R15/R16 file surface: the fix lives in src/app/api/ingestion/jobs/[id]/retry/route.ts, just outside the branch's stated file surface, but it is the concrete site of a headline DATA-LOSS finding. Flagging explicitly.
• R11 companion (out of surface): the narrower concurrent janitor vs. still-pending ledger row window needs the janitor-side guard in scripts/cleanup-storage.ts (skip rows whose document_id still resolves — FK is on delete set null, so a non-null id proves the doc is alive). Not in this branch's surface → follow-up.
• R24d/R24c companion: scoping deep-memory deletes to its own generated_by (and the worker-inline vs agent overlap, R24c) is left as follow-up — it needs live verification of the generated_by data model.
• Migration/RPC-body findings deferred: R1/R2 (lease heartbeat + locked_by fences), R5, R7, R9, R17, R23, R24e drift. These require editing live RPC bodies via migrations. schema.sql is known to diverge from live in load-bearing places (R24e), so authoring create or replace function migrations from schema.sql risks writing the drift into live. They need the live function source + a live-verification window and per the branch rules must not be applied blind. Recommend a dedicated follow-up with live Supabase access.

Verification

• npm run verify:cheap — green (lint + typecheck + 1253 tests passed, 1 skipped)
• npm run verify:ui — N/A (no UI/routing/styling change)
• npm run verify:release — not run (no release/handoff claim)
• npm run format:check — prettier clean on changed files
• npm run eval:retrieval:quality — N/A: no retrieval/ranking/selection/chunking/scoring change (queue-state + delete/enrichment concurrency guards only)
• npm run eval:rag / eval:quality — N/A (no answer-generation change)
• npm run check:production-readiness — could not run here: requires live Supabase creds (this env is demo mode per check:supabase-project). Please run with live keys.

Domain gates npm run check:indexing and npm run reindex:health (requested per-fix gates) both require live Supabase creds and exit 1 in this demo-mode environment (NEXT_PUBLIC_SUPABASE_URL / SUPABASE_SERVICE_ROLE_KEY unset). They need a live run by a maintainer.

Clinical Governance Preflight

• Source-backed claims still require linked source verification before clinical use — unchanged
• No patient-identifiable document workflow introduced/expanded
• Supabase target remains Clinical KB Database (sjrfecxgysukkwxsowpy) — no env/config change
• Service-role keys and private document access remain server-only — routes unchanged in auth posture
• Demo/synthetic content remains clearly separated (demo-mode guards untouched)
• Source metadata, review status, and outdated/unknown-source behavior remain conservative — unaffected
• Deployment classification/TGA SaMD impact — no clinical decision-support behavior changed (queue/storage concurrency only)

Notes

• Each fix is a pure-application-code change; the four new/edited helpers are unit-tested (tests/ingestion.test.ts, tests/ingestion-recovery.test.ts, tests/ingestion-mutation-safety.test.ts).
• No migrations included; no live database modified.

🤖 Generated with Claude Code

[supabase]

This pull request has been ignored for the connected project sjrfecxgysukkwxsowpy because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.

---

Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.