diff --git a/.claude/hooks/issues-surface.sh b/.claude/hooks/issues-surface.sh index dab08f83d..f22141372 100755 --- a/.claude/hooks/issues-surface.sh +++ b/.claude/hooks/issues-surface.sh @@ -43,6 +43,22 @@ rows="$(awk ' } ' "$ledger" 2>/dev/null || true)" +# --- parse the recommended execution queue ---------------------------------- +queue_rows="$(awk ' + /^## Recommended execution queue/ { inqueue=1; next } + /^## / { if (inqueue) inqueue=0 } + inqueue && /^\|[[:space:]]*[0-9]+[[:space:]]*\|/ { + n=split($0, c, "|") + ord=c[2]; ids=c[3]; acuity=c[4]; capability=c[5]; timing=c[6] + gsub(/^[ \t]+|[ \t]+$/, "", ord) + gsub(/^[ \t]+|[ \t]+$/, "", ids) + gsub(/^[ \t]+|[ \t]+$/, "", acuity) + gsub(/^[ \t]+|[ \t]+$/, "", capability) + gsub(/^[ \t]+|[ \t]+$/, "", timing) + printf "%s\t%s\t%s\t%s\t%s\n", ord, ids, acuity, capability, timing + } +' "$ledger" 2>/dev/null || true)" + total="$(printf '%s' "$rows" | grep -c . || true)" if [ "${total:-0}" -eq 0 ]; then echo "[issues] Outstanding-work memory (docs/outstanding-issues.md): no open items. Record one with /issues add …" @@ -56,6 +72,24 @@ c1="$(count "$p1")"; c2="$(count "$p2")"; c3="$(count "$p3")" echo "[issues] Outstanding-work memory — ${total} open (${c1}×P1, ${c2}×P2, ${c3}×P3). Source of truth: docs/outstanding-issues.md · read the full list back with /issues." +queue_total="$(printf '%s' "$queue_rows" | grep -c . || true)" +if [ "${queue_total:-0}" -gt 0 ]; then + echo "[issues] Recommended execution queue — ${queue_total} retained tasks (first 8):" + printf '%s\n' "$queue_rows" | head -n 8 | while IFS=$'\t' read -r ord ids acuity capability timing; do + echo " ${ord}. ${ids} · ${acuity} · ${timing} · ${capability}" + done +fi + +# Keep the priority summary complementary to the queue instead of repeating +# the same recommended IDs in both sections. +queued_ids=" $(printf '%s\n' "$queue_rows" | grep -oE '#[0-9]+' | tr '\n' ' ' || true)" +unqueued_rows="$(printf '%s\n' "$rows" | awk -F'\t' -v queued="$queued_ids" ' + index(queued, " " $2 " ") == 0 +' || true)" +ungroup() { printf '%s\n' "$unqueued_rows" | awk -F'\t' -v p="$1" '$1==p'; } +u1="$(ungroup P1)"; u2="$(ungroup P2)"; u3="$(ungroup P3)" +uc1="$(count "$u1")"; uc2="$(count "$u2")"; uc3="$(count "$u3")" + print_group() { # $1=rows $2=max-to-list local data="$1" limit="$2" shown=0 more=0 pri id typ sum [ -z "$data" ] && return 0 @@ -75,9 +109,9 @@ EOF } # P1 = do-next, list all. P2 = should-do, list up to 8. P3 = collapse to a count. -[ "$c1" -gt 0 ] && print_group "$p1" 999 -[ "$c2" -gt 0 ] && print_group "$p2" 8 -[ "$c3" -gt 0 ] && echo " ${c3} × P3 (nice-to-have / revisit-when) — see /issues" +[ "$uc1" -gt 0 ] && print_group "$u1" 999 +[ "$uc2" -gt 0 ] && print_group "$u2" 8 +[ "$uc3" -gt 0 ] && echo " ${uc3} × P3 (nice-to-have / revisit-when) — see /issues" # --- capture reminder --------------------------------------------------------- case "$source_val" in diff --git a/.claude/skills/issues/SKILL.md b/.claude/skills/issues/SKILL.md index 228433931..ff420ba7c 100644 --- a/.claude/skills/issues/SKILL.md +++ b/.claude/skills/issues/SKILL.md @@ -5,9 +5,9 @@ description: Track and recall all outstanding tasks, recommendations, and issues # issues — the outstanding-work memory -`docs/outstanding-issues.md` is the durable, cross-session memory of everything still outstanding: -open **tasks**, **recommendations** not yet acted on, and **issues** not yet resolved. Chat context -resets; that file does not. This skill reads it back and keeps it current. +`docs/outstanding-issues.md` is the single universal, durable, cross-session ledger for recommended +execution order, open **tasks**, **recommendations**, **issues**, provider/operator work, and archive +history. Chat context resets; that file does not. This skill reads it back and keeps it current. **The ledger is the source of truth, not chat memory.** Never answer `/issues` from conversation recall — always read the file first, so the answer is correct even in a fresh session. @@ -20,13 +20,15 @@ recall — always read the file first, so the answer is correct even in a fresh ## Default: `/issues` (read-only) 1. Read `docs/outstanding-issues.md`. -2. State the **open items** back, grouped by priority (P1 → P3), each as +2. State the **Recommended execution queue** back in order, including acuity, timing, and gate. +3. Summarize any open items not represented in that queue, grouped by priority (P1 → P3), each as `#ID · type · summary — next action (source)`. -3. End with a one-line count, e.g. `5 open: 0×P1, 3×P2, 2×P3 · 0 resolved this session`. -4. Do **not** mutate the file or commit on a plain read. +4. End with a one-line open/recommended count. +5. Do **not** mutate the file or commit on a plain read. -If a filter is given, narrow step 2: `/issues P1` (by priority), `/issues issues` / `/issues recs` -/ `/issues tasks` (by type), `/issues ` (summary/detail substring match). +If a filter is given, filter the open items before rendering steps 2–3, then show only matching +queued tasks and matching non-queued items: `/issues P1` (by priority), `/issues issues` / +`/issues recs` / `/issues tasks` (by type), `/issues ` (summary/detail substring match). ## Mutating subcommands @@ -53,6 +55,10 @@ paragraph; put the smallest next action in **Detail / next action**. ## Writing rules - Keep the table format and column order exactly as in `docs/outstanding-issues.md`. One row per item. +- Add a retained task to the recommended queue with order, acuity, capability, timing, estimate, + gate, success criteria, verification, and stop rule. Reorder rather than duplicate related work. +- Remove a task from the recommended queue when it completes or is no longer recommended; retain + its evidence in the open or resolved table as appropriate. - IDs are monotonic and never reused — always allocate from the `issues:next-id` marker and bump it. - Escape `|` inside cell text (write `\|`) so the markdown table stays intact. - Respect the repo's RAG/clinical/privacy flagging rules if an item _itself_ touches a protected diff --git a/AGENTS.md b/AGENTS.md index 32d9bc6c0..13deab25f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -450,13 +450,18 @@ Run the matching planner command in `docs/productivity-workflows.md` without sid ## Outstanding-work memory (`/issues`) -`docs/outstanding-issues.md` is the durable, cross-session memory of every outstanding **task**, -**recommendation**, and **issue** for this repo. Chat context resets between sessions; that file does -not, so anything worth remembering after a session ends belongs there. +`docs/outstanding-issues.md` is the single universal, durable, cross-session ledger for every +outstanding **task**, **recommendation**, and **issue** in this repo. It owns evidence, resolution +history, recommended order, acuity, capability, timing, effort, approvals, verification, and stop +rules. Chat context resets; this file does not, so anything worth remembering belongs there. +Detailed runbooks such as `docs/operator-backlog.md` may support a task but must not become a second +status ledger. Update the universal ledger when work completes, is dropped, becomes stale, or is +materially re-scoped. Never restore completed, duplicate, speculative, superseded, or rejected work +to the recommended queue. - When the user types `/issues`, invoke the `issues` skill (`.claude/skills/issues/SKILL.md`): read - `docs/outstanding-issues.md` and state the open items back, grouped by priority. A plain `/issues` - is read-only — it mutates and commits nothing. + `docs/outstanding-issues.md`, state the recommended queue in order, then summarize other open + items by priority. A plain `/issues` is read-only — it mutates and commits nothing. - `/issues add|done|update|capture …` mutate the ledger; each mutation commits **only** `docs/outstanding-issues.md` (no push unless the user asks or you are already handing off). - Proactively offer to `capture` unresolved follow-ups, deferrals, and known risks into the ledger diff --git a/docs/README.md b/docs/README.md index 0b743e80c..cb90326ff 100644 --- a/docs/README.md +++ b/docs/README.md @@ -45,7 +45,8 @@ npm run docs:check-links - [supabase-migration-reconciliation.md](supabase-migration-reconciliation.md) — migration drift and repair policy - [observability-slos.md](observability-slos.md) — health probes, SLO counters, degraded modes - [openai-rag-operations.md](openai-rag-operations.md) — OpenAI/RAG provider operations and modes -- [operator-backlog.md](operator-backlog.md) — pending operator debt backlog +- [outstanding-issues.md](outstanding-issues.md) — single universal task ledger and repository memory +- [operator-backlog.md](operator-backlog.md) — provider/operator runbook detail (status is canonical in the universal ledger) - [deploy-corrector-public-titles.md](deploy-corrector-public-titles.md) — public-title corrector deploy notes ## Governance, safety, privacy diff --git a/docs/branch-review-ledger.md b/docs/branch-review-ledger.md index b9b2a8b96..65d17fc47 100644 --- a/docs/branch-review-ledger.md +++ b/docs/branch-review-ledger.md @@ -20,7 +20,8 @@ Use this ledger to prevent repeated branch and PR reviews when the reviewed HEAD | Date | Branch or ref | Reviewed HEAD | Scope | Outcome | Checks | | ---------- | -------------------------------------------------------- | ---------------------------------------- | ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 2026-07-24 | `codex/supabase-document-change-trigger` | `9c7d9edf509a51478f5bebbabcca64e3926dc877` + reviewed working diff | Document-change ingestion trigger migration, schema mirror, grants, privacy and fail-safe delivery | APPROVE. No P0-P2 finding. The trigger is update-only, acts solely on a strict JSON boolean false/absent-to-true transition, sends only the receiver's allowlisted owner-scoped fields, fails open for document writes when Vault/GUC/pg_net is unavailable, and revokes execution from public/anon/authenticated. No production URL fallback exists. Highest residual risk is deliberate pg_net at-most-once delivery; the clear-then-flip recovery and data-preserving rollback are documented, and the trigger remains inert until both the Vault secret and environment base-URL GUC are configured. | Disposable `supabase/postgres:17.6.1.127` schema replay and drift-manifest regeneration passed (16s; scratch container removed); focused schema/drift/receiver Vitest 89/89; migration-role, function-grant (30 SECURITY DEFINER functions) and owner-scope guards; production-readiness CI mode READY with expected secretless-worktree warnings; offline RAG 21 suites/307 tests; `verify:cheap` 365 files, 3,241 passed/1 skipped; static trace of receiver payload, authoritative owner-scoped reload and idempotent enqueue path. No live provider mutation or migration apply. | +| 2026-07-24 | PR #1106 / `codex/task-ledger-final-11318f` | `5d128a2844c2298d0da36df64e5e2f7dda11e14b` + reviewed follow-up diff | Universal task-ledger workflow and protected-main merge readiness | APPROVE after follow-up. `docs/outstanding-issues.md` is the single durable task ledger, with retained work carrying order, acuity, timing, capability, effort, dependencies, success criteria, verification and stop rules. Four actionable review findings were fixed: filtered `/issues` reads now apply the filter to open items before rendering queued and non-queued results; the session hook excludes queued IDs from its priority summary; `#030` is consistently P2/A2 in the canonical open table and queue; and the sole A1/P1 blocker is first while `#052` is explicitly the first code task. No other actionable review thread remains in the reviewed scope. | Protected CI at the initial reviewed head passed policy, static, safety/config, unit coverage, Semgrep, Gitleaks, GitGuardian and the required aggregate; UI, build, migration replay and release browser matrix were correctly skipped for the docs/workflow scope. Follow-up proof: scoped Prettier; hook syntax/runtime plus exact ID-deduplication, P2-count and A1-first assertions; docs links (1,136 references); canonical skill catalog (32 skills, 8 aliases); `git diff --check`. Exact-head hosted CI remains required after the follow-up push. No OpenAI, Supabase, Railway, deployment or production-data operation ran. | +| 2026-07-24 | `codex/supabase-document-change-trigger` | `9c7d9edf509a51478f5bebbabcca64e3926dc877` + reviewed working diff | Document-change ingestion trigger migration, schema mirror, grants, privacy and fail-safe delivery | APPROVE. No P0-P2 finding. The trigger is update-only, acts solely on a strict JSON boolean false/absent-to-true transition, sends only the receiver's allowlisted owner-scoped fields, fails open for document writes when Vault/GUC/pg_net is unavailable, and revokes execution from public/anon/authenticated. No production URL fallback exists. Highest residual risk is deliberate pg_net at-most-once delivery; the clear-then-flip recovery and data-preserving rollback are documented, and the trigger remains inert until both the Vault secret and environment base-URL GUC are configured. | Disposable Supabase Postgres `17.6.1.127` schema replay and drift-manifest regeneration passed (16s; scratch container removed); focused schema/drift/receiver Vitest 89/89; migration-role, function-grant (30 SECURITY DEFINER functions) and owner-scope guards; production-readiness CI mode READY with expected secretless-worktree warnings; offline RAG 21 suites/307 tests; `verify:cheap` 365 files, 3,241 passed/1 skipped; static trace of receiver payload, authoritative owner-scoped reload and idempotent enqueue path. No live provider mutation or migration apply. | | 2026-07-23 | PR #1090 / `cursor/fix-phone-dock-edge-1b1d` | `761de7e9ad623b6bd8d634d849a9eb465d622e48` (merged as `09028ef217209fceb53f1122ac7738b509bce323`) | Phone safe-area and edge-to-edge search-dock UI review | MERGED. No P0-P2 finding. The branch was three commits behind, so current `origin/main` was merged before landing; the actual merge tree matched the reviewed synthetic tree. The dock remains flush to the viewport with safe-area padding inside the form, and the phone shell no longer retains the `dvh` clamp that created the Safari toolbar band. Zero actionable review threads. | `npm run ensure`; focused `ui-tools.spec.ts` phone-home and edge-to-edge scenarios: Chromium 2/2 and WebKit 2/2; refreshed hosted policy, security, unit, build, advisory UI, Production UI and required aggregate checks green; exact-head ancestry and local-main tree equality proved after merge. | | 2026-07-22 | PR #1087 / `codex/reconcile-product-truth` | `edbc2260fef59ca2fa7c6973dffb85e32354bce1` (merged as `05dc52fd8408a65117e22a6236e43252203bea92`) | Product-truth copy, account persistence and unavailable-SSO presentation | MERGED. Cross-device claims now match favourites/preferences persistence; recent searches are identified as browser-session data; the contradictory “never shared” statement is removed. All unavailable setup providers and Apple elsewhere use the connected accessible “coming soon” placeholder pattern. The single review finding was fixed, replied to and resolved. | Red DOM proof; focused 19/19; `verify:cheap` 3,220 passed / 1 skipped; `verify:ui` 265/265; PR-local build/secret scan/offline RAG; final hosted required, Production UI, policy and security checks green. No provider calls or RAG spend. | | 2026-07-22 | PR #1086 / `codex/reconcile-xlsx-budgets` | `5376880a40749b6526fd7e4603a7be9d04bc9624` (merged as `2963fba46eacd644618a588fa283f7597faa2644`) | XLSX resource-boundary review | MERGED. Enforces worksheet, non-empty-row, rendered-cell and UTF-8 output ceilings before result fragments are appended; sparse-column output is preserved. No actionable review threads. | Red 257-sheet reproducer; focused 4/4; `verify:cheap` 3,218 passed / 1 skipped; PR-local build/scan/offline RAG; hosted required/security/policy green. | diff --git a/docs/codebase-index.md b/docs/codebase-index.md index c7d597ed5..aa92ad7ad 100644 --- a/docs/codebase-index.md +++ b/docs/codebase-index.md @@ -334,6 +334,7 @@ One shared composer (`master-search-header.tsx`) serves every mode. Placement: | Full documentation index | `docs/README.md` | | Routes and modes | `docs/site-map.md` | | Search/RAG roadmap | `docs/search-rag-master-plan.md` | +| Universal task ledger | `docs/outstanding-issues.md` | | Reindex operations | `docs/reindex-runbook.md` | | Production readiness | `docs/production-readiness-checklist.md` | | Capacity / scale-up | `docs/capacity-review.md`, `docs/auth-connection-cap-runbook.md` | diff --git a/docs/operator-backlog.md b/docs/operator-backlog.md index eb749675b..31d9ec96a 100644 --- a/docs/operator-backlog.md +++ b/docs/operator-backlog.md @@ -1,8 +1,9 @@ -# Operator backlog +# Operator action detail -Single source of truth for **human-only / provider-gated actions** that cannot be done from a coding -session (they touch Supabase, Railway, OpenAI, or GitHub settings, per the AGENTS.md provider boundary). -This exists so that launch-blocking state lives in the repo instead of chat memory. +Detailed runbook index for **human-only / provider-gated actions** that cannot be done from a coding +session. Canonical task status, order, acuity, and completion live only in +[`outstanding-issues.md`](outstanding-issues.md); this file supplies provider-specific steps and +presence claims that must be verified before acting. **How to use:** work top to bottom; each row links to the detailed runbook. `Status` values are `⏳ pending`, `🔎 verify` (may already be done — confirm before repeating), `✅ done`, `—` (n/a). diff --git a/docs/outstanding-issues.md b/docs/outstanding-issues.md index e31b22152..5c47129ec 100644 --- a/docs/outstanding-issues.md +++ b/docs/outstanding-issues.md @@ -1,12 +1,15 @@ -# Outstanding Issues, Recommendations & Tasks +# Universal Task Ledger — Outstanding Issues, Recommendations & Tasks -Durable, cross-session memory of everything still outstanding for this repo: open **tasks**, -**recommendations** not yet acted on, and **issues** not yet resolved. Chat context is ephemeral -(sessions summarise and reset); this file is not — it is the single source of truth the -[`/issues` skill](../.claude/skills/issues/SKILL.md) reads back and updates. +Durable, cross-session repository memory of everything still outstanding: open **tasks**, +**recommendations**, **issues**, ordered execution metadata, operator decisions, provider gates, and +resolution history. Chat context is ephemeral; this file is the single universal task ledger the +[`/issues` skill](../.claude/skills/issues/SKILL.md) and session-start hook read and update. **Rule of thumb:** if it is worth remembering after this session ends, it belongs here. +Detailed runbooks may live elsewhere, including [`operator-backlog.md`](operator-backlog.md), but +task status, priority, order, dependencies, and completion state are canonical only here. + ## How this is used - Say `/issues` in Claude Code → the skill reads this file and states the open items back, @@ -27,7 +30,54 @@ Durable, cross-session memory of everything still outstanding for this repo: ope - Resolving an item moves its row to **Resolved / archive** with the date and a one-line outcome — rows are archived, not deleted, so the history stays auditable. - +## Recommended execution queue + +This queue contains only work that remains recommended. Revalidate a row against current `main` +before starting. Its position is not authority to call providers, spend money, change production, +commit, push, open a PR, or deploy. + +Order numbers remain stable within a reconciliation snapshot. A gap means an item was completed or +removed after current-main verification; it is not missing recommended work. + +- **A1 — urgent:** active safety, privacy, data-loss, or release/launch blocker. One operator/legal + item is retained; there is no current A1 code defect. +- **A2 — important:** confirmed correctness, clinical, privacy, reliability, or evaluation work. +- **A3 — planned:** worthwhile work that is safe to defer until its trigger. +- **Optional:** start only when measured need, ownership, and cost justify it. +- **Capability:** Standard = established pattern; High = cross-module senior work; Specialist = + database/RAG/clinical/privacy expertise; Operator = named provider/product/legal authority. +- **Estimate:** focused active time, excluding approval, hosted runtime, soak, and review waits. + +| Order | ID(s) | Acuity | Capability | When | Estimate | Outcome, gate, verification, and stopping condition | +| ----: | ---------------------- | -------- | ------------------------------------------- | ------------------------------------------------------------------ | ---------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 1 | `#053` | A1 | Operator — legal/privacy | Start now; finish before real patient use/privacy-approved release | 4–8 hours internal; 1–6 weeks elapsed | Execute DPAs; decide ZDR/residency; obtain cache behavior in writing; review subprocessors; obtain APP 8 and APP 5/1 counsel sign-off. Do not change public copy before approval. | +| 2 | `#052` | A2 | High — ingestion concurrency | Ready; first code task | 0.5–1 day | Add red single/bulk tests, then block full/retry reindex during a fresh agent-enrichment lease while keeping stale leases and enrichment mode unchanged. Run focused safety tests and `verify:cheap`; stop if current `main` cannot reproduce it. | +| 3 | `#030` | A2 | High — evaluation semantics | After order 2 | 2–4 hours | Require distinct source identities for distinct comparison slots. Run focused matching tests, typecheck, and `verify:cheap`; stop without changing strict aliases, retrieval, or ranking. | +| 4 | `#019` | A2 | Specialist — RAG answer pipeline | Local reproducer now; behavior change after order 8 | 0.5–1 day reproducer | Reproduce admission-source loss in the fallback layer using PR #1096’s source shape. Any behavior change needs protected review and an approved baseline/post canary; stop if independently non-reproducible. | +| 6 | `#054` | A2 | Standard locally; Operator hosted | Local safety identifier now; hosted next approved window | 15–30 min local; 1–2 hours hosted | Presence-check and fill confirmed secret/config gaps with distinct per-environment values. Never record values. Require clean readiness/secret checks; stop on ambiguous environment or project identity. | +| 7 | `#022` | A2 | Operator — clinical governance + Specialist | Decision-ready | 1–2 hours policy; 0.5–1 day first ten | Decide BMJ attestation policy and review the ten highest-impact local documents. Record reviewer/evidence/time; stop after ten and remeasure warning debt. | +| 8 | `#051`, `#023` | A2 | Specialist — RAG diagnostics | After scheduled 2026-07-26 run | 2–4 hours | With GitHub-read approval, compare structured canary/browser/irrelevant-at-10 artifacts without dispatching a rerun. Record deterministic/provider/latency deltas and disposition residuals; stop without spending. | +| 9 | `#018` | A2 | Specialist — clinical RAG/retrieval | After order 8, one mechanism at a time | 1–2 days diagnosis | Give lithium, ADHD, and metabolic residuals separate current-main reproducers and candidates. Behavior canaries require approval; stop any item without a deterministic reproducer or on regression. | +| 10 | `#029` | A2 | Specialist — answer quality/clinical safety | After orders 8–9 | 0.5–1 day inventory; 1–3 days per fix | Re-enumerate current fallback stubs and fix one causal cluster at a time without weakening grounding/citation gates. Stop if a change merely makes the metric easier to pass. | +| 11 | `#001` | A2 | Specialist — retrieval/ranking | After order 8 and rollout approval | 0.5–1 day plus canary | Keep semantic reranking off unless an approved ambiguity comparison preserves 36/36, recall 1.0, zero per-case regressions, and shows measured gain; otherwise record keep-off and stop. | +| 12 | `#025` | A2 | Operator — Railway/GitHub/chat/Supabase | Next approved observability window | 1–3 hours/channel | Choose owned deployment, CI, ingestion, and SLO alerts; mock first, then one approved controlled provider event/channel. The merged Supabase trigger remains inert until its verified inputs are configured. Stop without an accountable responder. | +| 13 | `#055` | A2 | Specialist release owner + Operator | Before next full-confidence release/handoff | 2–4 hours plus runtime | On one exact SHA, run local/provider gates, Firefox/WebKit, required hosted CI, and close actionable GitHub threads. Stop at first failure and rerun only the repaired smallest gate. | +| 14 | `#056` | A2 | Operator — Supabase/Railway + Specialist | After cost/ownership approval | 0.5–1 day | Provision isolated `Clinical KB Staging` with synthetic data and distinct secrets. Verify identity, schema, indexing, health, and data boundary; never copy production clinical documents. | +| 15 | `#057` | A2 | High — release/SRE + Operator | After order 14 | 2–4 hours plus soak | Run documented staging soak and rollback against an exact candidate. Retain latency/error/rollback evidence; stop on unsafe data, identity mismatch, or unowned rollback. | +| 16 | `#058` | A2 | Operator — production data + Specialist | Next approved production verification window | 30–60 min read-only; 1–2 hours if needed | Verify registry/differentials/medications are non-empty before writing; seed only confirmed gaps idempotently. Stop when healthy or owner/project identity is ambiguous. | +| 17 | `#007` | A3 | Operator decision + Standard frontend | When product chooses canonical Tools route | 15–30 min decision; 0.5 day | Align navigation, redirect, sitemap, and reachability around one entry point. Stop while the standalone page has an unresolved requirement. | +| 18 | `#011` | A3 | Operator — Supabase capacity | Immediately before first compute scale-up | 30–60 min plus observation | Switch Auth to percentage allocation, record before/after, and run approved advisor/health checks. Stop if no scale-up is planned. | +| 19 | `#017` | A3 | High — performance/browser | Before order 23; approved live-site window | 1–2 hours | Capture reproducible mobile/desktop Lighthouse/Web-Vitals evidence and decide whether payload work is justified. Stop if metrics are acceptable or evidence is too noisy. | +| 20 | `#024` | A3 | High — Next.js/Playwright/WebKit | After order 8 or real Safari reproduction | 0.5–1 day | Distinguish test interception from a Safari defect. Apply test-only correction only with a discriminating repro; keep access-control assertions meaningful. | +| 21 | `#033` | A3 | Specialist — prompt/source governance | After orders 7–8 | 1–2 days plus approved eval | Design unknown-vs-adverse metadata wording and prompt tests. Require no supported-grounding drop and zero citation failures; stop on broad over-caveating or degradation. | +| 22 | `#037` | A3 | Operator — clinical/product + Standard | Next trust-policy review | 30–60 min; up to 0.5 day | Decide whether routine claims cap at medium trust. Record policy; if accepted, change only the flag/expectations and run focused tests. | +| 23 | `#012`, `#013`, `#016` | A3 | High — bundling/runtime performance | After order 19 or equivalent evidence | 0.5–2 days/route | Optimize only a production route with measured payload/render/motion harm. Require material gain plus focused, `verify:cheap`, and browser evidence; stop on small gain. | +| 24 | `#027` | Optional | Operator — SRE/provider | When an owned external alert path is wanted | 1–2 hours | Decide vendor/cost/privacy/owner; if accepted, prove one non-PHI outage and recovery alert. Stop when no responder owns it. | +| 25 | `#028` | Optional | Specialist privacy/observability + Operator | After privacy/ownership/cost approval | 1–3 days | Define vendor/region/retention/redaction/sampling/source-map envelope before SDK work. Prove no clinical text, identifiers, or secrets leave; stop if unacceptable. | +| 26 | `#038` | Optional | High — product/design architecture | When a new comparison surface is approved | 0.5–1 day | Define a shared interaction contract without flattening mode-specific content. Stop when no concrete new surface exists. | +| 27 | `#040` | Optional | High — visual QA/accessibility | When baseline owner/update workflow exist | 1–2 days | Establish a small stable desktop/mobile/accessibility baseline set. Do not make it blocking if flake or maintenance cost outweighs detection value. | + + ## Open items @@ -39,6 +89,13 @@ Durable, cross-session memory of everything still outstanding for this repo: ope | ---- | --- | ----- | --------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | ---------- | | #051 | P2 | task | Stabilise the live answer-quality canary before more RAG tuning | Diagnostics landed in PR #1095: structured JSON/Markdown artifacts now record the actual checked-out SHA, run identity and latency context, and the offline trend tool separates content, provider-route and latency outcomes. First validating run `30018289898` recorded the expected tree and cost, with 36/36 retrieval green, but one report cannot establish variability; PR #1097 prevents a single failure being mislabeled as repeated. Next: compare the scheduled 2026-07-26 structured report with this run. Do not spend on an immediate retry or reapply the archived lithium guard before that comparison. | PR #1095; run `30018289898`; PR #1097; archive ref `refs/archive/rejected-rag/20260723/monitoring-subject-gate` | 2026-07-23 | | #001 | P2 | task | Semantic reranking still gated off | `RAG_SEMANTIC_RERANK_ENABLED=false` from PR #901. Do not enable until the provider-backed 36/36 retrieval-quality gate **and** an ambiguity-focused canary are explicitly approved and recorded. | `docs/process-hardening.md` (Semantic reranking rollout debt); PR #901 | 2026-07-21 | +| #052 | P2 | issue | Reindex can overlap a fresh agent-enrichment pass | Full/retry single and bulk routes consult `ingestion_jobs` but not the implemented `hasActiveAgentEnrichmentJob` predicate. Add red route tests, then block fresh `indexing_v3_agent_jobs.status='processing'` leases before mutation while keeping stale leases and enrichment mode unchanged. | `src/lib/ingestion-mutation-safety.ts`; single/bulk reindex routes; repository audit 2026-07-24 | 2026-07-24 | +| #053 | P1 | task | Execute cross-border privacy/legal package | Execute OpenAI and Railway DPAs; decide ZDR and Australian data residency; obtain prompt-cache behavior in writing; review subprocessors; obtain APP 8 and APP 5/1 counsel sign-off. Do not represent the release as privacy-approved or alter final public privacy wording before sign-off. | `docs/openai-cross-border-basis.md`; `docs/privacy-impact-assessment.md` | 2026-07-24 | +| #054 | P2 | task | Reconcile local and hosted secrets/config | Presence-check safety-identifier, query-hash, deep-probe, Supabase service-role, OpenAI, project-identity, and schedule settings; set only confirmed gaps with distinct per-environment values. Never record secret values. Provider reads/writes require approval. | `.env.example`; production-readiness warning; `docs/operator-backlog.md` | 2026-07-24 | +| #055 | P2 | task | Run one exact-SHA full release and PR gate | Before the next full-confidence release/handoff, record the candidate/PR SHA and run the local/provider release gates, Firefox/WebKit, required hosted CI, and actionable GitHub review-thread closure once. Stop at the first actionable failure and rerun only the repaired smallest gate. | `docs/launch-operator-runbook.md`; `docs/codex-review-protocol.md` | 2026-07-24 | +| #056 | P2 | task | Provision isolated staging environment | After explicit cost/ownership approval, provision `Clinical KB Staging` Supabase and Railway tiers with distinct secrets and synthetic/non-clinical data. Verify identity, schema, indexing, health, and the production-data boundary. | `docs/staging-setup.md`; `docs/operator-backlog.md` | 2026-07-24 | +| #057 | P2 | task | Complete staging soak and rollback rehearsal | After #056, run the documented soak and rollback against an exact candidate; retain latency/error/rollback evidence. Stop on unsafe data, identity mismatch, or an unowned rollback decision. | `docs/launch-operator-runbook.md`; `docs/capacity-review.md` | 2026-07-24 | +| #058 | P2 | task | Verify production content before any seed write | Against `Clinical KB Database`, verify registry, differentials, and medications surfaces are non-empty before writing. Seed only confirmed gaps idempotently with approved owner/project identity and confirmation flags. | `docs/launch-operator-runbook.md`; `docs/operator-backlog.md` | 2026-07-24 | | #005 | P3 | rec | `finalScore` saturates at clamp ceiling | Base + ~40 stacked boosts routinely exceed 1.0, so strong matches tie at 1.0 and order by an arbitrary `document_id` tiebreak. If ranking is ever revisited, break ties by the **pre-clamp** score rather than raising the `[0,1]` ceiling (downstream gates assume `[0,1]`). Ordering already sorts by the unbounded pre-clamp `rankScore` (`clinical-search.ts:1735,1927,1950-1955`), so the clamp confines only the reported confidence value, not result order. Not a defect on the current golden set; any change here is a protected RAG surface (canary required). | `docs/rag-hybrid-findings-and-todo.md` P1 item 4; `src/lib/clinical-search.ts:1735` | 2026-07-21 | | #007 | P3 | rec | `/tools` vs `/?mode=tools` parallel Tools entry points | `/tools` (standalone `ApplicationsLauncherPage`) has no inbound in-app link; the sidebar Tools item uses `/?mode=tools`. Decide the canonical entry point and wire nav consistently, or drop the standalone `/tools` page + `/applications` redirect. Currently allowlisted in `tests/route-reachability.test.ts`. | `src/app/tools/page.tsx`; `src/app/applications/route.ts` | 2026-07-21 | | #009 | P3 | rec | Confirm `/api/jobs` is intentionally server/ops-only | No client `fetch()` reaches `/api/jobs` (only tests import it). Confirm it is a deliberate ops/manual surface; if abandoned, remove it. | `src/app/api/jobs/route.ts` | 2026-07-21 | @@ -59,7 +116,7 @@ Durable, cross-session memory of everything still outstanding for this repo: ope | #027 | P3 | rec | External uptime monitor independent of GitHub/Railway | `live-domain-monitor.yml` runs on GitHub's cron, so it won't run in exactly the outage it should catch (Actions or the deploy itself down). Add an off-platform synthetic monitor (UptimeRobot / Better Stack / Checkly) hitting `/api/health` with a webhook alert. Provider setup, not code. | session 2026-07-22 webhook review | 2026-07-22 | | #028 | P3 | rec | Runtime error tracking (Sentry or similar) | No error tracking in the repo — production exceptions on `psychiatry.tools`, including how often `RAG_PROVIDER_MODE=auto` silently degrades to source-only, are invisible. Weigh adding `@sentry/nextjs` (dependency + DSN secret + instrumentation) vs cost; alert → chat/issue. Provider-backed; needs explicit sign-off before adding the dependency. | session 2026-07-22 webhook review | 2026-07-22 | | #029 | P2 | issue | 12 of 30 answer-quality cases return the fallback stub | run #61 --dump-answers: 12/30 quality cases emit the source_backed_review_fallback boilerplate with answer_sections: [], all grounded with 4-6 citations. Some still PASS targeting because the stub echoes query keywords (the contraindication/document_lookup matchers need only a keyword), so the targeting metric MASKS the problem for those intents. Superset of #018 — fix in the extractive composer, validate with the provider-backed answer eval. | run #61 dump artifact; session 2026-07-22 | 2026-07-22 | -| #030 | P3 | issue | Wide-tier alias lets one doc satisfy both comparison slots | In src/lib/eval-document-matching.ts, "Admission to Discharge for Mental Health Inpatients" appears in BOTH the AdmissionCommunityPts and Discharge alias lists, so a single document can satisfy both expectedFiles slots and make allHit true — a latent false-pass on admission-discharge cases. Not firing today (that doc is not in the failing top-5) but it would mask a real miss. Tighten the tables so one doc cannot fill both sides. | src/lib/eval-document-matching.ts:32-65; session 2026-07-22 | 2026-07-22 | +| #030 | P2 | issue | Wide-tier alias lets one doc satisfy both comparison slots | In src/lib/eval-document-matching.ts, "Admission to Discharge for Mental Health Inpatients" appears in BOTH the AdmissionCommunityPts and Discharge alias lists, so a single document can satisfy both expectedFiles slots and make allHit true — a latent false-pass on admission-discharge cases. Not firing today (that doc is not in the failing top-5) but it would mask a real miss. Tighten the tables so one doc cannot fill both sides. | src/lib/eval-document-matching.ts:32-65; session 2026-07-22 | 2026-07-22 | | #032 | P3 | rec | Governance ranking weighting: REFUTED, not debt | The source-governance audit (PR #1051) flagged three "gaps": `review_due` carries no ranking penalty, `unknownCurrentnessPenalty` ships at 0, and `selectBestSourceRecommendation` ignores governance metadata. **These are deliberate, measured decisions — do NOT implement them as written.** Blanket metadata boosts/penalties in selection ordering were measured on 2026-07-02 to regress the golden retrieval eval to 16/23 (doc-recall@5 1.0→0.76, mrr 0.75→0.64). Two corpus facts make it unsafe: scores saturate at the clamp so stacked boosts fully override lexical relevance, and the corpus is only partially metadata-enriched while `normalizeSourceMetadata` coerces unenriched docs to `unknown`/`unverified` — so "unknown" ≠ "bad" and blanket weighting swings ranking approx. 0.35 for reasons unrelated to relevance. Even governance-as-tiebreak buried correct unenriched docs (3 designs bisected). Next action: none — treat as a guardrail. If ever revisited, RC8 (source-strength as a _filter_) is the tracked path, gated on `eval:retrieval:quality` 36/36 plus a live canary pair. | PR #118; `docs/rag-behaviour/refuted-approaches.md`; PR #1051 items 4/5/6 | 2026-07-22 | | #033 | P3 | rec | Source governance metadata absent from the LLM prompt | `buildRagSourceBlock` omits `document_status`, `clinical_validation_status`, and `extraction_quality`, so the model cannot self-caveat during generation and governance is enforced only post-hoc. Generation-surface change: needs `eval:rag` plus `eval:quality --rag-only` (grounded-supported must not drop, citation-failure 0) and explicit approval. Carries the same "unknown ≠ bad" hazard as #032 — on a partially-enriched corpus the model would likely over-caveat correct sources, so design the prompt wording before spending an eval. | `src/lib/rag/rag-source-block.ts:126-198`; PR #1051 audit item 8 | 2026-07-22 | | #034 | P3 | issue | Answer cache can serve stale governance metadata | `cacheIndexingVersion` derives the version from `updated_at` / `indexed_at` / `index_generation_id`, so a metadata-only `document_status` flip that bumps none of those is invisible to the passive guard. **Already mitigated**: every known status-write path calls `invalidateRagCachesForOwner` or `invalidateRagCachesForDocumentMutation`. Residual risk only — a future write path that omits the invalidator would serve stale governance until TTL. Next action: add a regression test pinning the invalidator call on status-mutating routes (cheaper and safer than touching the protected cache key). | `src/lib/rag/rag-cache.ts:382-438`; PR #1051 audit item 10 | 2026-07-22 | diff --git a/docs/production-readiness-checklist.md b/docs/production-readiness-checklist.md index 1c1bd835b..4a0002dec 100644 --- a/docs/production-readiness-checklist.md +++ b/docs/production-readiness-checklist.md @@ -1,7 +1,8 @@ # Clinical KB Production Readiness Checklist (Executable Today) **Status: reusable release-candidate checklist, not an outstanding-task ledger.** Live/provider-gated -actions and their current status are tracked only in [`operator-backlog.md`](operator-backlog.md). +action detail is indexed in [`operator-backlog.md`](operator-backlog.md); canonical task status is +tracked only in [`outstanding-issues.md`](outstanding-issues.md). Unchecked boxes below are rerun per release candidate; they do not imply abandoned repository work. This is the runbook to make the app publishable in one focused pass.