From 501b949e33ea1ac35abef7644a3cdc0aeb18cefd Mon Sep 17 00:00:00 2001
From: BigSimmo <87357024+BigSimmo@users.noreply.github.com>
Date: Mon, 13 Jul 2026 01:33:57 +0800
Subject: [PATCH] =?UTF-8?q?docs(rag):=20record=20A3=20Phase-2=20gate=20che?=
 =?UTF-8?q?ck=20=E2=80=94=20#12=E2=80=9314=20remain=20deferred?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Task A3 ("RAG optimization Phase 2 (architectural)") re-examined injection
findings #12–14 (source-authority tier, per-claim prose entailment, structural
role separation) against the standing rule: no re-index / retrieval change
without a real golden miss proving a gain, with the bar that eval:retrieval:quality
must improve (not merely not-regress).

Disposition: all three stay correctly deferred; the gate is not met. Documents
per-finding unblock conditions in docs/rag-injection-threat-model.md §8:
- No concrete golden-eval miss exists (golden set all-green; miss promotion
  privacy-blocked).
- #13/#14 don't touch retrieval ranking, so eval:retrieval:quality can't register
  a gain — they belong to the injection harness (mitigation #11), not this task.
- #12 risks the recall regression PR #118 fixed; authority stays a tie-breaker,
  not a sort key.

Docs-only. No code, retrieval, or answer-path change. eval:retrieval:quality is
provider-backed and was not run (speculative miss-hunting is owner-approved spend).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---
 docs/rag-injection-threat-model.md | 31 ++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/docs/rag-injection-threat-model.md b/docs/rag-injection-threat-model.md
index 6a1bfbeb8..2d48f94b5 100644
--- a/docs/rag-injection-threat-model.md
+++ b/docs/rag-injection-threat-model.md
@@ -170,4 +170,35 @@ Three of the deferred items had a low-risk, offline-verifiable "simplest safe" f
 
 ---
 
+## 8. Phase-2 architectural gate check (A3) — 2026-07-13
+
+Task **A3 "RAG optimization Phase 2 (architectural)"** re-examined findings #12–14 against the
+standing rule — **no re-index / retrieval change without a real golden miss proving a gain** — and
+the acceptance bar **`eval:retrieval:quality` must improve, not merely not-regress**. Outcome: **all
+three remain correctly deferred; the gate is not met. No code change was made.**
+
+- **No concrete golden-eval miss exists to explore against.** The retrieval golden set is all-green
+  on record (`document_recall@5=1.0`, `content_recall@5=1.0`, `top_k_hit_rate=1.0`; the reindex gate's
+  `content_mrr@10≈0.93`). Miss promotion from `rag_query_misses` → `rag_aliases` is still
+  privacy-blocked (§7 item 17 in `docs/rag-hybrid-findings-and-todo.md`), so no miss has been promoted
+  that a #12–14 change would fix.
+- **#13 (per-claim prose entailment) and #14 (structural role separation) do not touch retrieval
+  ranking.** `eval:retrieval:quality` (recall/MRR over clean clinical queries) therefore cannot
+  register a gain from them, so the task's verify bar is structurally unmeetable for them. They are
+  answer-path injection mitigations that belong to the seeded-chunk **injection** harness
+  (mitigation #11), which itself requires live model generation — out of scope for a retrieval-gated
+  task.
+- **#12 (source-authority tier) is the only retrieval-touching one, and on an all-green set the
+  realistic outcome is a recall _regression_.** Reintroducing content-authority as a primary sort key
+  is exactly what PR #118 removed. It stays deferred until a promoted miss demonstrates that authority
+  used strictly as a _tie-breaker/caveat trigger_ (not a sort key) lifts `eval:retrieval:quality`.
+
+`eval:retrieval:quality` is provider-backed (`scripts/eval-retrieval.ts` calls `requireServerEnv` +
+`requireOpenAIEnv` → live Supabase + OpenAI). It was **not** run for this disposition: running it
+speculatively to hunt a miss is itself owner-approved spend, and the standing rule forbids
+retrieval work without a miss already in hand. The correct completion of A3 is this recorded
+deferral plus the per-finding unblock conditions above.
+
+---
+
 _Originally prepared as an analysis artifact on branch `claude/safety-analysis`; every BLOCK/MISS verdict was verified against the code at the cited `file:line`. Section 6 records the mitigations subsequently implemented on `claude/llm-pipeline-review`; section 7 records the follow-ups implemented on `claude/deferred-rag-improvements`._