Citation Websearch Verification Layer (Phase G5) — Zero-Hallucination Footnote Certification

[Number531]

Summary

Introduce a new pipeline phase G5 (citation-websearch-verification) that independently websearches every VERIFIED and INFERRED footnote from consolidated-footnotes.md to confirm citations actually exist, producing a zero-hallucination certification report before final synthesis.

Spec: docs/pending-updates/validation-citations-layer.md

Problem

The current G4 citation-validator validates footnotes structurally (renumbering, tag presence, pincites, Bluebook compliance) but never independently confirms that citations exist in the real world. A [VERIFIED: CourtListener Opinion IDs 4593878] tag means the research subagent claimed it verified during research — but no post-hoc double-check occurs.

In a legal memorandum context, false citations constitute professional malpractice risk. The pipeline needs an independent verification layer.

Solution

A new citation-websearch-verifier subagent that:

1. Reads consolidated-footnotes.md from G4
2. Classifies each footnote by tag type:
   • [VERIFIED:] / [INFERRED:] → websearch verification required
   • [ASSUMED:] / [METHODOLOGY:] → skip (inherently non-verifiable)
3. Verifies every verifiable footnote via batched websearch:
   • Statutory citations (U.S.C., C.F.R.): auto-confirmed by structural validity
   • URL-bearing: WebFetch the URL → HTTP 200 confirms
   • Text-only: WebSearch with citation text query → matching result confirms
4. Produces qa-outputs/citation-verification-certificate.md (modeled on QA certificate)
5. Writes citation-websearch-verifier-state.json for compaction recovery
6. Acts as HARD GATE — PASS (≥95%) / PASS_WITH_EXCEPTIONS (≥85%) / HARD_FAIL (<85%)

Pipeline Position

G4: citation-validation → citation-validator (HARD GATE)
G5: citation-websearch-verification → citation-websearch-verifier (HARD GATE)  ← NEW
A1: final-synthesis → memo-final-synthesis

Feature Flag

CITATION_WEBSEARCH_VERIFICATION=false (default off, zero behavior change when disabled)

Implementation Scope

Create

• src/config/legalSubagents/agents/citation-websearch-verifier.js — New agent definition (~650 lines)

Modify

• src/config/featureFlags.js — Add CITATION_WEBSEARCH_VERIFICATION flag
• src/config/legalSubagents/index.js — Import + register new agent
• src/utils/hookSSEBridge.js — Add classifyAgent + classifyDocument entries
• prompts/memorandum-orchestrator.md — Add G5 phase to sequence, gate logic, state schema
• src/config/domainMcpServers.js — Add empty domain mapping

Key Design Decisions

Decision	Choice	Rationale
Verification scope	Full (every footnote)	Legal malpractice risk; no sampling
Verification tools	Standard WebSearch + WebFetch	Cross-domain coverage; no MCP dependency
Batching strategy	5 priority tiers	Token budget: ~41 turns for 400 footnotes
Gate thresholds	95% PASS / 85% EXCEPTIONS	Accommodates paywalled or recently-published sources
Feature flag default	false	Zero behavior change; opt-in activation
Max gate attempts	2	Then force PASS_WITH_EXCEPTIONS

Verification Budget (400-footnote memo)

Batch	Items	Method	Turns
Statutory auto-confirm	~60	Structural	0
URL VERIFIED	~120	WebFetch (10/turn)	12
URL INFERRED	~30	WebFetch (10/turn)	3
Text VERIFIED	~100	WebSearch (5/turn)	20
Text INFERRED	~30	WebSearch (5/turn)	6
Total	~340 verifiable		~41 turns

Expected duration: 10–20 minutes at Sonnet 4.6

Certificate Output

The verification certificate (qa-outputs/citation-verification-certificate.md) includes:

• Certification status (PASS / PASS_WITH_EXCEPTIONS / HARD_FAIL)
• Confirmation rate with breakdown by category
• Per-section summary table
• Detailed table of every UNCONFIRMED and ERROR citation
• Gate determination with zero-tolerance check (critical citations)
• Certification statement with limitations

Labels

enhancement, pipeline, quality-assurance

---

🤖 Generated with Claude Code

[Number531]

Duplicate of #16 (Citation Websearch Verification Layer — G5 implementation tracking). Closing in favor of #16.