v6.11.0: Dynamic KG entity extraction via fact-validator entities.json sidecar

.claude/skills/api-integration/SKILL.md

@@ -596,10 +596,19 @@ git push origin --delete feature/{name}-api-client
 
 After merge to `main`, these updates MUST be completed:
 
-### 6.1 Frontend Catalog Display
+### 6.1 Frontend Catalog Display + Prompt-Enhancer Dynamic Catalog
 
-**`src/config/catalogDisplay/domainDisplayMeta.js`**:
-- [ ] Add 25-30 word description for the new domain (matching existing format)
+**`src/config/catalogDisplay/domainDisplayMeta.js`** (REQUIRED — dual-consumer):
+- [ ] Add a 25-50 word capability description for the new domain (matching existing format)
+- [ ] Description should cite: what data the domain provides, what subagents consume it, any feature-flag gating
+
+**Why this is REQUIRED (not optional)**: `DOMAIN_DISPLAY_META` is consumed by TWO surfaces:
+1. The frontend `/api/catalog` endpoint (current consumer — domain card rendering)
+2. **The prompt-enhancer dynamic catalog** (`src/config/promptEnhancerCatalog.js`, feature flag `PROMPT_ENHANCER_DYNAMIC_CATALOG` #43) — injects the domain table into Haiku's enhancer system prompt so it knows what data the orchestrator's specialists have access to
+
+Without an entry: the new domain appears in the catalog table with `(no description registered in DOMAIN_DISPLAY_META)` placeholder text. Haiku then doesn't know what the domain provides, so the routing directive's specialist-deliverable mapping is incomplete. `/feature-compliance-scaffold` D11-catalog dimension will flag this as WARNING pre-merge.
+
+**Canonical pattern**: see existing `sec`, `fred`, `equities` entries — capability-rich, cites specific tool capabilities, identifies any auth or gating.
 
 **`test/react-frontend/app.js`** (`PC_DOMAIN_SUIT` mapping):
 - [ ] Map new domain to a suit: `'gov'`, `'court'`, `'sec'`, `'patent'`, `'trade'`, `'financial'`, `'exa'`, `'web'`

.claude/skills/feature-compliance-scaffold/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: feature-compliance-scaffold
-description: Defensive pre-PR-merge audit that validates a new feature against 10 cross-cutting dimensions — auditability, OTel traceability, EU AI Act + GDPR coverage, RBAC, embeddings, provenance, dual-path schema evolution, WORM storage consistency, metrics + alerts, and hook persistence. Static-analysis only; never mutates code or DB. Use after building a feature via api-integration or code-execution-models, before opening the PR. Triggers — compliance check, feature compliance, pre-merge audit, cross-cutting check, /feature-compliance-scaffold. Supports flags — --git-range <ref>, --feature-spec <path>, --feature-type api|subagent|endpoint|model, --name <name>, --dimensions D1,D2,..., --format json|markdown.
+description: Defensive pre-PR-merge audit that validates a new feature against 11 cross-cutting dimensions — auditability, OTel traceability, EU AI Act + GDPR coverage, RBAC, embeddings, provenance, dual-path schema evolution, WORM storage consistency, metrics + alerts, hook persistence, and prompt-enhancer catalog correctness. Static-analysis only; never mutates code or DB. Use after building a feature via api-integration or code-execution-models, before opening the PR. Triggers — compliance check, feature compliance, pre-merge audit, cross-cutting check, /feature-compliance-scaffold. Supports flags — --git-range <ref>, --feature-spec <path>, --feature-type api|subagent|endpoint|model, --name <name>, --dimensions D1,D2,..., --format json|markdown.
 ---
 
 # Feature Compliance Scaffold — Defensive Pre-PR Gate
@@ -24,7 +24,7 @@ description: Defensive pre-PR-merge audit that validates a new feature against 1
 
 Exit codes: `0` = all PASSED (or N/A), `1` = at least one WARNING, `2` = at least one FAILED. Pre-PR gates should treat exit ≥1 as a blocker pending operator review.
 
-## What it validates (10 dimensions)
+## What it validates (11 dimensions)
 
 | ID | Dimension | What it checks |
 |----|-----------|----------------|
@@ -38,6 +38,7 @@ Exit codes: `0` = all PASSED (or N/A), `1` = at least one WARNING, `2` = at leas
 | D8 | Storage | Per-client GCS bucket pattern (`super-legal-worm-{client-id}-us-east1`); retention class consistent with deployment dominant class |
 | D9 | Metrics | New metric in `sdkMetrics.js` (cardinality budget); new alert in `alerts.yml` with receiver routing |
 | D10 | Hooks | New `event_type` registered in `hookDBBridge.js`; high-volume types added to `dbFrontendRouter.js` analytics-exclusion lists; synthetic event_types tracked separately |
+| D11 | Catalog | New subagent has `AGENT_DISPLAY_META[name].expertise` ≥100 chars + `MUST BE USED when:` block (when applicable); new MCP domain has `DOMAIN_DISPLAY_META[name]` entry. Required for prompt-enhancer dynamic catalog (`PROMPT_ENHANCER_DYNAMIC_CATALOG`, flag #43) and frontend `/api/catalog`. Severity: WARNING (catalog degrades gracefully but Haiku's routing fidelity suffers) |
 
 Full dimension catalog with examples + remediation: `references/dimensions-catalog.md`.
 

.claude/skills/feature-compliance-scaffold/scripts/dimensions/D11-catalog.py

@@ -0,0 +1,294 @@
+#!/usr/bin/env python3
+"""D11 — Catalog correctness for prompt-enhancer dynamic catalog.
+
+Validates that any new subagent or new MCP domain shipped in this feature has
+the metadata required by the dynamic prompt-enhancer catalog
+(src/config/promptEnhancerCatalog.js, feature flag #43
+PROMPT_ENHANCER_DYNAMIC_CATALOG) populated correctly. Without these fields the
+catalog still works (defensive degradation falls back to agent.description /
+shows "(no description registered)" placeholder), but every enhancement call
+emits a warning to Cloud Logging and Haiku's routing fidelity degrades.
+
+Checks (severity WARNING — non-blocking but visible pre-merge):
+
+  D11.1  For each new agent_type, AGENT_DISPLAY_META[name].expertise is
+         present and ≥100 chars (capability-rich paragraph)
+
+  D11.2  For each new agent_type, agent description includes a
+         "MUST BE USED when user mentions:" block (so trigger extraction
+         works — required for the orchestrator's routing logic)
+
+  D11.3  For each new domain (detected via DOMAIN_GROUPS addition in
+         domainMcpServers.js), DOMAIN_DISPLAY_META[domain] is present
+
+See: src/config/promptEnhancerCatalog.js (the consumer)
+See: plans/floating-cooking-flute.md (planning doc)
+"""
+import re
+import sys
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).parent))
+from _shared import parse_args, emit_findings, is_noqa
+
+
+def find_repo_root(start: Path) -> Path:
+    cur = start.resolve()
+    for _ in range(10):
+        if (cur / "super-legal-mcp-refactored").is_dir():
+            return cur
+        if cur.parent == cur:
+            break
+        cur = cur.parent
+    raise SystemExit("D11: cannot locate repo root")
+
+
+def get_agent_display_meta_keys(repo_root: Path) -> set:
+    """Extract the set of agent names that have entries in AGENT_DISPLAY_META."""
+    f = (
+        repo_root
+        / "super-legal-mcp-refactored"
+        / "src"
+        / "config"
+        / "catalogDisplay"
+        / "agentDisplayMeta.js"
+    )
+    if not f.exists():
+        return set()
+    text = f.read_text()
+    # Match keys like 'equity-analyst': {
+    return set(re.findall(r"^\s*'([a-z][a-z0-9-]+)':\s*\{", text, re.MULTILINE))
+
+
+def get_agent_display_meta_expertise_lengths(repo_root: Path) -> dict:
+    """For each AGENT_DISPLAY_META key, extract expertise field length."""
+    f = (
+        repo_root
+        / "super-legal-mcp-refactored"
+        / "src"
+        / "config"
+        / "catalogDisplay"
+        / "agentDisplayMeta.js"
+    )
+    if not f.exists():
+        return {}
+    text = f.read_text()
+    lengths = {}
+    # Match each agent block; extract expertise: 'text' (single-line string)
+    # Pattern: 'name': { ... expertise: '...content...' ... dealContext: ... }
+    for m in re.finditer(
+        r"^\s*'([a-z][a-z0-9-]+)':\s*\{[\s\S]*?expertise:\s*'([^']*(?:\\'[^']*)*)'",
+        text,
+        re.MULTILINE,
+    ):
+        name = m.group(1)
+        # Unescape single quotes for accurate length
+        expertise = m.group(2).replace("\\'", "'")
+        lengths[name] = len(expertise)
+    return lengths
+
+
+def get_agent_description(repo_root: Path, agent_name: str) -> str:
+    """Read the description field from src/config/legalSubagents/agents/<name>.js."""
+    f = (
+        repo_root
+        / "super-legal-mcp-refactored"
+        / "src"
+        / "config"
+        / "legalSubagents"
+        / "agents"
+        / f"{agent_name}.js"
+    )
+    if not f.exists():
+        return ""
+    text = f.read_text()
+    # Match description: `...` (template literal — may span multiple lines)
+    m = re.search(r"description:\s*`([\s\S]*?)`", text)
+    return m.group(1) if m else ""
+
+
+def get_domain_display_meta_keys(repo_root: Path) -> set:
+    """Extract the set of domain names that have entries in DOMAIN_DISPLAY_META."""
+    f = (
+        repo_root
+        / "super-legal-mcp-refactored"
+        / "src"
+        / "config"
+        / "catalogDisplay"
+        / "domainDisplayMeta.js"
+    )
+    if not f.exists():
+        return set()
+    text = f.read_text()
+    return set(re.findall(r"^\s*'([a-z][a-z0-9-]+)':\s*'", text, re.MULTILINE))
+
+
+def detect_new_domains_from_diff(symbols: dict) -> list:
+    """Detect new MCP domains added by this feature.
+
+    The extract-feature-symbols.py script identifies new domains by scanning
+    git-diff additions to DOMAIN_GROUPS in domainMcpServers.js. The key may
+    appear under symbols.domains, symbols.mcp_domains, or be absent. We
+    tolerate any of these.
+    """
+    s = symbols.get("symbols") or {}
+    for key in ("domains", "mcp_domains", "domain_groups"):
+        v = s.get(key)
+        if isinstance(v, list) and v:
+            return v
+    return []
+
+
+def main():
+    truth, symbols, args = parse_args(__doc__)
+    suppressed, reason = is_noqa(symbols, "D11")
+    if suppressed:
+        emit_findings(
+            "D11",
+            [
+                {
+                    "dimension": "D11",
+                    "status": "N/A",
+                    "check": "suppressed",
+                    "message": f"D11 suppressed via noqa: {reason}",
+                    "remediation": "",
+                }
+            ],
+        )
+        return
+
+    repo_root = find_repo_root(Path(args.symbols).parent)
+    s = symbols.get("symbols") or {}
+    findings = []
+
+    new_agents = s.get("agent_types") or []
+    new_domains = detect_new_domains_from_diff(symbols)
+
+    if not new_agents and not new_domains:
+        # D11 doesn't apply — feature doesn't add agents or domains
+        emit_findings("D11", [])
+        return
+
+    # ── D11.1 + D11.2 — per-agent checks ──────────────────────────────
+    display_keys = get_agent_display_meta_keys(repo_root)
+    expertise_lengths = get_agent_display_meta_expertise_lengths(repo_root)
+
+    for agent in new_agents:
+        # D11.1 — AGENT_DISPLAY_META.expertise present + ≥100 chars
+        if agent not in display_keys:
+            findings.append(
+                {
+                    "dimension": "D11",
+                    "status": "WARNING",
+                    "check": "D11.1 agent_display_meta_missing",
+                    "message": (
+                        f"agent '{agent}' is registered but has no AGENT_DISPLAY_META entry. "
+                        f"The dynamic prompt-enhancer catalog (flag #43) will fall back to "
+                        f"the agent's raw description and emit a `catalog_agent_meta_missing` "
+                        f"warning on every enhancement call."
+                    ),
+                    "remediation": (
+                        f"Add an entry in src/config/catalogDisplay/agentDisplayMeta.js: "
+                        f"'{agent}': {{ role: '...', expertise: '<≥100 char paragraph>', "
+                        f"dealContext: '...' }}. See equity-analyst entry for canonical pattern."
+                    ),
+                }
+            )
+        else:
+            length = expertise_lengths.get(agent, 0)
+            if length < 100:
+                findings.append(
+                    {
+                        "dimension": "D11",
+                        "status": "WARNING",
+                        "check": "D11.1 agent_expertise_too_short",
+                        "message": (
+                            f"agent '{agent}' AGENT_DISPLAY_META.expertise is {length} chars "
+                            f"(threshold: ≥100). Short descriptions reduce Haiku's routing "
+                            f"fidelity in the dynamic catalog."
+                        ),
+                        "remediation": (
+                            f"Expand the expertise paragraph to cite: (1) what MCP domains "
+                            f"the agent uses, (2) what specialist deliverable it produces, "
+                            f"(3) any code-execution models it powers, (4) any feature-flag gating."
+                        ),
+                    }
+                )
+
+        # D11.2 — agent description has MUST BE USED block
+        description = get_agent_description(repo_root, agent)
+        if description:
+            has_proactively = bool(re.search(r"Use PROACTIVELY for:", description, re.IGNORECASE))
+            has_must_be_used = bool(
+                re.search(r"MUST BE USED when user mentions:", description)
+            )
+            # Only flag when "Use PROACTIVELY for:" exists (signals it's a research agent
+            # meant to have triggers). Synthesis/QA agents legitimately omit both.
+            if has_proactively and not has_must_be_used:
+                findings.append(
+                    {
+                        "dimension": "D11",
+                        "status": "WARNING",
+                        "check": "D11.2 agent_triggers_missing",
+                        "message": (
+                            f"agent '{agent}' description has 'Use PROACTIVELY for:' but lacks "
+                            f"'MUST BE USED when user mentions:' block. The dynamic catalog "
+                            f"will render an empty trigger list and emit a "
+                            f"`catalog_agent_triggers_missing` warning each call. The "
+                            f"orchestrator's keyword-routing logic depends on this block."
+                        ),
+                        "remediation": (
+                            f"Add 'MUST BE USED when user mentions: <comma-separated keywords>' "
+                            f"at the end of the agent's description field in "
+                            f"src/config/legalSubagents/agents/{agent}.js. See "
+                            f"equity-analyst.js or financial-analyst.js for examples."
+                        ),
+                    }
+                )
+
+    # ── D11.3 — per-domain checks ─────────────────────────────────────
+    if new_domains:
+        domain_keys = get_domain_display_meta_keys(repo_root)
+        for domain in new_domains:
+            if domain not in domain_keys:
+                findings.append(
+                    {
+                        "dimension": "D11",
+                        "status": "WARNING",
+                        "check": "D11.3 domain_display_meta_missing",
+                        "message": (
+                            f"domain '{domain}' added to DOMAIN_GROUPS but missing from "
+                            f"DOMAIN_DISPLAY_META. The dynamic catalog domain table will "
+                            f"render '(no description registered)' placeholder text. "
+                            f"Frontend /api/catalog will also lack the description."
+                        ),
+                        "remediation": (
+                            f"Add an entry in src/config/catalogDisplay/domainDisplayMeta.js: "
+                            f"'{domain}': '<25-50 word capability description citing data "
+                            f"source + auth requirements>'. See 'sec' or 'equities' for pattern."
+                        ),
+                    }
+                )
+
+    # If we triggered (had agents or domains to check) but nothing failed,
+    # emit an explicit PASSED finding for visibility.
+    if not findings:
+        findings.append(
+            {
+                "dimension": "D11",
+                "status": "PASSED",
+                "check": "catalog metadata complete",
+                "message": (
+                    f"All {len(new_agents)} new agent(s) and {len(new_domains)} new domain(s) "
+                    f"have complete catalog metadata (AGENT_DISPLAY_META.expertise ≥100 chars + "
+                    f"MUST BE USED triggers where applicable; DOMAIN_DISPLAY_META entries present)."
+                ),
+                "remediation": "",
+            }
+        )
+
+    emit_findings("D11", findings)
+
+
+if __name__ == "__main__":
+    main()