Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 95 additions & 5 deletions .claude/skills/api-integration/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,13 +140,17 @@ export class {Name}WebSearchClient extends BaseWebSearchClient {
}

async searchItemsWeb(args = {}) {
const { query } = args;
// A3 (Exa Deep parallelization) — destructure additionalQueries so the
// augmentor pipeline can route it to the request body. Must spread into
// executeExaSearch options. Inert if undefined or feature flag is off.
const { query, additionalQueries } = args;
// CRITICAL: use site: with SUBDIRECTORY anchoring (not just domain)
const exaQuery = `site:example.gov/data "${query || ''}" relevant keywords`;
const results = await this.executeExaSearch(exaQuery, args.limit || 25, {
domain: 'category_name', // categorical string
includeDomains: this.domains, // array of allowed domains
summaryQuery: 'keywords guiding AI summary extraction from page content'
summaryQuery: 'keywords guiding AI summary extraction from page content',
...(additionalQueries !== undefined && { additionalQueries }) // A3 forwarding
});
return {
content: [{ type: 'text', text: JSON.stringify({
Expand All @@ -168,6 +172,7 @@ export class {Name}WebSearchClient extends BaseWebSearchClient {
- [ ] Each method name = native method + `Web` suffix (e.g., `searchItems` → `searchItemsWeb`)
- [ ] `site:` queries use **subdirectory** anchoring (e.g., `site:fred.stlouisfed.org/series/` not just `site:fred.stlouisfed.org`)
- [ ] `executeExaSearch` called with `{ domain, includeDomains, summaryQuery }`
- [ ] **A3 plumbing**: destructure `additionalQueries` from args, spread to `executeExaSearch` options (`...(additionalQueries !== undefined && { additionalQueries })`)
- [ ] Response maps to `{ search_type, total_results, results: [{ title, url, summary, snippet }] }`
- [ ] `node --check` passes

Expand Down Expand Up @@ -264,29 +269,58 @@ Append before the closing `};`:

### 3.2 Tool Schemas — `src/tools/toolDefinitions.js`

Add new array BEFORE `codeExecutionTools`:
**A3 trait declaration**: tools with Exa-fallback paths should declare traits so the augmentor pipeline auto-injects the `additionalQueries` parameter (with domain-tuned axis menu, distinctness telemetry, and A/B sampling hooks).

If your tool has Exa websearch fallback (i.e., its WebSearchClient method calls `executeExaSearch`):

```javascript
export const {name}Tools = [
const _raw{Name}Tools = [
{
name: "tool_name",
description: "...", // 1-2 sentences with PE/IB/M&A context
traits: ["exa-routable", "domain:<your-domain>"], // A3 augmentor activation
inputSchema: {
type: "object",
properties: { ... },
properties: { ... }, // do NOT include additionalQueries here — augmentor injects it
required: ["..."]
}
}
];

export const {name}Tools = applyAugmentors(_raw{Name}Tools, A3_AUGMENTORS);
```

Domain options (existing axis menus in `src/tools/augmentors/exaAdditionalQueries.js`):
- `case-law` (CourtListener cases/opinions/citations)
- `securities` (SEC filings/EDGAR)
- `federal-register` (regulations.gov, Federal Register)
- `clinical-trials` (ClinicalTrials.gov)
- `legislative` (Congress.gov)
- `patent` (USPTO/CPC classification)
- `epa-facility` / `epa-violation` (EPA ECHO)
- `fda-recall` / `fda-510k` / `fda-warning-letter` (FDA enforcement)
- `cpsc-recall` (CPSC product safety)
- `government-contracts` (SAM.gov/USAspending)
- `judges` (judicial-search-specific axes)
- `patent-appeals` (PTAB IPR/PGR/CBM)
- `general` (catch-all axis menu for `exa_web_search` and undefined domains)

If your domain doesn't match any existing entry, **add a new one** to `src/tools/augmentors/exaAdditionalQueries.js DOMAIN_DESCRIPTIONS`. Pattern:
1. Identify 4-6 axes the domain experts use (CFR sections, statute references, doctrines, document types, time windows, etc.)
2. Author a description with explicit GOOD/BAD worked example showing axis-shift vs. paraphrase
3. The schema description is byte-loaded into the model's context — keep it under ~900 chars

Add to `allTools` spread (after `...fredTools,`, before `...directFetchTools,`):
```javascript
...{name}Tools,
```

- [ ] Each schema has `name`, `description`, `inputSchema` with `type`, `properties`, `required`
- [ ] Description includes the tool's series ID examples or key param hints
- [ ] **A3**: `traits` declared if tool has Exa-fallback path
- [ ] **A3**: Domain matches existing entry in augmentor's DOMAIN_DESCRIPTIONS, OR new entry added
- [ ] **A3**: Tool array wrapped via `applyAugmentors(_rawXxx, A3_AUGMENTORS)`
- [ ] **A3**: Inline `additionalQueries` field NOT declared in schema (augmentor injects)
- [ ] Exported and added to `allTools`

### 3.3 Tool Implementations — `src/tools/toolImplementations.js`
Expand Down Expand Up @@ -344,6 +378,36 @@ Append tool reference block after the FRED block (before `## ENRICHMENT PROTOCOL
- [ ] Tool names match schema names exactly
- [ ] `MCP_FALLBACK_INSTRUCTIONS` block present at `_promptConstants.js:721`

#### 3.5a A3 Subagent Prompt Guidance (when subagent uses Exa-routable tools)

If any subagent registered for the new domain has access to A3-eligible tools (i.e., tools with `traits: ["exa-routable", ...]` declared), the subagent's prompt template must interpolate `EXA_ADDITIONAL_QUERIES_GUIDANCE` to teach the orchestrator the 2-3 axis-distinct variation pattern.

In `src/config/legalSubagents/agents/{subagent-name}.js`:

```javascript
import {
REPORT_SAVING_INSTRUCTIONS,
MCP_FALLBACK_INSTRUCTIONS,
DATABASE_URL_TEMPLATES,
EXA_ADDITIONAL_QUERIES_GUIDANCE // ← add this import
} from '../_promptConstants.js';

export const def = {
prompt: `... existing prompt ...
${EXA_ADDITIONAL_QUERIES_GUIDANCE} // ← interpolate, typically before MCP_FALLBACK_INSTRUCTIONS
${MCP_FALLBACK_INSTRUCTIONS}
${DATABASE_URL_TEMPLATES}
${REPORT_SAVING_INSTRUCTIONS}`,
...
};
```

The constant teaches the orchestrator: 2-3 variations targeting DISTINCT axes (doctrine/jurisdiction/CFR/seminal-case), NOT paraphrases. Per realistic test data: prompt-level reinforcement lifts adoption from ~44% to ~94.5% on a 134-tool list.

- [ ] Subagent imports `EXA_ADDITIONAL_QUERIES_GUIDANCE` from `_promptConstants.js`
- [ ] Subagent prompt template interpolates `${EXA_ADDITIONAL_QUERIES_GUIDANCE}` once
- [ ] Test added to `test/sdk/exa-prompt-guidance.test.js` `A3_RELEVANT_SUBAGENTS` array

### 3.6 Client Registry — `src/server/clientRegistry.js`

Two additions:
Expand Down Expand Up @@ -425,6 +489,32 @@ NODE_OPTIONS=--experimental-vm-modules npx jest test/sdk/domain-mcp-servers.test

- [ ] All tests pass with updated assertion count

### 4.2.1 A3 Augmentor Snapshot Test (REQUIRED if tool declares `traits`)

If your tool has `traits: ["exa-routable", "domain:X"]`, add the tool name + traits to `TOOL_TRAITS` in `test/sdk/exa-augmentor-snapshot.test.js`:

```javascript
// test/sdk/exa-augmentor-snapshot.test.js
const TOOL_TRAITS = {
// ...existing entries...
'your_new_tool': ['exa-routable', 'domain:your-domain'],
};
```

Update the `expect(found.length).toBe(N)` count by 1.

```bash
EXA_WEB_TOOLS=true NODE_OPTIONS=--experimental-vm-modules npx jest test/sdk/exa-augmentor-snapshot.test.js --no-coverage
```

The snapshot test verifies:
- Augmented schema is byte-equivalent across runs (idempotent)
- `additionalQueries` is the LAST property in `inputSchema.properties` (Anthropic prompt cache key invariance)
- `required` array order preserved (some tests use `.toEqual()` which is order-sensitive)
- `traits` field stripped from output (no MCP wire-format leak)

- [ ] Snapshot test passes for new tool (3 assertions: byte-equivalence + property order + required-array order)

### 4.3 Unit Test — `test/sdk/{name}-hybrid-client.test.js`

Create mock-driven test following `BaseHybridClient.test.js` pattern:
Expand Down
8 changes: 6 additions & 2 deletions .claude/skills/subagent-scaffold/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: subagent-scaffold
description: Generate a new Claude Agent SDK subagent across all 7 mandatory wiring files. Mirrors the equity-analyst canonical template — agent file in legalSubagents/agents/, index.js import + registration tuple, _promptConstants.js CAPABILITY constant, domainMcpServers.js SUBAGENT_DOMAIN_MAP entry, hookSSEBridge.js classifyAgent map, optional p0GateHook.js RESEARCH_AGENTS Set, catalogDisplay/agentClassifications.js + agentDisplayMeta.js. Triggers — subagent scaffold, new subagent, generate agent, /subagent-scaffold. Supports flags — --name <kebab-name>, --phase research|synthesis|qa, --domains <comma-list>, --keywords <comma-list>.
description: Generate a new Claude Agent SDK subagent across all 7 mandatory wiring files. Mirrors the equity-analyst canonical template — agent file in legalSubagents/agents/, index.js import + registration tuple, _promptConstants.js CAPABILITY constant, domainMcpServers.js SUBAGENT_DOMAIN_MAP entry, hookSSEBridge.js classifyAgent map, optional p0GateHook.js RESEARCH_AGENTS Set, catalogDisplay/agentClassifications.js + agentDisplayMeta.js. Triggers — subagent scaffold, new subagent, generate agent, /subagent-scaffold. Supports flags — --name <kebab-name>, --phase research|synthesis|qa, --domains <comma-list>, --keywords <comma-list>, --a3-eligible (auto-include EXA_ADDITIONAL_QUERIES_GUIDANCE for subagents that use Exa-routable tools).
---

# Subagent Scaffold — Generate a New Agent SDK Subagent
Expand All @@ -12,9 +12,12 @@ Creating a new subagent currently requires touching **7 files** in lockstep. Mis
## Workflow

```bash
/subagent-scaffold --name bond-analyst --phase research --domains sec,fred --keywords "bonds, yield curve, credit spreads"
/subagent-scaffold --name bond-analyst --phase research --domains sec,fred --keywords "bonds, yield curve, credit spreads" --a3-eligible
```

Flags:
- `--a3-eligible` (recommended for `--phase research`) — auto-imports `EXA_ADDITIONAL_QUERIES_GUIDANCE` from `_promptConstants.js` and interpolates it into the prompt template. Required when the subagent will call any tool with `traits: ["exa-routable", ...]` (i.e., A3-enabled tools — currently 20+ across SEC/case-law/Federal Register/clinical-trials/USPTO/EPA/FDA/CPSC/SAM.gov/PTAB).

Output: 1 new agent file + 6 patch suggestions, ready for operator review and apply.

## Pre-flight blockers
Expand Down Expand Up @@ -57,6 +60,7 @@ If `--phase research`, add agent name to `RESEARCH_AGENTS` Set (around L14). Thi
- `_parallelGroups.js` — wave membership for parallel dispatch
- `test/sdk/subagents.test.js` + `subagents-e2e.test.js` — assertion lists
- Agent-specific live test (e.g. `test/integration/<name>.test.js`)
- **`test/sdk/exa-prompt-guidance.test.js`** — when `--a3-eligible` is used, add agent name to the `A3_RELEVANT_SUBAGENTS` array. The 27 tests in this file assert that A3-eligible subagents have `EXA_ADDITIONAL_QUERIES_GUIDANCE` interpolated in their prompt; missing additions cause regressions visible at PR-review time.

## Output format

Expand Down
29 changes: 28 additions & 1 deletion .claude/skills/subagent-scaffold/references/template-anatomy.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,27 @@ import { featureFlags } from '../../featureFlags.js';
import {
REPORT_SAVING_INSTRUCTIONS,
MCP_FALLBACK_INSTRUCTIONS,
DATABASE_URL_TEMPLATES,
EXA_ADDITIONAL_QUERIES_GUIDANCE, // ← REQUIRED if subagent uses A3-eligible tools (Exa-routable)
<NAME>_CAPABILITY,
// optional: domain-specific instruction blocks
} from '../_promptConstants.js';
```

`buildScopedTools` is conditionally invoked when `featureFlags.SCOPED_MCP_SERVERS` is on — see Phase 3 of api-integration. With the flag off, agent inherits all tools.

### A3 (Exa Deep parallelization) prompt guidance

If the subagent has access to any tool with `traits: ["exa-routable", ...]` declared (most research-tier subagents do, via the catch-all `exa_web_search` plus per-domain tools like `search_sec_filings`, `search_cases`, `search_federal_register`, etc.), the prompt MUST interpolate `EXA_ADDITIONAL_QUERIES_GUIDANCE` to teach the orchestrator the 2-3 axis-distinct variation pattern.

Empirical data (from realistic LLM adoption test, n=55 trials):
- WITHOUT prompt-level guidance: 44% adoption (schema descriptions alone are diluted by 134-tool tool-list flooding)
- WITH guidance interpolation: 94.5% adoption

Token cost: ~600 input tokens per subagent invocation. Trivial vs. existing 40-50K-char subagent prompts.

**When NOT to add**: subagents that consume artifacts but don't author tool calls (memo synthesis, QA evaluators, citation validators). Adding the guidance to non-authoring agents is pure token bloat.

## 3. The `def` export

Three keys:
Expand All @@ -42,7 +56,20 @@ Three keys:
- When on, agent restricted to `STANDARD_TOOLS.withWriteAndWeb` + domain MCP tools

### `prompt`
Joined array of CAPABILITY constant + REPORT_SAVING_INSTRUCTIONS + MCP_FALLBACK_INSTRUCTIONS. Token budget on this side is paid every dispatch — keep CAPABILITY terse.
Template literal joining CAPABILITY constant + (if applicable) `EXA_ADDITIONAL_QUERIES_GUIDANCE` + `MCP_FALLBACK_INSTRUCTIONS` + `DATABASE_URL_TEMPLATES` + `REPORT_SAVING_INSTRUCTIONS`. Token budget on this side is paid every dispatch — keep CAPABILITY terse.

Canonical interpolation order (per equity-analyst.js):

```js
prompt: `... CAPABILITY-specific content ...

${EXA_ADDITIONAL_QUERIES_GUIDANCE} // ← if A3-eligible
${MCP_FALLBACK_INSTRUCTIONS}
${DATABASE_URL_TEMPLATES}
${REPORT_SAVING_INSTRUCTIONS}`,
```

**Order matters for recency bias.** Per realistic adoption testing, putting EXA_ADDITIONAL_QUERIES_GUIDANCE BEFORE the trailing instruction blocks (rather than at the very end) gives consistent 94.5% adoption. Placing it last (after REPORT_SAVING_INSTRUCTIONS) showed no advantage.

## What NOT to put in the agent file

Expand Down
26 changes: 21 additions & 5 deletions .claude/skills/subagent-scaffold/scripts/generate-agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
import {{
REPORT_SAVING_INSTRUCTIONS,
MCP_FALLBACK_INSTRUCTIONS,
DATABASE_URL_TEMPLATES,{a3_import_line}
{const_name},
}} from '../_promptConstants.js';

Expand Down Expand Up @@ -49,11 +50,10 @@
- TODO: domain-specific guardrails (e.g. "never offer investment advice", "always
flag when figures are unaudited")`,
...(featureFlags.SCOPED_MCP_SERVERS ? {{ tools: buildScopedTools('{name}') }} : {{}}),
prompt: [
REPORT_SAVING_INSTRUCTIONS,
MCP_FALLBACK_INSTRUCTIONS,
{const_name},
].join('\\n\\n'),
prompt: `... TODO domain-specific {const_name} content goes above this line ...
{a3_prompt_block}${{MCP_FALLBACK_INSTRUCTIONS}}
${{DATABASE_URL_TEMPLATES}}
${{REPORT_SAVING_INSTRUCTIONS}}`,
}};
'''

Expand All @@ -65,6 +65,13 @@ def main():
p.add_argument("--phase", required=True)
p.add_argument("--domains", default="")
p.add_argument("--keywords", default="")
p.add_argument(
"--a3-eligible",
action="store_true",
help="If set, includes EXA_ADDITIONAL_QUERIES_GUIDANCE import + interpolation. "
"Set this when the subagent will use any tool with `traits: ['exa-routable', ...]` "
"(most research-tier subagents — see docs/pending-updates/exa-a3-* docs).",
)
args = p.parse_args()

repo = Path(args.repo_root)
Expand All @@ -77,17 +84,26 @@ def main():
const_name = args.name.upper().replace("-", "_") + "_CAPABILITY"
keywords_quoted = ", ".join(f'"{k.strip()}"' for k in args.keywords.split(",") if k.strip())

# A3 (Exa Deep parallelization) — conditional injection
a3_import_line = "\n EXA_ADDITIONAL_QUERIES_GUIDANCE," if args.a3_eligible else ""
a3_prompt_block = "${EXA_ADDITIONAL_QUERIES_GUIDANCE}\n" if args.a3_eligible else ""

content = TEMPLATE.format(
name=args.name,
const_name=const_name,
keywords_quoted=keywords_quoted or '"<keyword>"',
a3_import_line=a3_import_line,
a3_prompt_block=a3_prompt_block,
)

target.write_text(content)

print("### Created files")
rel = target.relative_to(repo)
print(f"✓ {rel} (new — review template fields)")
if args.a3_eligible:
print(f" (A3 enabled: EXA_ADDITIONAL_QUERIES_GUIDANCE imported + interpolated)")
print(f" ⚠️ Add '{args.name}' to A3_RELEVANT_SUBAGENTS in test/sdk/exa-prompt-guidance.test.js")
print()


Expand Down
23 changes: 15 additions & 8 deletions .claude/skills/subagent-scaffold/scripts/scaffold.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,23 +10,29 @@ while [ "$cur" != "/" ]; do
done
[ -n "${REPO_ROOT:-}" ] || { echo "ERROR: cannot locate repo root" >&2; exit 2; }

NAME="" PHASE="" DOMAINS="" KEYWORDS=""
NAME="" PHASE="" DOMAINS="" KEYWORDS="" A3_ELIGIBLE=""

while [[ $# -gt 0 ]]; do
case "$1" in
--name) NAME="$2"; shift 2 ;;
--phase) PHASE="$2"; shift 2 ;;
--domains) DOMAINS="$2"; shift 2 ;;
--keywords) KEYWORDS="$2"; shift 2 ;;
--name) NAME="$2"; shift 2 ;;
--phase) PHASE="$2"; shift 2 ;;
--domains) DOMAINS="$2"; shift 2 ;;
--keywords) KEYWORDS="$2"; shift 2 ;;
--a3-eligible) A3_ELIGIBLE="--a3-eligible"; shift 1 ;;
-h|--help)
cat <<EOF
subagent-scaffold — generate a new Claude Agent SDK subagent

Usage:
scaffold.sh --name <kebab-name> --phase research|synthesis|qa --domains <csv> --keywords <csv>
scaffold.sh --name <kebab-name> --phase research|synthesis|qa --domains <csv> --keywords <csv> [--a3-eligible]

Flags:
--a3-eligible Include EXA_ADDITIONAL_QUERIES_GUIDANCE in prompt. Set this when
the subagent will use any tool with traits=['exa-routable', ...]
(most research-tier subagents — see api-integration skill).

Example:
scaffold.sh --name bond-analyst --phase research --domains sec,fred --keywords "bonds, yield curve"
scaffold.sh --name bond-analyst --phase research --domains sec,fred --keywords "bonds, yield curve" --a3-eligible
EOF
exit 0 ;;
*) echo "ERROR: unknown flag: $1" >&2; exit 2 ;;
Expand Down Expand Up @@ -58,7 +64,8 @@ python3 "$SCRIPT_DIR/generate-agent.py" \
--name "$NAME" \
--phase "$PHASE" \
--domains "$DOMAINS" \
--keywords "$KEYWORDS" || exit $?
--keywords "$KEYWORDS" \
$A3_ELIGIBLE || exit $?

# 2. Emit patch suggestions for the 6 registry files
python3 "$SCRIPT_DIR/wire-registries.py" \
Expand Down