feat(streaming-hook): enrich wire events with cost, LLM timing, model, thinking, and sub-agent attribution

[manojp99]

Summary

Enhanced the streaming hook (hook_streaming.py) to enrich wire events with cost tracking, LLM timing visibility, model/provider attribution, thinking event streaming, and sub-agent identification. This directly addresses reviewer feedback requesting budget safety nets and per-step visibility into what work is being performed and how long it takes.

Key additions:

• UsageNotification now includes cost, llmDurationMs, model, provider, cacheReadTokens, cacheWriteTokens, sessionCostTotal, agentName
• Tool events (tool/started, tool/completed) now carry agentName for delegated sessions
• New handlers for thinking:delta and thinking:final events (reasoning now visible on the wire)
• New session-total cost aggregation via orchestrator:complete handler

Problem

A user code review identified two critical gaps:

1. Cost visibility: "Plumbing cost data, once we have that we can allow users to set budgets for spend, giving a safety net (we will need visibility on cost for all harnesses that support wiring it in)"

2. Per-step visibility: "Increase visibility on 'what' is being done on each Amplifier step/sub-step, include time per step ... the current duration appears to be only for the tool calls and not the llm calls (the valuable one)"

Root cause: The streaming hook was extracting only token counts from LLM responses and discarding cost, duration, model, provider, and cache data already available from the kernel. It did not subscribe to thinking events at all, and did not track session-total cost.

Approach

Executed under TDD in three phases with full commit history:

Phase 1 — Protocol Extension (6 tasks)

• Extended UsageNotification TypedDict: added cost: NotRequired[str] (Decimal precision), llmDurationMs, model, provider, cacheReadTokens, cacheWriteTokens, sessionCostTotal, agentName
• Extended tool notification TypedDicts: added agentName: NotRequired[str] to ToolStartedNotification and ToolCompletedNotification
• Regenerated JSON schemas via uv run python -m amplifier_agent_lib.protocol._gen --output-dir src/amplifier_agent_lib/protocol
• All schemas pass staleness gate (byte-exact regeneration)

Phase 2 — Hook Enrichment (6 tasks)

• New _parse_agent_name(session_id) helper: parses {parent}-{child}_{agent_name} session ID format
• New inline _sum_cost_usd(results): Decimal aggregator, replicated from foundation to keep hook free of unnecessary coupling
• Enhanced on_llm_response: extracts duration_ms, model, provider, cost_usd, cache_read_tokens, cache_write_tokens; attaches optional fields only when present
• New on_thinking_delta and on_thinking_final handlers: emit canonical thinking/delta and thinking/final wire events
• New on_orchestrator_complete handler: calls coordinator.collect_contributions("session.cost"), aggregates via _sum_cost_usd, emits session-total usage event
• Attached agentName to tool and usage events when session is delegated
• Extended CANONICAL_WIRE_EVENTS tuple (5 → 7 types)
• mount() registers 10 handlers (was 7)

Phase 3 — Verification (3 tasks)

• Full-turn integration test driving realistic sub-agent delegation (tool → result → thinking → enriched llm:response → orchestrator:complete)
• Quality gates: ruff format + check, pyright, full pytest suite
• End-to-end DTU validation against real Anthropic claude-opus-4-5

Testing & Validation

Unit & Integration Tests

• Hook tests: 27 pass (was 14 → +13 new tests)
• Protocol notification tests: 11 pass
• Protocol staleness tests: 35 pass
• Conformance fixture tests: 28 pass
• Suite summary: 552 passed, 1 skipped, 19 deselected (all non-integration)
• Code quality: ruff check clean, pyright clean

Environmental Note

11 test_conformance_parity.py tests fail due to missing tsx (TypeScript runner) on the dev box. These are pre-existing integration-environment gaps, marked @pytest.mark.integration, excluded from CI runs, and not regressions introduced by this work.

Digital Twin Universe (End-to-End) Validation

Built a DTU environment from the local working tree (mirrored to Gitea, installed via uv tool install from Gitea mirror) and ran real delegated turns against Anthropic's claude-opus-4-5.

Integration bug surfaced: The original on_orchestrator_complete handler called collect_contributions synchronously, but the kernel exposes it as an async coroutine. Fixed in commit 0d734d3 (the mock was also tightened to async def so future tests would catch this mismatch).

Real wire output from a delegated turn:

{"type": "usage", "cost": "0.029421", "llmDurationMs": 8788,
 "model": "claude-opus-4-5", "provider": "anthropic",
 "inputTokens": 4202, "outputTokens": 442,
 "cacheReadTokens": 4192, "cacheWriteTokens": 2596}

{"type": "tool/started", "name": "delegate",
 "args": {"agent": "explorer", ...}}

{"type": "usage", "cost": "0.0324015", "llmDurationMs": 9968,
 "agentName": "explorer",
 "sessionId": "0000000000000000-6563bf04ec894842_explorer", ...}

{"type": "tool/started", "name": "bash", "agentName": "explorer",
 "args": {"command": "curl -sL https://docs.python.org/3.13/whatsnew/3.13.html"}}

{"type": "usage", "sessionCostTotal": "0.092882", ...}

Every requested field reaches the wire in realistic multi-step delegated turns.

Wire Taxonomy

Additive only — no breaking changes:

• No new canonical wire event types (thinking events already in CANONICAL_DISPLAY_EVENTS)
• All new fields are NotRequired (optional)
• Only type change: UsageNotification.cost (NotRequired[float] → NotRequired[str]); this slot existed but was never populated, so no live consumer was affected
• All schemas regenerated and pass staleness gate

Files Changed

File	Changes
src/amplifier_agent_lib/bundle/hook_streaming.py	Main hook enhancements (~140 lines)
src/amplifier_agent_lib/protocol/notifications.py	TypedDict additions (~15 lines)
src/amplifier_agent_lib/protocol/schemas/UsageNotification.schema.json	Regenerated
src/amplifier_agent_lib/protocol/schemas/ToolStartedNotification.schema.json	Regenerated
src/amplifier_agent_lib/protocol/schemas/ToolCompletedNotification.schema.json	Regenerated
tests/test_bundle_hook_streaming.py	13 new + 1 renamed test (~370 lines)
tests/test_protocol_notifications.py	2 new TDD tests + 1 type-contract update
docs/plans/2026-06-09-streaming-hook-enrichment-*.md	4 plan documents (index + 3 phases)

Related Issues / PRs

Closes: (reviewer feedback from recent PR review)

Generated with Amplifier