Symptom
Direct CLI invocation completes in ~1.3s with full JSON output:
node bin/adcp.js storyboard run http://127.0.0.1:4900/mcp sales_social --json --allow-http \
--auth sk_harness_do_not_use_in_prod --webhook-receiver
Same args via spawnSync from the matrix harness (agent-skill-storyboard.ts:runGrader) hits the 120s timeout:
[harness] grader: status=null signal=SIGTERM error=spawnSync node ETIMEDOUT stdout=0b stderr=0b
Reproducible across multiple verify-mode runs against a clean workspace (no zombie agents, ports 4900/4950 confirmed clear via lsof).
Hypothesis
--webhook-receiver boots a local HTTP listener. The grader's exit path may not always close it cleanly, so on subsequent harness-driven runs the bind leaks. From outside the harness it works because each direct invocation gets a fresh address. The harness's spawnSync may be hanging on the listener's open socket.
Other possibilities:
- The receiver waits for an inbound webhook that the agent never sends (storyboard has a webhook step, agent's response is async, harness's grader timeout is hit before the webhook arrives).
- stdio buffering in spawnSync that the direct shell flushes differently.
Why this matters now
The matrix harness's verify step relies on runGrader returning a parseable JSON result. Empty stdout + ETIMEDOUT makes the harness misreport storyboard=fail regardless of actual storyboard outcome. Concretely: the matrix-blind-fixtures validation run reports storyboard=fail, traffic=façade when running via the harness; running the grader directly against the same agent produces overall_status: passing with a full traffic profile.
The traffic-check signal is independent and still works (which is the whole point of the redesign). But the harness's secondary signal — storyboard pass/fail — is unreliable until this is fixed.
Workaround
runGrader could fall back to spawning without --webhook-receiver when the storyboard track doesn't require webhook conformance. Or the receiver's port could be passed in explicitly so the harness can guarantee a clean port per run.
Evidence
Symptom
Direct CLI invocation completes in ~1.3s with full JSON output:
Same args via
spawnSyncfrom the matrix harness (agent-skill-storyboard.ts:runGrader) hits the 120s timeout:Reproducible across multiple verify-mode runs against a clean workspace (no zombie agents, ports
4900/4950confirmed clear vialsof).Hypothesis
--webhook-receiverboots a local HTTP listener. The grader's exit path may not always close it cleanly, so on subsequent harness-driven runs the bind leaks. From outside the harness it works because each direct invocation gets a fresh address. The harness'sspawnSyncmay be hanging on the listener's open socket.Other possibilities:
Why this matters now
The matrix harness's verify step relies on
runGraderreturning a parseable JSON result. Emptystdout+ ETIMEDOUT makes the harness misreportstoryboard=failregardless of actual storyboard outcome. Concretely: the matrix-blind-fixtures validation run reportsstoryboard=fail, traffic=façadewhen running via the harness; running the grader directly against the same agent producesoverall_status: passingwith a full traffic profile.The traffic-check signal is independent and still works (which is the whole point of the redesign). But the harness's secondary signal — storyboard pass/fail — is unreliable until this is fixed.
Workaround
runGradercould fall back to spawning without--webhook-receiverwhen the storyboard track doesn't require webhook conformance. Or the receiver's port could be passed in explicitly so the harness can guarantee a clean port per run.Evidence
.context/matrix-sales-social-verify-{3,4,5}.logfrom PR Fix custom agent 404 error by accepting agentConfig in request body #20 branch.agent-skill-storyboard.ts:runGradercall site (now logsgrader: status= signal= error=on every call to make this debuggable).