feat(testing): createComplyController seller scaffold (#701)#708
Merged
Conversation
…_controller (#701) Add `createComplyController` to `@adcp/client/testing` — a domain-grouped (seed / force / simulate) seller-side scaffold for the comply_test_controller tool. Also extends TestControllerStore with seed methods and exports SeedFixtureCache / createSeedFixtureCache / SEED_SCENARIOS from the existing flat-store surface so both APIs support the full 12-scenario protocol. Hardened against common misuse: sandbox gate requires strict === true; seed cache is capped at 1000 net-new keys; fixture keys __proto__ / constructor / prototype are rejected; toolDefinition is frozen with a shallow-copied inputSchema; list_scenarios bypasses the gate so capability probes always succeed; register() warns when no gate is configured and no opt-out env flag is set. Closes #701. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
bokelley
added a commit
that referenced
this pull request
Apr 21, 2026
…ontroller skill refs Matrix perf: - New `--shared-node-modules <path>` flag on `agent-skill-storyboard.ts` symlinks the scratch workspace's `node_modules` to a cached template instead of running `npm install` per pair. The matrix driver prepares the template once in `.cache/harness-template/` (~30s) and every subsequent pair skips install (~20s saved per pair). - `run-skill-matrix.ts` parallel default auto-picks `min(4, cpus/2)` — was `1`. 14-pair full matrix drops from ~34 min serial to ~9 min on a typical laptop. - Fast-fail up-front if the `claude` CLI isn't on PATH (saves every pair from hitting the same wall). - Summary line now prints wall, cpu, and speedup so regressions are obvious. - `--no-shared-install` escape hatch for debugging template drift. Skills (createComplyController is the new scaffold from #708): - build-seller-agent/SKILL.md § Compliance Testing now leads with `createComplyController` (adapter-based, handles dispatch + idempotency + sandbox gating). `registerTestController` demoted to the "low-level alternative" footnote. - build-creative-agent/SKILL.md gains § Deterministic Testing covering seed.creative + force.creative_status adapters, required for `creative_generative/seller` and `deterministic_testing` storyboards. - docs/guides/VALIDATE-YOUR-AGENT.md gains § Deterministic testing section pointing at `createComplyController` with the canonical adapter example. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
bokelley
added a commit
that referenced
this pull request
Apr 21, 2026
…l 8 skills (#709) * docs(validation): VALIDATE-YOUR-AGENT index + skill-matrix harness New operator-facing docs/guides/VALIDATE-YOUR-AGENT.md: five-command checklist plus command-by-command references for storyboard runner, adcp fuzz (T1/T2/T3), adcp grade request-signing, multi-instance, webhook conformance, schema-driven validation hooks, custom --invariants, and SubstitutionEncoder/Observer. BUILD-AN-AGENT.md and root CLAUDE.md link to it so builders find the full surface without reading source. New scripts/manual-testing/skill-matrix.json + run-skill-matrix.ts driver, exposed as `npm run compliance:skill-matrix`. Fans out the existing agent-skill-storyboard harness across 14 skill × storyboard pairs (seller, brand-rights, creative, generative-seller, governance, retail-media, si, signals) with --filter, --parallel, --stop-on-first-fail. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(skills): unified § Validate Locally section across all 8 build-*-agent Replaces skill-specific § Validation sections (which varied in depth and didn't mention fuzz, webhook-receiver, multi-instance, substitution, --invariants, or the skill-matrix harness) with a consistent § Validate Locally block per skill. Each block points at docs/guides/VALIDATE-YOUR-AGENT.md for the full picture, then covers specialism-specific commands: canonical storyboard IDs, cross-cutting bundles (security_baseline, idempotency, schema_validation, error_compliance), rejection-surface fuzz with the right --tools list, and a decoder for the most common per-specialism failure modes. Skill-specific extras: - retail-media: SubstitutionEncoder.encode_for_url_context wiring for catalog-driven macro URLs; mentions substitution_encoding_violation / substitution_binding_missing grader failures. - seller: specialism bundle enumeration (sales_guaranteed, broadcast_tv, streaming_tv, social, etc.); signed-requests grader pointer; multi- instance pointer. - governance: --auto-seed fuzz (Tier 3 update-tool coverage); 3.0 GA authority_level → human_review_required reminder. Every skill also points at `npm run compliance:skill-matrix -- --filter <name>` as the port-less fallback — invokes the dogfood harness in an isolated workspace when local binding isn't available. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: prettier + changeset for PR #709 * feat(harness): shared node_modules + parallel default + createComplyController skill refs Matrix perf: - New `--shared-node-modules <path>` flag on `agent-skill-storyboard.ts` symlinks the scratch workspace's `node_modules` to a cached template instead of running `npm install` per pair. The matrix driver prepares the template once in `.cache/harness-template/` (~30s) and every subsequent pair skips install (~20s saved per pair). - `run-skill-matrix.ts` parallel default auto-picks `min(4, cpus/2)` — was `1`. 14-pair full matrix drops from ~34 min serial to ~9 min on a typical laptop. - Fast-fail up-front if the `claude` CLI isn't on PATH (saves every pair from hitting the same wall). - Summary line now prints wall, cpu, and speedup so regressions are obvious. - `--no-shared-install` escape hatch for debugging template drift. Skills (createComplyController is the new scaffold from #708): - build-seller-agent/SKILL.md § Compliance Testing now leads with `createComplyController` (adapter-based, handles dispatch + idempotency + sandbox gating). `registerTestController` demoted to the "low-level alternative" footnote. - build-creative-agent/SKILL.md gains § Deterministic Testing covering seed.creative + force.creative_status adapters, required for `creative_generative/seller` and `deterministic_testing` storyboards. - docs/guides/VALIDATE-YOUR-AGENT.md gains § Deterministic testing section pointing at `createComplyController` with the canonical adapter example. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: regen agent docs (creative_generative flow) * chore: regen docs/llms.txt after upstream sync (57 storyboards) --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
createComplyControllerto@adcp/client/testing— a domain-grouped (seed/force/simulate) seller-side scaffold for AdCP'scomply_test_controllertool. Returns{ toolDefinition, handle, handleRaw, register }so sellers wire the tool with a singlecontroller.register(server)call.TestControllerStorewith five seed methods (seedProduct,seedPricingOption,seedCreative,seedPlan,seedMediaBuy). Both the existingregisterTestControllerflat-store surface and the new grouped façade now cover the full 12-scenario protocol.=== truegate, capped seed cache, prototype-key rejection, frozen tool definition,list_scenariosbypasses the gate so capability probes always succeed, andregister()warns once per controller when no gate and no env opt-out is configured.Closes #701.
What the helper owns (so the seller doesn't)
scenarioto the correct adapter; unregistered adapters returnUNKNOWN_SCENARIO.INVALID_PARAMS.previous_state: "existing"; divergent fixture =INVALID_PARAMSwithout touching the adapter.TestControllerError(code, msg, currentState?)or return the success shape; the helper never leaks unexpected errors.sandboxGatewith strict=== truesemantics, fails closed on throws, bypassed forlist_scenarios.content+structuredContent+isErrorvia the existingtoMcpResponse.What stays adapter-side (deliberate)
CREATIVE_TRANSITIONStable the adapter consults.Scope trimmed from #701 (follow-up issues filed)
registerTool+outputSchemamigration — repo-wide story, filed as tech-debt: migrate MCP tool registrations from deprecatedtool()toregisterTool()#705.ctxresolver — needs design input from a real multi-tenant case, filed as design: per-tenant resolver / ctx enrichment forcreateComplyController#706.ListScenariosSuccess['scenarios']enum in the request schema doesn't includeseed_*values — filed separately onadcontextprotocol/adcp.Test plan
test/comply-controller.test.jscovering dispatch, list_scenarios, seed idempotency (fresh / equivalent / divergent / cross-kind / cache-eviction race), sandbox gate (allow / deny / throw / truthy-non-boolean / list_scenarios bypass), fixture hardening (__proto__/constructor/prototype/ non-object types), seed cache cap, register warning de-dupe + env opt-outs, and MCP envelope shape.test/server-test-controller.test.jspass unchanged (back-compat).tsc --noEmit); lint clean on new files.require('./dist/lib/testing')resolvescreateComplyController,SEED_SCENARIOS,createSeedFixtureCache.🤖 Generated with Claude Code