Skip to content

feat(storyboard): add adversarial builder for request-signing vector 028#1644

Merged
bokelley merged 1 commit into
mainfrom
bokelley/request-signing-vector-028-builder
May 10, 2026
Merged

feat(storyboard): add adversarial builder for request-signing vector 028#1644
bokelley merged 1 commit into
mainfrom
bokelley/request-signing-vector-028-builder

Conversation

@bokelley

Copy link
Copy Markdown
Contributor

Summary

Wires the SDK-side adversarial builder for AdCP conformance vector 028 (negative/028-unsigned-protocol-method-required.json). Vector 028 was added upstream in:

  • adcp#4326request_signing.protocol_methods_* namespace (schema + spec text + test-agent enforcement)
  • adcp#4327 — the conformance vector itself

Without this builder, the storyboard runner errored with No adversarial builder registered for negative vector "028-unsigned-protocol-method-required". test-agent.adcontextprotocol.org carried a skipVectors: ['028-unsigned-protocol-method-required'] workaround; this PR removes that pressure.

Changes

  1. New protocolMethodPassthrough mutator in src/lib/testing/storyboard/request-signing/builder.ts. The vector body is already a JSON-RPC envelope with method: "tasks/cancel" — NOT a tools/call envelope — so the standard MCP-mode applyTransport (which wraps the body in tools/call) is wrong. Pass through verbatim; target baseUrl directly when set.
  2. VerifierCapabilityFixture extended with protocol_methods_supported_for / protocol_methods_required_for, mirroring the schema landed in adcp#4326.
  3. capabilityMismatch extended in grader.ts to gate on protocol_methods_required_for. Agents that don't declare the bucket auto-skip with capability_profile_mismatch — same shape as the existing required_for gate.
  4. vector-loader.ts parses the new protocol_methods_* fields off vector fixtures.
  5. Inline-fixture unit tests (the cache doesn't ship vector 028 yet — that arrives with the next spec release) covering passthrough behavior, no-tools-call-wrapping, signature header stripping, and MCP-mode URL targeting.

Test plan

  • npm run build:lib
  • npx tsc --project tsconfig.lib.json --noEmit (clean)
  • node --test test/request-signing-vector-028-protocol-methods.test.js (5/5)
  • node --test test/request-signing-grader-vectors.test.js test/request-signing-vectors.test.js (88/88 — no regression on existing vector tests)
  • Once a release of the spec ships vector 028 to compliance/cache, the vector-loader count test in request-signing-grader-vectors.test.js will pick up negative.length === 28 automatically. Tracked under spec adcp#2331 (runner umbrella).
  • Remove the skipVectors: ['028-unsigned-protocol-method-required'] line from test-agent's server/tests/manual/run-storyboards.ts once a release of @adcp/sdk carrying this PR is consumed there.

Cross-namespace note

Cross-namespace match prevention (a signed tools/call with params.name: "tasks/cancel" MUST NOT satisfy protocol_methods_required_for) is a server-side rule enforced at the verifier — the SDK's adversarial builder doesn't construct such a probe. Test-agent's mcpOperationResolver covers the resolver-side defense and is unit-tested in adcp#4326.

🤖 Generated with Claude Code

Vector 028 was added in adcp#4326 / adcp#4327 (the
`request_signing.protocol_methods_*` namespace). It grades whether a
verifier that declares `protocol_methods_required_for: ["tasks/cancel"]`
actually rejects unsigned `tasks/cancel` JSON-RPC POSTs with
`request_signature_required`.

Without an SDK adversarial builder, the storyboard runner errored on 028
and test-agent.adcontextprotocol.org skipped via `skipVectors` as a
workaround. This PR removes that workaround:

- New `protocolMethodPassthrough` mutator: the vector body is a JSON-RPC
  envelope with `method: "tasks/cancel"`, NOT a `tools/call` envelope, so
  the standard MCP-mode `applyTransport` (which would wrap in tools/call)
  is wrong. Pass through verbatim, target `baseUrl` directly.
- `VerifierCapabilityFixture` extended with `protocol_methods_supported_for`
  / `protocol_methods_required_for`. `capabilityMismatch` gates on the new
  bucket — agents that don't declare it auto-skip with
  `capability_profile_mismatch`, same shape as the `required_for` gate.
- `vector-loader.ts` parses the new fields so the grader sees them when
  the cache ships vector 028.
- Inline-fixture unit tests (cache doesn't ship 028 yet) cover the
  passthrough behavior, no-tools-call-wrapping, signature header stripping,
  and MCP-mode URL targeting.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@bokelley bokelley force-pushed the bokelley/request-signing-vector-028-builder branch from e2a25d9 to 729886b Compare May 10, 2026 13:11
@bokelley bokelley merged commit 086f6d8 into main May 10, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant