feat: RFC 9421 request-signing conformance grader (#585) by bokelley · Pull Request #600 · adcontextprotocol/adcp-client

bokelley · 2026-04-19T02:39:15Z

Summary

Closes #585. Implements the AdCP Verified storyboard grader for the
signed-requests specialism — a black-box conformance runner that grades any
agent advertising request_signing.supported: true against the 28 RFC 9421
vectors shipped at
/compliance/{version}/test-vectors/request-signing/. Ships in three slices,
each independently tested and reviewable:

Slice 1 (a4bbb7d5) — vector loader + adversarial builder. One mutation
per negative vector, each starting from a freshly-signed baseline via
signRequest() from feat: RFC 9421 request-signing profile (AdCP 3.0 optional) #587. 27 unit tests.
Slice 2 (57bdeb7b) — test-kit YAML loader (adcp#2353 harness
contract), SSRF-guarded HTTP probe, and standalone grader orchestrator
(gradeRequestSigning(agentUrl, options)). Native handling for the three
stateful vectors — 016 repeat-send, 017 pre-revoked keyid, 020 cap+1 —
per the signed-requests-runner test-kit. 6 end-to-end tests against a
localhost reference verifier.
Slice 3 (22988abe) — storyboard-runner integration. Synthesizer
expands positive_vectors / negative_vectors phases with one step per
vector; new request_signing_probe task dispatches via PROBE_TASKS in
the runner. Operators tune via StoryboardRunOptions.request_signing
(skipRateAbuse, rateAbuseCap, skipVectors). 8 integration tests.

Upstream spec dependency adcp#2353 (harness contract for stateful vectors
016/017/020) landed during this work — synthesizer reads requires_contract
and test-revoked-2026, grader enforces the contract at runtime.

Scope

In: grading any agent advertising request_signing.supported: true,
regardless of SDK. Black-box mode only (white-box harnesses can still use
the vectors' test_harness_state directly).
Out: signer-side grading — easier problem, separate storyboard.
Out: CLI entry point — consume programmatically via
@adcp/client/testing/storyboard/request-signing for now.

Test plan

npm run build:lib clean.
npm test — 3623/3625 pass (2 skipped are pre-existing). 20
governance-e2e failures are pre-existing and gated on a training
agent at localhost:4100; verified they fail identically on a
pristine main checkout.
Grader-specific tests (41/41 pass):
- test/request-signing-grader-vectors.test.js — loader + builder
  (27 tests).
- test/request-signing-grader-e2e.test.js — standalone grader vs.
  reference verifier, capability-profile subtests for 007/018, tight-cap
  test for 020 (6 tests).
- test/request-signing-runner-integration.test.js — synthesis + probe
  dispatch, skip options, unknown-step handling, mismatch diagnostic
  (8 tests).

Upstream spec: adcp#2323 (profile), adcp#2331 (specialism), adcp#2353
(harness contract).
SDK verifier side: feat: RFC 9421 request-signing profile (AdCP 3.0 optional) #587 (merged).
Follow-ups already landed on main: feat: auto-apply RFC 9421 signing to outbound MCP/A2A (#578) #593 (auto-wire Auto-sign outbound MCP/A2A requests in AdcpClient using request-signing primitives #578),
fix(signing): structured-headers parser + bucketed replay store (#581, #582) #596 (structured-headers parser + bucketed replay).

🤖 Generated with Claude Code

Addresses six-agent expert review findings on #600. Correctness: - Skipped vectors now propagate via HttpProbeResult.skipped (new) through the storyboard runner, bypassing validations. Previously skipped vectors scored as FAILED because probe-dispatch set error, which the runner's fetchOk check treated as a failed fetch. - Synthesis failure surfaces as a failing synthesis_error phase instead of a silent empty-phase fallback. CI pipelines would have seen green for an infrastructural bug. - Vector 010 digest-mismatch now tests the intended invariant (signer commits a zero-byte digest in the signature base) rather than body-tampered-in-transit. - Vector 009 honors its pinned jwks_ref (test-gov-2026) directly. - runStoryboardStep now threads agentUrl into the ExecutionState so probe tasks receive the target URL (runStoryboard passes it already; the single-step entry point did not). Safety: - Vectors 016 + 020 auto-skip against non-sandbox endpoints unless allowLiveSideEffects: true. Prevents accidental live create_media_buy and replay flood against production. - endpoint_scope_warning → live_endpoint_warning (inverted, now true when scope is NOT sandbox — prior semantics read "sandbox is bad"). Hardening: - extractSignatureErrorCode constrains return to [a-z0-9_]+; prevents smuggled content in agent-controlled WWW-Authenticate from reaching diagnostics or LLM-consumption paths. - splitChallenges tracks quote state so adversarial error="foo, Bar baz" doesn't spuriously split. DX: - New CLI: adcp grade request-signing <agent-url> with full option set (--skip-rate-abuse, --rate-abuse-cap, --only, --skip, --allow-live-side-effects, --allow-http, --json). - GradeReport adds passed_count / failed_count / skipped_count. - GradeOptions.onlyVectors filters to a subset; replaces three hand-maintained 19-entry skip arrays across tests. - Barrel grouped into Public / Runner-hooks / Advanced sections with module JSDoc. - BuildOptions.baseUrl now prefixes the agent's mount path to vector paths so agents at /v1/adcp/* are reachable. Hygiene: - ContractId single source of truth in types.ts. - AdcpJsonWebKey.d is an explicit field with JSDoc (was via index sig). - loadRequestSigningVectors memoizes per-cacheDir (immutable during process lifetime). - Extracted startReferenceVerifier / makeExpressShim to test/utils/reference-verifier.js. - Dispatch wire-up test via runStoryboardStep catches someone removing request_signing_probe from PROBE_TASKS. Refs #585. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…ygiene Addresses non-blocking follow-ups from the round-4 expert re-reviews on #600. - grader.ts: default live_endpoint_warning to TRUE when no test-kit contract is loaded (can't prove sandbox → warn). Previously silently ran vectors 016/020 without the banner if the contract file was missing. - test-kit.ts: drop redundant Record<ContractId, ...> intersection from stateful_vector_contract interface; explicit shape is already there. - adcp-grade.js: --skip / --only now reject empty or missing values with a clear error (matches --timeout / --rate-abuse-cap ergonomics). - probe.ts: drop dead lastTokenWasScheme variable from splitChallenges. - Add three new preflightSkip unit tests covering operator skip paths (onlyVectors / skipVectors / skipRateAbuse). All 46 grader tests pass. Refs #585. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…ilder Internal module at src/lib/testing/storyboard/request-signing/ that walks the RFC 9421 conformance vectors under compliance/cache/{version}/test-vectors/ request-signing/, parses them into typed PositiveVector/NegativeVector values (including the `requires_contract` field coming from adcp#2353), and loads keys.json with the private scalars needed for dynamic re-signing. Adversarial builder registers one mutation per negative vector (20 total) — each starts from a freshly-signed baseline via src/lib/signing/signer.ts and applies the single documented mutation so the grader can send real requests to a live verifier rather than replay stale `reference_now` signatures. Stateful vectors (016/017/020) produce a single well-formed request; the storyboard runner phase (slice 2) orchestrates repeat/flood/revoked-keyid behavior per the signed-requests-runner test-kit contract. Refs #585. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…e verifier Standalone grader that consumes the Slice 1 loader+builder and runs all 28 conformance vectors against a live agent in black-box mode, returning a per-vector GradeReport. Three new modules under src/lib/testing/storyboard/request-signing/: - test-kit.ts loads the signed-requests-runner harness contract YAML (adcp#2353) — signing keyids, replay/revocation/rate-abuse contracts, endpoint_scope, harness_mode. - probe.ts POSTs a SignedHttpRequest to the agent and parses the response (status + WWW-Authenticate error code). Reuses the SSRF guards from storyboard/probes.ts. - grader.ts orchestrates the 28 vectors with native handling for the three stateful contracts: 016 uses the replay repeat-request behavior, 017 uses the pre-revoked keyid, 020 fills the per-keyid cap then probes cap+1. skipRateAbuse / rateAbuseCap / skipVectors options for operator tuning. Builder gains BuildOptions.baseUrl so signatures are computed against the operator-supplied agent URL rather than the vector's seller.example.com. test/request-signing-grader-e2e.test.js stands up the #587 Express verifier middleware on localhost and grades against it end-to-end: positive acceptance, 17 non-stateful negatives with byte-for-byte error-code match, replay/revocation/rate-abuse contracts, and capability-specific profiles (covers_content_digest = required / forbidden) for vectors 007 and 018. Storyboard-runner step synthesis (expanding specialism phases with per-vector steps + request_signing_probe task dispatch in runner.ts) is deferred to slice 3. Refs #585. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Wires the Slice 2 grader into the storyboard runner so the signed-requests specialism grades end-to-end through the existing pipeline. The YAML ships positive_vectors and negative_vectors phases with empty step lists — this slice synthesizes one step per vector at load time and dispatches request_signing_probe through the runner's probe path. New modules: - request-signing/synthesize.ts expands the vector phases with typed steps whose IDs encode the vector (positive-<id> / negative-<id>). skipVectors filters at synthesis time. - request-signing/probe-dispatch.ts decodes the step ID, invokes gradeOneVector, and maps VectorGradeResult into an HttpProbeResult so existing http_status / http_status_in validations work unchanged. Touched code: - compliance.ts hooks synthesis into loadBundleStoryboards with a fallback warning when vectors are missing from the cache. - loader.ts tolerates phases with no steps: key. - probes.ts registers request_signing_probe in PROBE_TASKS; runner.ts dispatches to the new module. - types.ts adds StoryboardRunOptions.request_signing {skipRateAbuse,rateAbuseCap,skipVectors}. - storyboard-completeness.test.js adds request_signing_probe to the HARNESS_TASKS exception list (no sample_request — request is built from fixtures). Integration tests verify synthesis output, probe dispatch against the reference verifier (positive, negative, skipRateAbuse, skipVectors, unknown-step, capability-profile mismatch). Refs #585. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Addresses six-agent expert review findings on #600. Correctness: - Skipped vectors now propagate via HttpProbeResult.skipped (new) through the storyboard runner, bypassing validations. Previously skipped vectors scored as FAILED because probe-dispatch set error, which the runner's fetchOk check treated as a failed fetch. - Synthesis failure surfaces as a failing synthesis_error phase instead of a silent empty-phase fallback. CI pipelines would have seen green for an infrastructural bug. - Vector 010 digest-mismatch now tests the intended invariant (signer commits a zero-byte digest in the signature base) rather than body-tampered-in-transit. - Vector 009 honors its pinned jwks_ref (test-gov-2026) directly. - runStoryboardStep now threads agentUrl into the ExecutionState so probe tasks receive the target URL (runStoryboard passes it already; the single-step entry point did not). Safety: - Vectors 016 + 020 auto-skip against non-sandbox endpoints unless allowLiveSideEffects: true. Prevents accidental live create_media_buy and replay flood against production. - endpoint_scope_warning → live_endpoint_warning (inverted, now true when scope is NOT sandbox — prior semantics read "sandbox is bad"). Hardening: - extractSignatureErrorCode constrains return to [a-z0-9_]+; prevents smuggled content in agent-controlled WWW-Authenticate from reaching diagnostics or LLM-consumption paths. - splitChallenges tracks quote state so adversarial error="foo, Bar baz" doesn't spuriously split. DX: - New CLI: adcp grade request-signing <agent-url> with full option set (--skip-rate-abuse, --rate-abuse-cap, --only, --skip, --allow-live-side-effects, --allow-http, --json). - GradeReport adds passed_count / failed_count / skipped_count. - GradeOptions.onlyVectors filters to a subset; replaces three hand-maintained 19-entry skip arrays across tests. - Barrel grouped into Public / Runner-hooks / Advanced sections with module JSDoc. - BuildOptions.baseUrl now prefixes the agent's mount path to vector paths so agents at /v1/adcp/* are reachable. Hygiene: - ContractId single source of truth in types.ts. - AdcpJsonWebKey.d is an explicit field with JSDoc (was via index sig). - loadRequestSigningVectors memoizes per-cacheDir (immutable during process lifetime). - Extracted startReferenceVerifier / makeExpressShim to test/utils/reference-verifier.js. - Dispatch wire-up test via runStoryboardStep catches someone removing request_signing_probe from PROBE_TASKS. Refs #585. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…ygiene Addresses non-blocking follow-ups from the round-4 expert re-reviews on #600. - grader.ts: default live_endpoint_warning to TRUE when no test-kit contract is loaded (can't prove sandbox → warn). Previously silently ran vectors 016/020 without the banner if the contract file was missing. - test-kit.ts: drop redundant Record<ContractId, ...> intersection from stateful_vector_contract interface; explicit shape is already there. - adcp-grade.js: --skip / --only now reject empty or missing values with a clear error (matches --timeout / --rate-abuse-cap ergonomics). - probe.ts: drop dead lastTokenWasScheme variable from splitChallenges. - Add three new preflightSkip unit tests covering operator skip paths (onlyVectors / skipVectors / skipRateAbuse). All 46 grader tests pass. Refs #585. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

# Conflicts: # bin/adcp.js

…wiring Framework — AdcpCapabilitiesConfig gains `request_signing` + `specialisms` fields. createAdcpServer-based agents can now advertise the RFC 9421 verifier capability and specialism claims via one-liner config instead of forking the capability assembly path. Framework — serve() gains `preTransport` middleware hook. Runs between path-matching and MCP transport connect; body is buffered into req.rawBody for signature verifiers. When the hook handles the response (e.g. 401 with WWW-Authenticate), the transport is skipped. Parsed body is passed to transport.handleRequest so the consumed stream doesn't race. Test agent — test-agents/seller-agent-signed.ts is a minimal signed verifier pre-configured per test-kits/signed-requests-runner.yaml: runner JWKS (test-ed25519-2026 + test-es256-2026 + test-gov-2026 + test-revoked-2026), pre-revoked test-revoked-2026, per-keyid replay cap = 100. Exposes /get_adcp_capabilities declaring specialisms: ['signed-requests']. Signed requests on any other path route operation from the last path segment. End-to-end validation: grader against this agent passes 25/25 graded vectors (3 skipped — capability-profile + rate-abuse opt-out). Proves the full signer → verifier → grader pipeline works against a live agent. Note: this is a raw-HTTP verifier, not MCP. Conformance vectors target per-operation AdCP paths (/adcp/create_media_buy); MCP agents expose a single /mcp endpoint with operation in the JSON-RPC body. An MCP-aware grader (envelope wrapping + single-endpoint routing) is deferred to a follow-up issue. Refs #585. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

# Conflicts: # src/lib/testing/storyboard/types.ts

…gning grader (#600) main's new request-signing grader (#585) introduced grader-specific skip reasons (probe_skipped, rate_abuse_opt_out, missing_test_kit_contract, live_side_effect_opt_in_required, operator_skip, not_in_only_vectors, grader_skipped). Keep the six canonical RunnerSkipReason values as the contract-conformant set on skip.reason, and widen skip_reason to also accept RunnerDetailedSkipReason. A DETAILED_SKIP_TO_CANONICAL map projects grader reasons onto their spec equivalents so runner-output consumers see a stable enum regardless of which subsystem emitted the skip. The probe self-skip branch now populates the full set of contract-required fields (request, response_record, extraction, skip) alongside the detailed skip_reason. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Closes #612. Adds transport-aware grading so the conformance grader works against MCP agents, not just raw-HTTP AdCP endpoints. GradeOptions.transport: 'raw' | 'mcp' (default 'raw'). In 'mcp' mode: - URL becomes baseUrl as-is (no path-join); MCP agents have one endpoint. - Body wraps in a JSON-RPC tools/call envelope; operation comes from the vector URL's last path segment. - Accept: application/json, text/event-stream added so MCP Streamable HTTP servers don't 406 the probe. Signature covers the envelope body. Applied uniformly through sign(), signWithParamOverride(), signWithComponents(), and the 001 mutation — every mutation path produces MCP-shaped requests when transport === 'mcp'. CLI: --transport <raw|mcp> on `adcp grade request-signing`. Validated; unknown values exit 2 with a clear error. Test agent: test-agents/seller-agent-signed-mcp.ts uses createAdcpServer (with request_signing + specialisms advertised) + serve({ preTransport }) with the verifier middleware. Valid requests flow into createMediaBuy; invalid requests get 401 + WWW-Authenticate before MCP dispatch. E2E test: test/request-signing-grader-mcp.test.js spawns the MCP agent, grades it in MCP mode, asserts 25/25 non-profile vectors pass + envelope structural invariants. All 3 subtests green. Full suite: 3953/3955 pass, 0 fail, 2 skipped (pre-existing). With this change + #600 (grader) + adcp#2368 (deploy verifier on the production test agent), the live-agent smoke test path is complete. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: grader MCP transport mode (#612) Closes #612. Adds transport-aware grading so the conformance grader works against MCP agents, not just raw-HTTP AdCP endpoints. GradeOptions.transport: 'raw' | 'mcp' (default 'raw'). In 'mcp' mode: - URL becomes baseUrl as-is (no path-join); MCP agents have one endpoint. - Body wraps in a JSON-RPC tools/call envelope; operation comes from the vector URL's last path segment. - Accept: application/json, text/event-stream added so MCP Streamable HTTP servers don't 406 the probe. Signature covers the envelope body. Applied uniformly through sign(), signWithParamOverride(), signWithComponents(), and the 001 mutation — every mutation path produces MCP-shaped requests when transport === 'mcp'. CLI: --transport <raw|mcp> on `adcp grade request-signing`. Validated; unknown values exit 2 with a clear error. Test agent: test-agents/seller-agent-signed-mcp.ts uses createAdcpServer (with request_signing + specialisms advertised) + serve({ preTransport }) with the verifier middleware. Valid requests flow into createMediaBuy; invalid requests get 401 + WWW-Authenticate before MCP dispatch. E2E test: test/request-signing-grader-mcp.test.js spawns the MCP agent, grades it in MCP mode, asserts 25/25 non-profile vectors pass + envelope structural invariants. All 3 subtests green. Full suite: 3953/3955 pass, 0 fail, 2 skipped (pre-existing). With this change + #600 (grader) + adcp#2368 (deploy verifier on the production test agent), the live-agent smoke test path is complete. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: address review feedback on #617 (items 1-8) Addresses expert review findings on PR #617. Must-fix: - Add build:test-agents script to package.json. The file headers of both test-agents/seller-agent-signed{,-mcp}.ts told operators to run it; it didn't exist. First-contact path now works. - Auto-skip canonicalization-edge positive vectors 005-008 in MCP mode. MCP flattens every vector URL to the same MCP endpoint, neutralizing the port/path/query/encoding edges. Skipping with a distinct skip_reason prevents the report from claiming coverage it didn't deliver. MCP_FLATTENED_VECTORS is an explicit list so future canonicalization vectors have to opt in. - Update seller-agent-signed.ts stale header — used to say MCP-aware grader was "future work (issue TBD)"; now cross-links to the MCP variant + --transport mcp. Should-fix: - wrapMcpEnvelope now lets JSON.parse throw on non-JSON vector bodies instead of silently degrading to a string. Vectors ship JSON by spec contract; a parse failure is a vector drift worth surfacing. - mcpRequestIdCounter removed — JSON-RPC ids now come from crypto.randomUUID(). Removes module-level mutable state that could leak between concurrent graders in the same process. - MCP e2e startup detection rejects on timeout OR on child exit before ready, with captured stderr/stdout. Previous 3s "assume started" fallback hid crashes behind 20s of connect errors in the grader. - Added negative-vector MCP-shape matrix test — iterates all 20 mutations, asserts each produces url === baseUrl + JSON-RPC tools/call envelope. Locks the applyTransport invariant at every mutation site. Follow-up: - CLI hint: when every graded vector fails with a 404/405/fetch-error shape AND --transport wasn't explicitly set to mcp, surface a 💡 hint suggesting --transport mcp. Based on the common mistake of grading an MCP agent in the default raw mode. Also: subprocess orphan-reaper via process.on('exit') in MCP e2e test — prevents EADDRINUSE on the next CI run when the parent crashes abnormally. All 4 MCP tests pass; full suite 3911/3913 pass (2 pre-existing skips). Refs #617. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(ci): auto-build test-agents in MCP e2e before() hook CI runs `npm test` without `npm run build:test-agents`, so the MCP e2e test's subprocess agent (test-agents/dist/seller-agent-signed-mcp.js) was missing on fresh runs and all 4 subtests cancelled with "Cannot find module". My round-1 startup-reject change helpfully surfaced the cause instead of hanging for 20s. Fix: before() hook calls tsc on test-agents/tsconfig.json if dist is missing. Cost is one-time per CI run (~1–2s tsc). No workflow file edit needed; the test self-builds. Bumped startup timeout to 10s for cold-start headroom. Attempted a tsx-based approach first but tsx doesn't honor the root tsconfig's `paths` mapping for `structured-headers`, so ParseError imports become undefined and the verifier crashes on instanceof. Dist path + auto-build is simpler and matches how other test files invoke compiled agents. All 4 MCP tests pass from a cold (no dist) start in 2.7s locally; full suite 3954/3956 pass (2 pre-existing skips). Refs #617. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Two upstream landings touched skill guidance: - #603 split the old plan.authority_level enum into plan.human_review_required (GDPR Art 22 / EU AI Act, covers data-subject decisions) and budget.reallocation_threshold / reallocation_unlimited (budget autonomy). Plan shape updated; handler code updated; spend-authority specialism row updated. - #600 shipped the RFC 9421 signed-requests grader with three phases (capability_discovery, positive_vectors, negative_vectors). Skill no longer claims "runner not yet implemented"; signed-requests section now documents the grading surface (WWW-Authenticate error codes), test-kit prerequisites, and the revoked-keyid / per-keyid cap requirements. No changeset — docs only.

…ilder testing (#624) * docs(skills): add specialism coverage to every build-*-agent skill Every SKILL.md now has a "Specialisms this skill covers" table mapping specialism IDs to concrete deltas, plus per-specialism sections where the compliance storyboard requires behavior the baseline did not cover. Root CLAUDE.md gets an inverse specialism → skill index. Derived from a fresh-builder test against all 21 specialisms; median build confidence was 3/5 on the prior skills, with six specialisms at ≤2/5. Upstream issues filed as adcontextprotocol/adcp#2284-#2288. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: align skills to AdCP 3.0 GA * fix(skills): correct shapes round-2 builder tests found wrong * fix: align skills with upstream main (escalated drop, signing API) Upstream merges today invalidated two pieces of guidance: - #592 dropped the 'escalated' check_governance status. The spec enum is now approved | denied | conditions. Human review is signalled via a denied decision with a critical-severity finding, and the buyer resolves review off-protocol before re-calling. - #593 landed the real signing API surface (verifyRequestSignature, createExpressVerifier, VerifierCapability). My earlier skill guidance imported a fictional verifyRfc9421Signature helper. Also fixes the approved-with-conditions response shape — was status:'approved' plus a conditions[] array, should be status:'conditions' (the enum itself has a conditions value). No changeset — docs only. * fix(skills): align governance + signed-requests to main Two upstream landings touched skill guidance: - #603 split the old plan.authority_level enum into plan.human_review_required (GDPR Art 22 / EU AI Act, covers data-subject decisions) and budget.reallocation_threshold / reallocation_unlimited (budget autonomy). Plan shape updated; handler code updated; spend-authority specialism row updated. - #600 shipped the RFC 9421 signed-requests grader with three phases (capability_discovery, positive_vectors, negative_vectors). Skill no longer claims "runner not yet implemented"; signed-requests section now documents the grading surface (WWW-Authenticate error codes), test-kit prerequisites, and the revoked-keyid / per-keyid cap requirements. No changeset — docs only. * fix(skills): round-3 bug-fix pass — shapes, phase field, signed-requests wiring Round-3 fresh-builder tests surfaced 4 residual bugs (6 specialisms tested, median 5/4): - governance-delivery-monitor summary-table row said 'delivery_evidence' but code and storyboard use 'delivery_metrics'. Fixed. - Skill claimed 'phase' is not on the wire — storyboard sends phase:'delivery' and llms.txt lists it as optional. Now: phase is optional-but-authoritative; handler routes on phase || delivery_metrics presence. - validate_content_delivery 'artifact.assets' was shown as an object; actual shape is an array of {type, url, width, height, duration_ms?}. Also added 'artifact.description' which the storyboard uses for calibration matching. - signed-requests: showed JWKS with a comment 'mark revoked before grading' but never showed the revocationStore.insert call, had no per-keyid cap example for vector 020, and no mount-order warning for express body-parsers (covers_content_digest silently breaks if JSON parser runs first). All three now have copy-pasteable snippets. Also fixed CLAUDE.md stale 'delete_audience' reference in the specialism index — deletion rides on sync_audiences, no separate tool. No changeset — docs only. * feat(skills): unify idempotency pattern and add OAuth guidance for sellers Two cross-cutting fixes on the seller skill's Protocol-Wide Requirements section: 1. Idempotency — replace the manual ctx.store.get('idempotency',...)/put pattern (which reinvents the wheel and misses payload-hash conflict detection) with the real createIdempotencyStore + createAdcpServer({ idempotency }) wiring. Matches what build-si-agent already uses. Framework handles replay detection, IDEMPOTENCY_KEY_CONFLICT, and ttl validation. 2. OAuth — section was missing entirely. Adds a focused subsection covering the one thing the seller actually has to do right (WWW-Authenticate Bearer challenge + RFC 9728 / RFC 8414 metadata documents). Clients handle discovery and NeedsAuthorizationError; sellers just need to advertise correctly. Points at npx @adcp/client diagnose-auth for validation. No changeset — docs only. * feat(skills): composition guide for OAuth + signing + idempotency Round-4 cross-cutting test scored 2/5 on composition despite 4-5/5 on each concern in isolation. The builder surfaced four specific gaps at the boundaries between OAuth, signing, idempotency, and async submitted flows: 1. Middleware mount order — OAuth → signature verify → JSON parse → MCP transport → framework idempotency. Raw body must reach the verifier; bearer validation runs first so we don't canonicalize unauthenticated requests. 2. Principal threading — resolveSessionKey MUST come from an authenticated identity (OAuth subject + verified signing keyid), not self-declared buyer metadata. Two buyers on one OAuth tenant with different signing keys must not share a replay namespace. 3. 401 disambiguation — OAuth fails first. Bearer challenge goes out alone when the bearer is missing/expired; Signature challenge only when the request is authenticated but signed wrong. RFC 7235 allows multiple challenges; the signed-requests grader doesn't exercise the combined case. 4. Idempotency × submitted — ground-truthed against src/lib/server/ idempotency/store.ts: the framework caches successful mutations including submitted envelopes, injects replayed:true on replay, and uses a 120s in-flight TTL matching the AdCP working-response timeout. Second IO is not created on replay; setup.url stays stable. All ground-truthed against src/ before writing; the example middleware chain is not speculative. No changeset — docs only. * docs(skills): name idempotency error codes, JWT claim mapping, cross-links Three R5 follow-ups, all ground-truthed against src/: 1. Name the three idempotency error codes the framework emits, with a table keyed on what triggers each and what the buyer should do. In-flight parallels return SERVICE_UNAVAILABLE with retry_after:1 (verified in src/lib/server/create-adcp-server.ts). Handler authors no longer need to grep framework source to write replay assertions. 2. OAuth claim-to-principal paragraph. Covers aud/iss/scope validation, using sub as the principal, multi-tenant composition (tenant:sub to avoid sub collisions across tenants), and the OAuth/signing keyid reconciliation rule. resolveSessionKey example updated to compose tenant, subject, and keyid. 3. Cross-links: the OAuth section and the signed-requests specialism section both point down to the composition subsection. Added an explicit anchor (composing-oauth-signing-and-idempotency) so the links are stable across heading edits. No changeset — docs only. * docs(skills): align to upstream serve() composition API * fix(skills): correct exports, preTransport wiring, and sales-guaranteed task shape * fix(skills): brand-rights schema accuracy * chore: regen schemas + add changeset for PR #624 * fix(skills): address expert review blockers on PR #624 * fix(skills): verifyApiKey/verifyBearer/anyOf import path * docs(skills): expert-review follow-ups (pwr sections, shape fixes, nits) * docs: add 5.1->5.2 migration guide; trim seller skill length * docs: link migration guide from CLAUDE.md * docs: migration guide covers 4.30.1 -> 5.2.0 * docs: migration guide + skills cover 5.2 webhook surface * fix(skills): webhook JWK example, measurement_windows shape, rights-terms exclusivity * docs(skills): distinguish submitted vs pending_creatives + inventory-list persistence --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

bokelley and others added 6 commits April 19, 2026 06:18

style: prettier formatting for grader fixes

c1372e1

bokelley force-pushed the bokelley/pr-585-review branch from 7e9a9fa to 1ad976b Compare April 19, 2026 10:20

bokelley added 2 commits April 19, 2026 06:22

chore: trigger CI on rebased branch

aee27f0

Merge remote-tracking branch 'origin/main' into bokelley/pr-585-review

9ce6dbd

# Conflicts: # bin/adcp.js

This was referenced Apr 19, 2026

signing: live-verifier storyboard that exercises HttpsJwksResolver + HttpsRevocationStore end-to-end #609

Closed

signing: signer-side conformance grader (complement to the verifier grader in #585) #610

Closed

bokelley mentioned this pull request Apr 19, 2026

request-signing grader: MCP-aware mode (JSON-RPC envelope wrapping) #612

Closed

github-advanced-security AI found potential problems Apr 19, 2026

View reviewed changes

Comment thread test-agents/seller-agent-signed.ts Fixed

Comment thread test-agents/seller-agent-signed.ts Fixed

bokelley mentioned this pull request Apr 19, 2026

Stand up signed-requests on test-agent.adcontextprotocol.org adcontextprotocol/adcp#2368

Closed

Merge remote-tracking branch 'origin/main' into bokelley/pr-585-review

c7711dc

# Conflicts: # src/lib/testing/storyboard/types.ts

bokelley merged commit d116a07 into main Apr 19, 2026
12 checks passed

bokelley mentioned this pull request Apr 19, 2026

test(signing): live-verifier integration for HTTPS stores #616

Merged

6 tasks

bokelley mentioned this pull request Apr 19, 2026

feat: grader MCP transport mode (#612) #617

Merged

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: RFC 9421 request-signing conformance grader (#585)#600

feat: RFC 9421 request-signing conformance grader (#585)#600
bokelley merged 10 commits into
mainfrom
bokelley/pr-585-review

bokelley commented Apr 19, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

bokelley commented Apr 19, 2026

Summary

Scope

Test plan

Related

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants