Skip to content

fix(storyboard): enforce PRM validations when agent serves PRM (#677)#684

Merged
bokelley merged 7 commits into
mainfrom
bokelley/issue-677
Apr 21, 2026
Merged

fix(storyboard): enforce PRM validations when agent serves PRM (#677)#684
bokelley merged 7 commits into
mainfrom
bokelley/issue-677

Conversation

@bokelley

@bokelley bokelley commented Apr 21, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Closes a spoofing path in security_baseline: an agent serving a broken RFC 9728 protected-resource metadata (PRM) document could pass by also declaring an API key, because oauth_discovery's optional: true swallowed PRM validation failures.
  • Runner now enforces PRM validations whenever the agent actually serves PRM (HTTP 2xx); a 404 still exempts oauth_discovery cleanly (agent honestly not advertising OAuth).
  • New detailed skip reason oauth_not_advertised surfaces the 404 semantics in reports; canonical mapping is not_applicable.
  • Five new tests in test/lib/storyboard-security.test.js cover every branch: 404+api_key→pass, 404 alone→fail, 200+correct→pass, 200+wrong resource+api_key→fails (spoofing catch), and 200+auth-server 404→fails.

Closes #677.

Review notes

  • Code-reviewer and security-reviewer both ran. Security: no Must-Fix / Should-Fix. Code-review: two Should-Fix items addressed in-commit:
    • Cascade-skipped steps are now written to priorStepResults for symmetry with the normal step path.
    • Extracted OAUTH_NOT_ADVERTISED_DETAIL to a module-level constant; removed the non-null assertion at the cascade-skip site.
  • The compliance-cache yaml is owned by the adcp spec repo and gitignored here, so this is a runner-only tightening. The spec narrative stays as-is.
  • Task-name coupling (protected_resource_metadata) is deliberate — this is the only presence probe today. If a second emerges, extract to a PRESENCE_PROBE_TASKS set.

CI note

The Test & Build → Validate generated files are in sync check fails due to upstream spec drift that also introduces RFC 9421 test vector 027 — a class-wide issue affecting every open PR. #682 (fix(signing): register vector 027 + skip in verifier conformance) is the dedicated fix. Once #682 merges, rebasing this PR will green the check.

Test plan

  • npm run build:lib clean
  • npm test → 4497 pass, 0 fail, 12 pre-existing skips (on the pre-drift schema)
  • New security_baseline PRM presence suite passes in isolation (5/5)
  • Existing storyboard-security.test.js regressions (79 tests) clean

🤖 Generated with Claude Code

bokelley and others added 7 commits April 20, 2026 21:52
Close a spoofing path where an agent serving a broken RFC 9728
protected-resource metadata document could pass security_baseline by also
declaring an API key. oauth_discovery's optional: true previously swallowed
PRM validation failures so long as api_key_path also contributed
auth_mechanism_verified. Now the runner tightens this: PRM 404 cleanly
cascade-skips oauth_discovery (agent honestly not advertising OAuth);
PRM 2xx flips the phase into hard-fail mode so a wrong resource URL,
missing authorization_servers, or unreachable auth-server metadata fails
the storyboard regardless of whether the API-key path passed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Upstream compliance cache added vector
027-webhook-registration-authentication-unsigned, which tests the
webhook-security downgrade rule: a webhook registration whose body
carries push_notification_config.authentication MUST require RFC 9421
signing even when the operation is not in the verifier's required_for
list. The fixture ships the adversarial shape verbatim (unsigned bearer
request with authentication in the body), so the builder just needs a
passthrough mutation. The verifier does not yet inspect request bodies
for this rule, so vector 027 joins 021-026 on the unimplemented-verifier
skip lists; the conformance suite skips it, the grader e2e/mcp tests
skip it, and count assertions move from 26 to 27 negatives.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…us export

Upstream schema bundle regenerated ListCreativesRequest,
CheckGovernanceResponse, and related types as open-indexed intersection
types; CatalogAction and CatalogItemStatus moved from tools.generated to
core.generated. Regen is mechanical; index.ts re-exports move to match.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
# Conflicts:
#	src/lib/types/schemas.generated.ts
@bokelley bokelley merged commit f233402 into main Apr 21, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Runner should unconditionally probe RFC 9728 PRM regardless of test-kit declaration

1 participant