Skip to content

feat(storyboards): refs_resolve validation on media_buy_seller (closes #2597)#2601

Merged
bokelley merged 5 commits into
mainfrom
bokelley/refs-resolve
Apr 21, 2026
Merged

feat(storyboards): refs_resolve validation on media_buy_seller (closes #2597)#2601
bokelley merged 5 commits into
mainfrom
bokelley/refs-resolve

Conversation

@bokelley

Copy link
Copy Markdown
Contributor

Summary

  • Documents refs_resolve cross-step validation primitive in storyboard-schema.yaml — sets up the contract for asserting that refs in a source set (e.g. products[*].format_ids[*]) resolve to members of a target set (e.g. formats[*].format_id), matched on declared match_keys.
  • Wires the check onto the list_formats step of media_buy_seller. Every format_id returned on products must resolve to a format in this agent's list_creative_formats response. Scope filter via $agent_url enforces integrity only for refs on the agent under test; third-party refs surface as observations, not failures.
  • Without this check, broken format_id references are silent until sync_creatives fails at runtime — after the media buy is committed.

Closes #2597.

Blocked on

adcp-client#671refs_resolve runtime implementation, shipping in @adcp/client 5.7.0. Our existing ^5.5.0 range already permits 5.7.0, so no package.json bump is needed; CI picks it up on npm install once published. Marked draft until 5.7.0 is released — at that point the refs_resolve validation will execute instead of returning Unknown validation check.

Test plan

  • Unit suite (627 tests) passes
  • Typecheck clean
  • After @adcp/client 5.7.0 ships: run server/tests/manual/storyboard-validation.ts --storyboard media_buy_seller against the reference test agent and confirm the new check executes (pass expected — test agent's format_ids should resolve)
  • Negative manual test: point at an agent with a deliberately-broken format_id and confirm the check names the bad {agent_url, id} in actual.missing

Follow-ups (separate)

  • Cross-agent integrity (calling list_creative_formats on third-party agent_urls) is deferred — currently those refs warn. Gating on a test-kit flag is the natural next step.

🤖 Generated with Claude Code

…ller

Document the refs_resolve check primitive in storyboard-schema.yaml and
wire it onto the list_formats step of media_buy_seller. Every format_id
returned on products is asserted to resolve to a format in this agent's
list_creative_formats response, matched on {agent_url, id}. Scope filtering
via $agent_url enforces integrity only for refs on the agent under test;
third-party refs (format_ids pointing at a different creative agent)
surface as observations rather than failures.

Without this check, broken format_id references are silent until
sync_creatives fails at runtime — after the media buy is already committed.

Closes #2597.
Runtime support ships in @adcp/client 5.7.0 (adcp-client#671).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@bokelley

Copy link
Copy Markdown
Contributor Author

@adcp/client 5.8.0 is out with refs_resolve (adcp-client#671 merged). Installed and verified end-to-end against the reference test agent:

  • Runner recognizes the check — no Unknown validation check error
  • list_formats step now carries 7 validations; the new refs_resolve entry passes
  • Structured observations emitted per out-of-scope ref (as designed)

One real issue surfaced during the probe: on the reference MCP agent, every ref falls out-of-scope because $agent_url expands to the full MCP URL (.../mcp) while format_ids correctly name the bare agent URL. The scope comparison never matches, so the check degrades to warning-only — not enforcing integrity. Filed as adcp-client#710 for the canonicalization fix.

That fix is runner-side and orthogonal to this PR — the storyboard wiring here is correct. Marking ready.

…on follow-up

- Changeset: 5.7.0 → 5.8.0 (actual release containing refs_resolve runtime)
- storyboard-schema.yaml: note adcp-client#710 canonicalization work on
  $agent_url so authors understand the MCP out-of-scope behavior observed
  during verification is a known runner-side refinement, not a spec gap

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@bokelley

Copy link
Copy Markdown
Contributor Author

Ran through protocol / code / security expert review. Findings actioned:

Landed in this PR (commit 9f25acf):

  • Changeset version 5.7.0 → 5.8.0 (actual release)
  • storyboard-schema.yaml notes adcp-client#710 canonicalization work so authors understand the MCP out-of-scope observations are a known runner-side refinement, not a spec gap

Dismissed (false-positive):

  • Code reviewer flagged context.products as never populated. Wrong — CONTEXT_EXTRACTORS.get_products populates it via convention-based extraction; empirical probe already confirmed 4 source refs resolved correctly.

Runner follow-ups filed:

  • adcp-client#710 updated with expert guidance: reuse SDK getCanonicalUrl(), origin-only canonicalization (safer than blind transport-path strip), align with adcp#2343 URL canonicalization
  • adcp-client#711 — warn when scope filter excludes 100% of refs (catches today's silent-no-op on MCP agents)
  • adcp-client#712 — pagination of list_creative_formats in refs_resolve target
  • adcp-client#714 — observations payload hygiene (userinfo strip, size caps, escape contract)

This PR is self-contained and correctly wires the check. The runner refinements land separately in adcp-client and flow through on next release.

# Conflicts:
#	package-lock.json
#	package.json
@bokelley bokelley merged commit 9f2f62c into main Apr 21, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

New storyboard: validate format_id references on products — prevent invalid agent_url/id combos

1 participant