Skip to content

fix(compliance): flow-derived id captures, context-echo cleanup, format_id integrity step#2600

Merged
bokelley merged 2 commits into
mainfrom
bokelley/storyboard-hygiene
Apr 21, 2026
Merged

fix(compliance): flow-derived id captures, context-echo cleanup, format_id integrity step#2600
bokelley merged 2 commits into
mainfrom
bokelley/storyboard-hygiene

Conversation

@bokelley

Copy link
Copy Markdown
Contributor

Summary

Closes #2585, #2589, #2597 at the storyboard-YAML layer. Each of these issues has a spec/doc side that already landed (#2585 + #2589 via #2593, #2597 is net-new); this PR completes the storyboard migration so the training-agent conformance suite stops tripping on hardcoded fixtures and missing context blocks.

#2588 is not touched — test:error-codes lint + alias file landed in #2595 + #2596; issue stayed open because the close-on-merge marker didn't fire.

What changed

#2585 — flow-derived IDs via $context.media_buy_id. Seven storyboards hardcoded mb_* literals against media buys created earlier in the same run. Each create_media_buy step now declares context_outputs: [{ name: media_buy_id, path: "media_buy_id" }], and downstream get_media_buys / update_media_buy / get_media_buy_delivery / report_plan_outcome steps resolve "$context.media_buy_id". Storyboards covered: protocols/media-buy/index.yaml, protocols/governance/index.yaml (governance_escalation phase), and specialisms sales-guaranteed / sales-non-guaranteed / sales-proposal-mode / sales-broadcast-tv / sales-catalog-driven / creative-generative/generative-seller. creative-ad-server and governance-delivery-monitor keep their literals — those storyboards reference prerequisite media buys (Pattern A / fixtures: block) with no creating step to capture from.

#2589 — two context-echo authoring bugs. Both storyboards asserted context.correlation_id on the response but did not send context: on the sample_request, relying on runner-synthesized context that #2593 now explicitly forbids.

  • specialisms/content-standards/index.yaml:calibrate_after_policy_change — sends context.correlation_id explicitly (assertion value was also stale — it pointed at the prior step's correlation id).
  • protocols/media-buy/scenarios/governance_denied_recovery.yaml:create_media_buy_denied — sends context: explicitly and tightens field_present: context to a field_value: context.correlation_id echo assertion. Exercises the echo-on-error contract directly (the step has expect_error: true).

#2597list_formats_integrity step on media_buy_seller. New step in product_discovery phase between get_products_brief and create_buy. Filters list_creative_formats by format_ids: ["$context.product_format_id"] (captured FormatID object from get_products) and asserts:

  • formats[0] resolves (stale/typo'd format_id → empty array → fail).
  • formats[0].format_id equals $context.product_format_id verbatim (agent cannot substitute a different format under the filter).

Implements the issue's Option B (baseline check for every seller agent). Cross-agent dispatch — calling the creative agent at format_ids[].agent_url directly rather than the sales agent under test — is a runner follow-up: for single-agent sellers the current step is an integrity check; for true cross-agent sellers the sales agent either proxies the lookup or fails with an empty result, which still catches the failure mode the issue describes.

Review addressed

  • ad-tech-protocol-expert + docs-expert: narrative/expected on new step no longer claims "asks the creative agent" (dispatch is to the sales agent under test); dropped redundant field_present checks on schema-required subfields; added field_value equality on the round-tripped FormatID so the narrative claim is actually enforced by a validation (previously only field_present: formats[0]).
  • code-reviewer: capture shape verified end-to-end (namekey alias in loader.js:31, explicit outputs applied after convention extractors in runner.js:842). $context.<name> substitution confirmed to walk nested objects and array elements and resolve to captured values including FormatID objects. test:storyboard-scoping, test:error-codes, test:schemas, test:examples, test:composed, test:unit, typecheck, build:compliance all green. Kept explicit context_outputs: on create_media_buy (even though the convention extractor at context.js:63 produces the same key) to match the Pattern B example in docs/contributing/storyboard-authoring.md — authoring-doc consistency over terseness.
  • security-reviewer: no Must-Fix or Should-Fix findings. $context.<k> substitution is anchored whole-string regex (no prompt-injection surface). Context accumulator is run-scoped via a per-run object identity (no cross-run leakage). Correlation_id values are non-secret storyboard identifiers already committed to the repo.

Test plan

  • npm run test:storyboard-scoping — 3/3 pass
  • npm run test:error-codes — clean (0 errors, 1 pre-existing YAML-parse warning on runner-output-contract.yaml)
  • npm run test:schemas — 7/7 pass
  • npm run test:examples — 31/31 pass
  • npm run test:composed — 15/15 pass
  • npm run test:unit — 627/627 pass
  • npm run typecheck — clean
  • npm run build:compliance — clean
  • Runner-integration verification in @adcp/client against the training agent (follow-up; this repo has no runner-integration test suite)
  • Follow-up issue for cross-agent list_creative_formats dispatch in list_formats_integrity

🤖 Generated with Claude Code

bokelley and others added 2 commits April 20, 2026 19:47
…at_id integrity step

Closes #2585, #2589, #2597 at the storyboard-YAML layer.

- Seven storyboards now capture media_buy_id via context_outputs and
  reference it via $context.media_buy_id in downstream steps, replacing
  hardcoded mb_* literals that only resolved under the prior
  scenario-seeded fixtures.
- Two context-echo authoring bugs fixed: validators asserted on
  context.* but sample_request sent none, relying on runner synthesis
  the spec now forbids.
- New list_formats_integrity step on media_buy_seller catches stale
  format_ids[] references at discovery rather than silently at
  sync_creatives, and asserts the format_id round-trips verbatim
  through the list_creative_formats filter.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…correlation_id fix

Address three pre-merge observations on PR #2600:

1. Changeset flags the content-standards correlation_id correction as a
   spec-aligned tightening, not a new requirement — agents that were
   "passing" by echoing the stale prior-step id will now correctly fail.

2. governance_denied_recovery:create_media_buy_retry now declares
   context_outputs on the successful retry, so a future phase asserting
   pacing/delivery on the retried buy can consume $context.media_buy_id
   without re-discovering the capture.

3. media-buy/index.yaml verified — create_media_buy captures
   media_buy_id upstream of every downstream consumer (check_buy_status
   and get_delivery across distinct phases). No change needed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@bokelley bokelley merged commit 546a15d into main Apr 21, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Spec: storyboard fixtures block / pattern — replace hardcoded fixture IDs

1 participant