Skip to content

fix(compliance): add sample_request to webhook-emission trigger steps (#2758) - #2779

Merged
bokelley merged 2 commits into
mainfrom
bokelley/issue-2758
Apr 22, 2026
Merged

fix(compliance): add sample_request to webhook-emission trigger steps (#2758)#2779
bokelley merged 2 commits into
mainfrom
bokelley/issue-2758

Conversation

@bokelley

Copy link
Copy Markdown
Contributor

Summary

  • Adds a minimal sample_request block to all three trigger steps in universal/webhook-emission.yaml (trigger_webhook_operation, trigger_retry_scenario, trigger_signed_webhook).
  • Each block carries push_notification_config.url = {{runner.webhook_url:<step_id>}} (no authentication, so the 9421 baseline is in effect), a generated idempotency_key (keeps the trigger call itself retry-safe), and a context.correlation_id.
  • Operation-specific required fields (brand, packages, budget for create_media_buy etc.) remain the responsibility of whichever test_kit resolves $test_kit.operations.primary_webhook_emitter.

Why

The three steps described passing push_notification_config.url = {{runner.webhook_url:<step_id>}} in prose but had no sample_request. The runner's mustache expander only substitutes placeholders that appear in a step's sample_request, so no webhook URL ever reached the agent under test — expect_webhook_presence, expect_key_stable_across_retries, and expect_signature_valid all timed out with "webhook never arrived" across every webhook-emitting agent. All three phases failed vacuously for the same bug; fixing only trigger_webhook_operation (as the issue proposed) would leave the other two phases broken under the same pattern.

The lint from #2768 skips $test_kit.* schema_refs, so the new blocks don't need an allowlist entry.

Closes #2758.

Test plan

  • npm run test:storyboard-sample-request-schema — passes (no new drift)
  • npm run test:storyboard-scoping, branch-sets, contradictions, context-entity, auth-shape, test-kits — pass
  • npm run test:unit — 631 passed
  • npm run typecheck — clean
  • Runner rerun against training agent: three webhook-emission phases advance past "webhook never arrived" (needs runner environment, not available locally)

🤖 Generated with Claude Code

bokelley and others added 2 commits April 21, 2026 21:07
…#2758)

The three trigger steps in universal/webhook-emission.yaml described passing
push_notification_config.url={{runner.webhook_url:<step_id>}} in their
narratives but had no sample_request block. The runner only substitutes
placeholders that appear in sample_request, so no webhook URL reached the
agent — every expect_webhook step then timed out with "webhook never
arrived" and the three failing phases blocked all webhook-emitting agents.

Each trigger now carries a minimal operation-agnostic sample_request;
operation-specific required fields come from whichever test_kit resolves
primary_webhook_emitter.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…hook_operation

Reviewer feedback: the extra paragraph explaining why push_notification_config
and idempotency_key are the two universal fields diverged from the parallel
`expected` blocks on trigger_retry_scenario and trigger_signed_webhook. The
narrative above each step already explains the 9421-baseline point.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@bokelley
bokelley merged commit e694e0f into main Apr 22, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

webhook_emission storyboard: trigger_webhook_operation missing sample_request with push_notification_config

1 participant