Skip to content

ci(storyboards): skip two SDK-blocked steps on /sales, ratchet floor#4069

Merged
bokelley merged 1 commit into
mainfrom
bokelley/sales-known-failing-skips
May 4, 2026
Merged

ci(storyboards): skip two SDK-blocked steps on /sales, ratchet floor#4069
bokelley merged 1 commit into
mainfrom
bokelley/sales-known-failing-skips

Conversation

@bokelley

@bokelley bokelley commented May 4, 2026

Copy link
Copy Markdown
Contributor

Summary

Two storyboard steps on /sales fail on framework-routed sellers due to SDK gaps. Adding both to KNOWN_FAILING_STEPS with linked source-of-truth issues so the runner reports them as skipped rather than failed; ratcheting the /sales floor to capture the lift.

Step 1: media_buy_seller/create_media_buy_async/create_media_buy_submitted — adcp-client#1554

The v5 createMediaBuy handler returns a hand-rolled { status: 'submitted', task_id } envelope when the force_create_media_buy_arm directive is set. The v6 framework's projector (from-platform.js:1438) rejects that shape — the only path into the submitted arm is ctx.handoffToTask(fn), which assigns a framework-issued task_id. The test-controller directive requires the seller to return the caller-supplied task_id (so the runner can assert against forced.task_id), which handoffToTask cannot satisfy until the SDK exposes a { task_id } overload.

Step 2: media_buy_seller/vendor_metric_accountability/get_delivery_with_vendor_metrics — adcp-client#1552

The SDK runner drops extension params (e.g. vendor_metric_values) from comply_test_controller requests before they hit the wire. The agent's simulate_delivery handler never sees the array, so the downstream delivery assertion finds nothing on by_package[].vendor_metric_values. Pure runner gap — the spec declares params as passthrough.

Floor lift on /sales:

Tenant Old New Delta
/sales 65 / 252 67 / 252 +2 / 0

Both skipped storyboards (create_media_buy_async, vendor_metric_accountability) now grade clean — only one step is skipped per storyboard; the rest pass.

Other tenants flat — these storyboards are sales-only.

Test plan

  • Local matrix run — all six tenants pass new floors
  • CI matrix run on PR
  • Once adcp-client#1552 + adcp-client#1554 ship: drop the skip entries, ratchet floor again

🤖 Generated with Claude Code

create_media_buy_submitted step blocked on adcp-client#1554 (handoffToTask
needs caller-supplied task_id overload). get_delivery_with_vendor_metrics
blocked on adcp-client#1552 (runner drops vendor_metric_values from
comply params). Both added to KNOWN_FAILING_STEPS with linked issues.
Sales floor 65→67 clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@bokelley bokelley merged commit 6d6eae7 into main May 4, 2026
19 checks passed
@bokelley bokelley deleted the bokelley/sales-known-failing-skips branch May 4, 2026 13:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant