spec(compliance): first cross-step assertions on universal/idempotency (#2639)#2663
Merged
Conversation
#2639) Wires two programmatic gates to the universal idempotency storyboard via the new `invariants: [...]` YAML field (shipped in @adcp/client 5.8+): - idempotency.conflict_no_payload_leak — IDEMPOTENCY_CONFLICT error bodies must contain only allowlisted envelope fields. Leaking cached payload fingerprints (budget, start_time, cached_payload, etc.) turns stolen idempotency keys into a read oracle. Converts the key_reuse_conflict phase's reviewer_check into runtime enforcement. - context.no_secret_echo — no response on any step may echo bearer literals, the test-kit's declared api_key, or suspect field names (Authorization, api_key, bearer, x-api-key) at any depth. Modules live in server/src/compliance/assertions/ and register via @adcp/client/testing's registry at import time. Runners must load them before calling runStoryboard (CLI --invariants flag, or direct import); the runner throws at start on unresolved ids. Bumps @adcp/client to ^5.8.1 for the registry re-exports. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI failed on the parent PR because adding `invariants: [...]` to universal/idempotency.yaml made the SDK runner throw at start on unresolved ids — no caller was loading the assertion modules. Wire them in at the choke points every storyboard-running flow already touches: - server/src/services/storyboards.ts — API / comply() flows go through this wrapper; importing the assertions as a side effect registers them before any downstream runStoryboard call. - server/tests/manual/run-storyboards.ts — the CI entry point; explicit import keeps the registration visible to the runner script itself. - server/tests/manual/run-one-storyboard.ts, storyboard-smoke.ts — standalone manual runners that don't touch the service wrapper. Node's module cache dedupes the actual registerAssertion calls, so the overlap is harmless. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced Apr 21, 2026
…oor rebaseline (#2639) Adds the third cross-step assertion per expert review: - After any denial signal (GOVERNANCE_DENIED, CAMPAIGN_SUSPENDED, PERMISSION_DENIED, POLICY_VIOLATION, TERMS_REJECTED, COMPLIANCE_UNSATISFIED, or check_governance status=denied) on a plan, no subsequent step may acquire a resource for that plan. Plan-scoped via plan_id; falls back to run-scoped for signals without plan linkage. - Denial is sticky within a run — a later passing check_governance does not clear state. Requiring a fresh governance_context is correct semantics; this assertion encodes within-run monotonicity. - Resource-acquired detection is task-allowlist (create_media_buy, update_media_buy, activate_signal, property/collection list CRUD, acquire_rights) + server-minted id fields. sync_* tasks deferred to follow-up (batch envelopes need schema-specific traversal). - plan_id extraction reads response body first, then recorded request payload. Deliberately does NOT fall back to accumulated context — stale plan_id from earlier steps would bind denials to wrong plans. Wired to governance-spend-authority (index + denied), governance-aware-seller, governance-delivery-monitor. Silent on runs with no denial signal. Tests: 41 passing including table-driven coverage of all 6 denial codes, negative cases for excluded codes (GOVERNANCE_UNAVAILABLE, ACCOUNT_SUSPENDED, CONFLICT, IDEMPOTENCY_CONFLICT), plan-scoped correctness, sticky state, onStart reset, nested error.code shapes. Rebases storyboard CI floors (legacy 35→27 clean, 279→271 passing; framework 21→19 clean, 237→226 passing) to match @adcp/client 5.8.1 baseline. The drop is from upstream validation tightening (schema-driven hooks #694, strict sync_creatives #661, refs_resolve cross-step #670, branch_set grading #693, PRM enforcement #677), not this assertion — the invariants: YAML additions are inert against the SDK-bundled compliance cache until @adcp/client re-publishes its tarball. Follow-up filed as adcp#2667 to diagnose the newly-failing storyboards. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
3 tasks
bokelley
added a commit
that referenced
this pull request
Apr 21, 2026
#2666 restored the training-agent responses that the @adcp/client 5.6→5.8.1 bump had broken (creative format fixtures, get_media_buys total_budget, validate_property_delivery shape) but didn't bump the CI non-regression floors. Raise the gate to the recovered baselines so future regressions actually fail CI: - legacy: 27→36 clean, 271→295 passing (pre-bump was 35/279) - framework: 19→21 clean, 226→241 passing (pre-bump was 21/237) Also: - .gitignore: exclude /dist/compliance/assertions/ from the tracked /dist/compliance/ tree so tsc output from server/src/compliance/** (the assertion modules landed in #2663) doesn't collide with the published spec-tarball path. - property-handlers.ts: comment explaining why violations[] was moved into the closed-shape features[] channel. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Ships the first two programmatic assertions for the cross-step invariant framework (adcp#2639 — follow-up to the adcp-client-side #692/#713). Both were previously reviewer-only checks in
universal/idempotency.yamlanduniversal/security.yaml; converting them to runtime gates means a seller who leaks cached state or echoes credentials on error fails the storyboard at run time rather than sliding past review.Wires the two assertions to
static/compliance/source/universal/idempotency.yamlvia the new top-levelinvariants: [...]field.What changed
static/compliance/source/universal/idempotency.yaml— addsinvariants: [idempotency.conflict_no_payload_leak, context.no_secret_echo]with a comment explaining how runners load the modules.server/src/compliance/assertions/context-no-secret-echo.ts— on every step response, walks the body and fails if it containsAuthorization: Bearer <token>literals, the verbatim value oftest_kit.auth.api_key(when ≥8 chars), or property names likeAuthorization/api_key/bearer/x-api-keyat any depth.server/src/compliance/assertions/idempotency-conflict-no-payload-leak.ts— on error steps whose code isIDEMPOTENCY_CONFLICT/CONFLICT, flags any property not in the short allowlist (code,message,status,retry_after,correlation_id,request_id,operation_id). Catchesbudget/start_time/nestedcached_payload/error.*shapes that a naive handler might serialize for "debugging".server/src/compliance/assertions/index.ts— import barrel that triggers registration at load time.server/tests/unit/compliance-assertions.test.ts— 14 unit tests covering both hooks: passes on clean responses, fails on each leak pattern, ignores short api_key placeholders, walks arrays, etc.package.json— bumps@adcp/clientto^5.8.1for theregisterAssertionre-exports from@adcp/client/testing.Why these two first
Each converts an existing reviewer check (text in YAML that humans were expected to eyeball) into a runtime gate. That's where the leverage is — the reviewer-check prose in
key_reuse_conflictexplicitly flags the stolen-key read oracle, and security.mdx warns against credential echo. Without runtime enforcement those pass silently.Deferred to follow-ups:
governance.denial_blocks_mutation— needs per-specialism design and coverage acrossgovernance-*storyboards.status.monotonic— ties into the lifecycle formalization work (Formalize creative status state machine and transition rules #1612-Formalize catalog item and creative approval state machines #1616).Loading modules
Runners that drive this storyboard must import the assertion modules before calling
runStoryboard. Either:or programmatically:
The runner fails fast at start on unresolved ids, so a misconfigured loader surfaces immediately rather than silently skipping conformance.
Test plan
npx vitest run server/tests/unit/compliance-assertions.test.ts— 14 passednpx vitest run server/tests/unit/{storyboards,compliance-assertions,collection-lists-storyboard}.test.ts— 29 passed, no regressions in storyboard wrapper testsnode scripts/build-compliance.cjs— clean,invariantsfield round-trips intodist/compliance/latest/universal/idempotency.yamltsc --project server/tsconfig.json --noEmit— clean (pre-commit hook runs it)Follow-ups
governance.denial_blocks_mutation— needs stepwise resource-acquisition tracking across governance storyboardsstatus.monotonic— after lifecycle formalization (Formalize creative status state machine and transition rules #1612-Formalize catalog item and creative approval state machines #1616) fixes per-resource status vocabularies@adcp/compliance-assertionspackage so third-party implementations can reuse the wiring instead of copying.Closes the "wire first assertions" leg of #2639.
🤖 Generated with Claude Code