Skip to content

feat(cli): --invariants flag to register storyboard assertion modules#697

Merged
bokelley merged 5 commits into
mainfrom
bokelley/cli-invariants-flag
Apr 21, 2026
Merged

feat(cli): --invariants flag to register storyboard assertion modules#697
bokelley merged 5 commits into
mainfrom
bokelley/cli-invariants-flag

Conversation

@bokelley

Copy link
Copy Markdown
Contributor

Summary

Follow-up to #692 (storyboard assertion registry). Adds a CLI surface so operators can dynamic-import assertion modules before the runner resolves storyboard.invariants: [...].

adcp storyboard run ... --invariants ./my-invariants.js,@adcp/invariants imports each specifier at CLI entry, triggering each module's registerAssertion(...) calls. Without this, storyboards that declare invariants would fail at runner start with "Storyboard references unregistered assertion" because nothing is wired to populate the registry.

The framework intentionally keeps concrete assertions out of @adcp/client (they ship from adcontextprotocol/adcp alongside the specialisms that own them). This flag is the missing hook for that to happen.

Shape

adcp storyboard run test-mcp sales-non-guaranteed \
  --invariants ./my-invariants.js,@adcp/invariants
  • Comma-separated list; relative paths resolve against cwd; bare specifiers pass through as npm packages; path.isAbsolute for Windows parity.
  • Runs before the --dry-run gate so bad specifiers surface during preview, not after auth/agent resolution.
  • Wired at all three storyboard entry points: handleStoryboardRun, handleMultiInstanceStoryboardRun, and the comply path through runFullAssessment.
  • Filtered from positionalArgs so module specifiers aren't mistaken for agent aliases.

Test plan

  • Single module loads and registers at import time (marker-file verified)
  • Comma-separated multi-module load
  • Missing value → exit 2 with clear message
  • Broken module (throws at import) → exit 2 with error surfaced
  • Missing path → exit 2 with error surfaced
  • Positional filter doesn't confuse specifier with agent arg
  • Whitespace-only value (\" , , \") → exit 2, not silent no-op
  • Multi-instance path (--url ... --url ...) also loads modules
  • 44 neighbor tests (registry, webhook-receiver, file-flag, multi-instance) still green

Related

🤖 Generated with Claude Code

bokelley and others added 2 commits April 21, 2026 07:22
Adds `adcp storyboard run ... --invariants <module[,module...]>` so
operators can dynamic-import assertion modules before `runStoryboard`
resolves `storyboard.invariants`. Without this the registry from
adcp#2639 has no external surface — YAMLs referencing invariant ids
would fail at runner start because nothing ever called
registerAssertion().

Wired at all three storyboard entry points (single-run, multi-instance,
full-assessment/comply) and runs before the --dry-run gate so bad
specifiers surface in preview. Relative paths resolve against cwd via
pathToFileURL; bare specifiers pass through to dynamic import as
packages. Uses path.isAbsolute for Windows parity.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@bokelley

Copy link
Copy Markdown
Contributor Author

CI "Test & Build" failure is pre-existing on main: same 2 failures in test/lib/storyboard-completeness.test.js (expect_substitution_safe has a registered response schema and storyboard: sales_catalog_driven). Both tasks are newly added upstream in the compliance tarball and the SDK hasn't caught up; unrelated to this PR. Verified locally by running the test on main.

bokelley and others added 3 commits April 21, 2026 07:31
Upstream adcp compliance cache added `expect_substitution_safe` as a
pseudo-task used by the sales-catalog-driven storyboard. Like the
existing `expect_webhook*` family, it's a runner-observed assertion
(delegates to the SubstitutionObserver primitive), not an agent tool,
so it has no request builder, sample_request, or response schema.

Pre-existing failure on main that was blocking CI across PRs. Runtime
support for actually executing the step will land separately.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Conflicts resolved:
- test/lib/storyboard-completeness.test.js: both sides added
  `expect_substitution_safe` to HARNESS_TASKS with different comment
  wording. Took main's wording since it landed first (PR #698).
- src/lib/types/core.generated.ts, schemas.generated.ts: took main's
  version and re-ran sync-schemas + generate-types to confirm no drift.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant