compliance(storyboards): richer env fingerprint + requires_scenarios resolution#2679
Merged
Merged
Conversation
…resolution Closes #2670 part 1 and #2671. Env fingerprint (#2670 part 1): - fingerprintEnv now includes doc.prerequisites.test_kit and a hash of top-level doc.fixtures. Two storyboards sharing id + comply_scenario but loading different test kits or seeding different prerequisite state are now discriminated instead of collapsing into one bucket. - Non-breaking: no change to current storyboards' contradiction outcomes. Removal of sb=<doc.id> (so cross-storyboard contradictions surface) remains tracked at #2670. Orphan-contribution: requires_scenarios resolution (#2671): - buildScenarioFlagIndex walks the source tree once, producing Map<doc.id, Set<flag>> of asserted flags per storyboard/scenario. - lintDoc accepts a scenarioFlagIndex option. When the parent doc declares requires_scenarios, the linked scenarios' asserted flags union into the parent's own before the orphan check. A parent that delegates grading to a shared scenario no longer false-positives. - Duplicate doc.id across files throws: order-dependent resolution could silently mask a real orphan elsewhere, so collisions surface at lint time. - Unresolved scenario ids fall through (scoping lint concern); non- string requires_scenarios entries are skipped defensively. Tests: 21 contradiction tests (2 new env-discrimination cases for test_kit and fixtures), 19 branch-set tests (4 new: requires_scenarios resolution, unresolved-scenario fallback, duplicate-id throw, non-string entry skip). Build: 36/56 clean / 295 passed — unchanged from main. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced Apr 21, 2026
5 tasks
bokelley
added a commit
that referenced
this pull request
Apr 21, 2026
…ios spec Two hygiene follow-ups from #2679 expert review. Fixture hash normalization (#2682): - normalizeFixturesForHashing sorts arrays within each documented fixture category by its primary id before stableStringify. - Categories + primary ids: products/product_id, pricing_options/ pricing_option_id, creatives/creative_id, plans/plan_id, media_buys/media_buy_id. - Unknown fixture category throws: schema-doc and lint updates must land together so a new seed category can't silently create a false-negative env-fingerprint bucket. - Closes false-negative envelope before #2670 part 2 removes sb=<doc.id> and makes env-fingerprint precision load-bearing. Spec: flag flow across requires_scenarios (#2683): - storyboard-schema.yaml now normatively describes how branch_set contribution flags flow between parent and linked scenarios. - Parent-flag flow: contributed in X, asserted in X or in any scenario Y listed in X.requires_scenarios. - Reverse flow prohibited — scenarios MUST be internally self- grading to preserve standalone lintability and prevent cycles in the composition graph. - Scenario IDs must match the referenced file's top-level id: and the file MUST exist at build time. Tests: contradiction suite 25 (3 new — array-order stability, content-difference still discriminated, unknown category throws + documented categories round-trip). Follow-up #2687 filed for unresolved_scenario_reference grading (symmetric with the duplicate-id throw already in place). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced Apr 21, 2026
bokelley
added a commit
that referenced
this pull request
Apr 21, 2026
… fingerprint (#2670 pt2) (#2714) * compliance(storyboards): drop sb=<doc.id> from contradiction lint env fingerprint (#2670 part 2) Including the storyboard id was conservative-by-design during the lint's initial rollout (#2661) but suppressed the exact class of bug the lint was built to catch — cross-storyboard contradictions (#2627, #2628, #2629). Prerequisites for the removal are now in place: #2679 added test_kit + top-level fixtures precision, PR #2710 (from #2684) audited principal identity and added role= as the forward guard, and #2708 tracks the remaining auth= structural gap. Experiment on the current 56-storyboard suite: removing sb= surfaces 10 multi-member groups (7 spanning ≥2 files), including 25 files agreeing that get_adcp_capabilities returns success on acme-outdoor and 6 doing the same on nova-motors. Zero contradictions — the suite is genuinely consistent and the lint now exercises cross-storyboard checks as originally intended. Inverts one test (no-contradiction-when-ids-differ → contradictions- surface-when-ids-differ-but-env-matches) to pin the new behavior, adds a complementary negative test (different test_kit + different ids → no contradiction) to pin test_kit= as a cross-file separator now that sb= is gone, and updates the `test_kit discriminates env` comment to note test_kit= is its sole discriminator post-removal. Protocol review surfaced a residual hole that sb= removal exposes: steps inheriting the transport's default auth emit no auth= component, so two storyboards with different transport defaults could collide. Out of scope here; tracked as #2711. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * compliance(storyboards): resolved-auth fingerprint shape (#2708 + #2711) (#2723) Extracts auth emission into a describeStepAuth() helper that always returns a stable token. Fixes two structural gaps that removing sb=<doc.id> (#2670 part 2) exposed: #2711 — inherited-default steps (336 of 340 in the current suite) used to emit no auth= component, so they could collide silently with explicit steps authored against divergent transport defaults. Now emits auth=kit_default. #2708 — auth.from_test_kit was read as boolean regardless of whether it was true or a string path, collapsing distinct principals in multi-principal kits. Now emits auth=<type>:from_test_kit:<path> for string form. Literal auth.value is sha1-8hex hashed so different literals don't collide (defense-in-depth; #2720 tracks flagging literals as a code smell). Zero grouping change on the current suite — no storyboard uses string-form from_test_kit or literal value; all 336 inheriting steps get the new kit_default token uniformly. Tests: - describeStepAuth shape-matrix unit test, every branch including defensive fallbacks. Literal hashes pinned to precomputed sha1 values for stability. - Inherited-default cross-storyboard test asserting kit_default emission and contradiction surface when outcomes disagree. - Multi-principal forward-guard test pinning that from_test_kit path selection discriminates, with direct fingerprintEnv comparison asserting the path token appears verbatim. Protocol review surfaced one structural question deferred as #2721: runner test kits (webhook-receiver, substitution-observer, signed-requests) have no auth: block — if a storyboard ever composes runner + primary kit, kit_default becomes ambiguous. No storyboard does that today. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Bundles two follow-ups filed after #2676:
Both are precision-adding, non-breaking changes. Contradiction counts on the current source tree are unchanged (36/56 clean, 295 passed).
Env fingerprint (#2670 part 1)
Before:
```
sb=<doc.id>|scenario=<comply_scenario>|auth=type:strat|seed=<controller_seeding>
```
After:
```
sb=<doc.id>|test_kit=<prerequisites.test_kit>|fixtures=|scenario=<comply_scenario>|auth=type:strat|seed=<controller_seeding>
```
Covers two legitimate "same id+scenario, different outcome" cases:
The `sb=<doc.id>` component stays in place — removing it (so cross-storyboard contradictions surface) remains tracked as part 2 of #2670 with triage time to match.
Orphan contribution + requires_scenarios (#2671)
`buildScenarioFlagIndex` walks the source tree once and produces `Map<doc.id, Set>` keyed on every storyboard/scenario file's top-level `id:`. `lintDoc` accepts a `scenarioFlagIndex` option; when a parent doc declares `requires_scenarios:`, the linked scenarios' asserted flags union into the parent's own before the orphan check runs.
Duplicate `doc.id` across files throws — order-dependent resolution of a collision could silently mask a real orphan elsewhere, so we surface the duplicate at lint time rather than papering over it.
Unresolved scenario ids fall through silently (scoping-lint concern per the issue); non-string `requires_scenarios` entries are skipped defensively.
Tests
Expert review
One round pre-merge:
🤖 Generated with Claude Code