Skip to content

chore(compliance): lint universal-storyboard doc tables match filesystem#3102

Merged
bokelley merged 2 commits into
bokelley/compliance-catalog-universal-table-refreshfrom
bokelley/lint-universal-storyboard-doc-parity
Apr 25, 2026
Merged

chore(compliance): lint universal-storyboard doc tables match filesystem#3102
bokelley merged 2 commits into
bokelley/compliance-catalog-universal-table-refreshfrom
bokelley/lint-universal-storyboard-doc-parity

Conversation

@bokelley

Copy link
Copy Markdown
Contributor

Summary

Build-time + unit-test lint that prevents the drift #3099 just fixed from re-accumulating.

Stacked on #3099 (the doc refresh) so the lint sees an already-aligned tree. Once #3099 lands, this rebases to main automatically.

What it catches

  • Forward parity: a new universal storyboard ships without a doc-table row → specific "missing rows for X" error.
  • Reverse parity: a doc keeps a row for a renamed/deleted storyboard → "references X but no graded storyboard exists" error.
  • Either index page loses its "Universal" heading → "missing expected heading" error.

How it works

  • scripts/lint-universal-storyboard-doc-parity.cjs — new module exporting `lint({ sourceDir, repoRoot })` plus helpers, with a CLI entrypoint.
  • scripts/build-compliance.cjs — calls lint inside `generateIndex`, so the compliance build fails loudly on drift.
  • `tests/lint-universal-storyboard-doc-parity.test.cjs` — 10 tests: source-tree guard, clean fixture, non-graded fixtures filtered, forward parity (both docs), reverse parity (both docs), missing heading, helper unit tests.
  • `package.json` — `test:storyboard-doc-parity` wired into the umbrella `test` target alongside other storyboard lints; CI picks it up automatically.

"Graded" = YAML has a `phases:` array. Filters out the three non-graded fixtures (`storyboard-schema.yaml`, `runner-output-contract.yaml`, `fictional-entities.yaml`) which live alongside graded storyboards.

The two docs use different slug conventions by design (existing convention, not changed):

  • `conformance.mdx` uses snake_case YAML `id` values in markdown links
  • `compliance-catalog.mdx` uses kebab-case filename slugs in plain backticks

Test plan

  • `npm run test:storyboard-doc-parity` — 10/10 pass.
  • `node scripts/build-compliance.cjs` — passes against current docs (post-docs(conformance, catalog): refresh universal-storyboards tables #3099 refresh).
  • Sanity-checked drift detection: temporarily removed `webhook_emission` row → fired with clear error; injected ghost row → fired with reverse-parity error.
  • TypeScript precommit hook — passes.

Why `--empty` changeset

Build-script + tests + new lint logic — no protocol-spec surface change. The lint encodes existing convention (the two index pages should agree with `static/compliance/source/universal/`), it doesn't introduce a new normative rule.

🤖 Generated with Claude Code

…e filesystem

Prevents the drift #3099 just fixed from re-accumulating. Every graded
universal storyboard MUST appear in both docs/building/conformance.mdx
and docs/building/compliance-catalog.mdx; every backtick-quoted slug in
those tables MUST resolve to a real graded storyboard on disk.

Catches:
- New universal storyboard ships without a doc-table row (forward).
- Doc keeps a row for a renamed/deleted storyboard (reverse).
- Either index page loses its "Universal" heading.

Wiring:
- scripts/lint-universal-storyboard-doc-parity.cjs — new module exporting
  lint({ sourceDir, repoRoot }) plus helpers, with a CLI entrypoint.
- scripts/build-compliance.cjs — calls lint inside generateIndex, so the
  compliance build fails loudly on drift.
- tests/lint-universal-storyboard-doc-parity.test.cjs — 10 tests:
  source-tree guard, clean fixture, non-graded fixtures filtered out,
  forward parity (both docs), reverse parity (both docs), missing
  heading, helper unit tests.
- package.json — test:storyboard-doc-parity wired into the umbrella
  test target alongside other storyboard lints; CI picks it up
  automatically via npm run test.

"Graded" = YAML has a phases:[] array. Filters out the three non-graded
fixtures (storyboard-schema.yaml, runner-output-contract.yaml,
fictional-entities.yaml) that live alongside graded storyboards.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@bokelley bokelley added this to the 3.0.1 milestone Apr 25, 2026
Two cheap improvements from the DX-expert review:

1. Forward-parity error message now mentions the slug form (kebab-case
   filename slug for compliance-catalog, snake_case YAML id for
   conformance). Parallels the reverse error which already had this hint.
   Helps a contributor adding a brand-new storyboard pick the right
   form without comparing existing rows.

2. JSX comment above each universal-storyboards table pointing at the
   lint script. Discoverability for contributors editing the docs
   directly — they see the rule before the build fails.

Tests still pass 10/10; build clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@bokelley

Copy link
Copy Markdown
Contributor Author

Ran the lint by code-reviewer + dx-expert. Both said ship.

Code-reviewer: verified the regex/section-extraction edge cases hold up (trailing backtick anchors the slug capture; indexOf is fine for headings since a fenced ## Universal storyboards doesn't appear in either MDX); style matches existing scripts/lint-*.cjs neighbors; placement between verifyEnumParity and lintStoryboardIdempotency is correct ordering.

DX-expert: ship-with-nice-to-haves. Applied the two cheap ones in the follow-up commit:

  1. Forward-parity error now says "using the {YAML id | filename slug} form" — parallels the reverse error and helps a contributor adding a new storyboard pick the right form without grepping existing rows.
  2. JSX comment above each universal-storyboards table points at scripts/lint-universal-storyboard-doc-parity.cjs — discoverability for contributors editing the docs directly.

Skipped DX nice-to-have #3 (cross-form token detection — "oh, you put kebab-case in the snake-case doc") as more involved than the value justifies. The tokenLabel hint in the existing reverse-parity error already mitigates.

Tests still 10/10, build clean.

@bokelley bokelley merged commit 8a8ca11 into bokelley/compliance-catalog-universal-table-refresh Apr 25, 2026
4 checks passed
@bokelley bokelley deleted the bokelley/lint-universal-storyboard-doc-parity branch April 25, 2026 11:46
bokelley added a commit that referenced this pull request Apr 25, 2026
* docs(conformance, catalog): refresh universal-storyboards tables

Both docs/building/conformance.mdx and docs/building/compliance-catalog.mdx
had stale universal-storyboards tables. static/compliance/source/universal/
now contains 9 graded storyboards; the docs listed 5–7. Drift accumulated
as new storyboards landed without back-filling the index pages.

- Add webhook-emission (universal since #2417 / 3.0; never indexed)
- Add pagination-integrity (recently landed; missing from both)
- Add idempotency to compliance-catalog (had it in conformance.mdx only)
- Add signed-requests to compliance-catalog (had it in conformance.mdx only,
  added there in #3077)

Framing fix in compliance-catalog: the lead-in claimed every agent runs
every universal storyboard regardless of claims — true in scope, but
confusing on capability-gated rows. Reworded to acknowledge the gating
(deterministic-testing, signed-requests) plus a closing paragraph that
gated storyboards can't be partially implemented (advertise false rather
than ship a partial surface).

Both index pages now reflect the same 9 graded universal storyboards in
the same order, matching the published /compliance/{version}/universal/
directory.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(compliance): lint universal-storyboard doc tables match filesystem (#3102)

* chore(compliance): lint that universal-storyboard doc tables match the filesystem

Prevents the drift #3099 just fixed from re-accumulating. Every graded
universal storyboard MUST appear in both docs/building/conformance.mdx
and docs/building/compliance-catalog.mdx; every backtick-quoted slug in
those tables MUST resolve to a real graded storyboard on disk.

Catches:
- New universal storyboard ships without a doc-table row (forward).
- Doc keeps a row for a renamed/deleted storyboard (reverse).
- Either index page loses its "Universal" heading.

Wiring:
- scripts/lint-universal-storyboard-doc-parity.cjs — new module exporting
  lint({ sourceDir, repoRoot }) plus helpers, with a CLI entrypoint.
- scripts/build-compliance.cjs — calls lint inside generateIndex, so the
  compliance build fails loudly on drift.
- tests/lint-universal-storyboard-doc-parity.test.cjs — 10 tests:
  source-tree guard, clean fixture, non-graded fixtures filtered out,
  forward parity (both docs), reverse parity (both docs), missing
  heading, helper unit tests.
- package.json — test:storyboard-doc-parity wired into the umbrella
  test target alongside other storyboard lints; CI picks it up
  automatically via npm run test.

"Graded" = YAML has a phases:[] array. Filters out the three non-graded
fixtures (storyboard-schema.yaml, runner-output-contract.yaml,
fictional-entities.yaml) that live alongside graded storyboards.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(compliance): apply DX-review nice-to-haves to doc-parity lint

Two cheap improvements from the DX-expert review:

1. Forward-parity error message now mentions the slug form (kebab-case
   filename slug for compliance-catalog, snake_case YAML id for
   conformance). Parallels the reverse error which already had this hint.
   Helps a contributor adding a brand-new storyboard pick the right
   form without comparing existing rows.

2. JSX comment above each universal-storyboards table pointing at the
   lint script. Discoverability for contributors editing the docs
   directly — they see the rule before the build fails.

Tests still pass 10/10; build clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant