feat(validator): add docs/modes.md consistency check (#11)#652
Merged
Conversation
Add validate_modes_doc_consistency (SOFT, category modes-doc-consistency) to the skill-and-tool-validator. The check compares docs/modes.md against live skills/*/SKILL.md frontmatter across four dimensions: 1. Listed skill missing from disk — row exists in modes.md but skills/<slug>/ does not. 2. Mode mismatch — a skill's mode: frontmatter differs from the section it appears in (e.g. listed under Triage but mode: Mentoring). 3. Skill count mismatch — the integer in "Modes at a glance" Skill count column does not equal the actual row count in that mode's section. 4. Unlisted skill — a live skill has a valid mode: frontmatter value but is absent from the corresponding docs/modes.md section. On the live tree the check surfaces two real gaps: - reviewer-routing has mode: Triage but is absent from docs/modes.md - good-first-issue-sweep has mode: Mentoring but is absent from docs/modes.md Ships 19 tests in TestParseModesDocs + TestValidateModeDocConsistency; all 283 validator tests pass. Generated-by: Claude (Opus 4.7)
38a5ba4 to
d13d6fc
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add validate_modes_doc_consistency (SOFT, category modes-doc-consistency) to the skill-and-tool-validator. The check compares docs/modes.md against live skills/*/SKILL.md frontmatter across four dimensions:
On the live tree the check surfaces two real gaps:
Ships 19 tests in TestParseModesDocs + TestValidateModeDocConsistency; all 283 validator tests pass.
Generated-by: Claude (Opus 4.7)
Type of change
.claude/skills/<name>/) — eval fixtures updated belowtools/<system>/*.md)tools/*/withpyproject.toml)docs/,README.md,CONTRIBUTING.md)projects/_template/)prek, workflows, validators)Test plan
prek run --all-filespassesuv run pytest/ruff check/mypypasses(
PYTHONPATH=tools/skill-evals/src python3 -m skill_evals.runner tools/skill-evals/evals/<skill>/)(a regression test for the bug fixed / the behaviour added — see CONTRIBUTING.md)