diff --git a/tools/skill-evals/evals/setup-override-upstream/step-6-pr-confirm/fixtures/case-1-shows-all-sections/report.md b/tools/skill-evals/evals/setup-override-upstream/step-6-pr-confirm/fixtures/case-1-shows-all-sections/report.md index 67048e7d..79fe3c21 100644 --- a/tools/skill-evals/evals/setup-override-upstream/step-6-pr-confirm/fixtures/case-1-shows-all-sections/report.md +++ b/tools/skill-evals/evals/setup-override-upstream/step-6-pr-confirm/fixtures/case-1-shows-all-sections/report.md @@ -22,7 +22,7 @@ the gate on upgrade. Adopters who prefer the old behaviour (triage on draft PRs) can opt out by setting `skip_draft_prs: false` in their project config. ## Test plan -- [ ] Ran `skill-validate` — passes. +- [ ] Ran `skill-and-tool-validate` — passes. - [ ] Manually tested against a draft PR in the adopter repo before opening this PR. - [ ] Verified triage runs normally on non-draft PRs after the change. diff --git a/tools/spec-loop/AGENTS.md b/tools/spec-loop/AGENTS.md index 1c4b7761..03d96c0a 100644 --- a/tools/spec-loop/AGENTS.md +++ b/tools/spec-loop/AGENTS.md @@ -28,7 +28,7 @@ Run the spec's own **Validation** block first. General checks: ```bash # Validate skill definitions (frontmatter, links, placeholders) -uv run --project tools/skill-validator --group dev skill-validate +uv run --project tools/skill-and-tool-validator --group dev skill-and-tool-validate # A skill's behavioural eval suite (every skill must have one) uv run --project tools/skill-evals skill-eval tools/skill-evals/evals// diff --git a/tools/spec-loop/specs/adoption-and-setup.md b/tools/spec-loop/specs/adoption-and-setup.md index 7bb0510c..2622afbc 100644 --- a/tools/spec-loop/specs/adoption-and-setup.md +++ b/tools/spec-loop/specs/adoption-and-setup.md @@ -68,7 +68,7 @@ gitignored skill symlinks, and committed agent-readable override files. ```bash test -f docs/setup/README.md -uv run --project tools/skill-validator --group dev skill-validate +uv run --project tools/skill-and-tool-validator --group dev skill-and-tool-validate ``` ## Known gaps diff --git a/tools/spec-loop/specs/drafting-mode.md b/tools/spec-loop/specs/drafting-mode.md index 8c183898..7c12981d 100644 --- a/tools/spec-loop/specs/drafting-mode.md +++ b/tools/spec-loop/specs/drafting-mode.md @@ -59,12 +59,12 @@ merges its own work. 1. No drafting skill merges or force-pushes. 2. Security drafts pass the confidentiality scrub before any public write. -3. `skill-validate` passes on the drafting-family skills. +3. `skill-and-tool-validate` passes on the drafting-family skills. ## Validation ```bash -uv run --project tools/skill-validator --group dev skill-validate +uv run --project tools/skill-and-tool-validator --group dev skill-and-tool-validate ``` ## Known gaps diff --git a/tools/spec-loop/specs/mentoring-mode.md b/tools/spec-loop/specs/mentoring-mode.md index 626919f9..7b2eb624 100644 --- a/tools/spec-loop/specs/mentoring-mode.md +++ b/tools/spec-loop/specs/mentoring-mode.md @@ -65,7 +65,7 @@ contributor-empowerment lever the wider ecosystem most needs. ```bash test -f docs/mentoring/spec.md -uv run --project tools/skill-validator --group dev skill-validate +uv run --project tools/skill-and-tool-validator --group dev skill-and-tool-validate ``` ## Known gaps diff --git a/tools/spec-loop/specs/meta-and-quality-tooling.md b/tools/spec-loop/specs/meta-and-quality-tooling.md index f3a9fd8d..99741956 100644 --- a/tools/spec-loop/specs/meta-and-quality-tooling.md +++ b/tools/spec-loop/specs/meta-and-quality-tooling.md @@ -8,7 +8,7 @@ kind: feature mode: infra source: > README.md § Skill families (utilities) and AGENTS.md § Reusable skills. - Implemented by tools/skill-validator/, tools/skill-evals/, + Implemented by tools/skill-and-tool-validator/, tools/skill-evals/, tools/sandbox-lint/, tools/dashboard-generator/, tools/probe-templates/, and the write-skill / list-steward-skills utility skills. acceptance: @@ -31,9 +31,9 @@ trustworthy as it grows. ## Where it lives -- `tools/skill-validator/` — validates `SKILL.md` frontmatter (required - `name`, `description`, `license`), internal link integrity, and - placeholder conventions. CLI: `skill-validate`. +- `tools/skill-and-tool-validator/` — validates `SKILL.md` frontmatter (required + `name`, `description`, `license`) and tool definitions, internal link integrity, and + placeholder conventions. CLI: `skill-and-tool-validate`. - `tools/skill-evals/` — harness for measuring skill behaviour. - `tools/sandbox-lint/` — lints the sandbox/permissions configuration. - `tools/dashboard-generator/` — read-only HTML dashboards over campaign @@ -44,7 +44,7 @@ trustworthy as it grows. build iterations to mechanically select the next work item. - `tools/spec-validator/` — validates spec-loop spec frontmatter (required keys, valid `status`/`kind`/`mode` values, body-section - presence); the spec-side counterpart to `skill-validator`. + presence); the spec-side counterpart to `skill-and-tool-validator`. - Skills: `write-skill` (author/update a skill), `list-steward-skills` (live, generated index of every skill, grouped by family). @@ -53,7 +53,7 @@ trustworthy as it grows. - **Generated, never cached.** `list-steward-skills` reads the live `.claude/skills/*/SKILL.md` frontmatter on every run, so the index never goes stale. -- **Deterministic checks.** `skill-validator` and `sandbox-lint` are +- **Deterministic checks.** `skill-and-tool-validator` and `sandbox-lint` are heuristic/text tools with no model calls — reproducible in CI. - **Hard vs soft rules.** The validator fails on missing frontmatter or broken links; advisories are warnings unless `--strict`. @@ -66,15 +66,15 @@ trustworthy as it grows. ## Acceptance criteria -1. `skill-validate` enforces required frontmatter + link integrity. +1. `skill-and-tool-validate` enforces required frontmatter + link integrity. 2. `list-steward-skills` generates its index from live frontmatter. 3. Each meta tool ships with its own tests. ## Validation ```bash -uv run --project tools/skill-validator --group dev pytest -uv run --project tools/skill-validator --group dev skill-validate +uv run --project tools/skill-and-tool-validator --group dev pytest +uv run --project tools/skill-and-tool-validator --group dev skill-and-tool-validate ``` ## Known gaps diff --git a/tools/spec-loop/specs/pairing-mode.md b/tools/spec-loop/specs/pairing-mode.md index 8eb91118..da48e3f3 100644 --- a/tools/spec-loop/specs/pairing-mode.md +++ b/tools/spec-loop/specs/pairing-mode.md @@ -64,7 +64,7 @@ protecting the ASF contribution path (contributor → committer → PMC). ```bash ls .claude/skills/ | grep -q '^pairing-' && echo "pairing skill present" || echo "GAP: no pairing skill" -uv run --project tools/skill-validator --group dev skill-validate +uv run --project tools/skill-and-tool-validator --group dev skill-and-tool-validate ``` ## Known gaps diff --git a/tools/spec-loop/specs/security-issue-lifecycle.md b/tools/spec-loop/specs/security-issue-lifecycle.md index ee131867..3c2d16a6 100644 --- a/tools/spec-loop/specs/security-issue-lifecycle.md +++ b/tools/spec-loop/specs/security-issue-lifecycle.md @@ -66,7 +66,7 @@ publication, with a human gate and an audit-log entry at every step. ## Validation ```bash -uv run --project tools/skill-validator --group dev skill-validate +uv run --project tools/skill-and-tool-validator --group dev skill-and-tool-validate uv run --project tools/vulnogram/generate-cve-json --group dev pytest ``` diff --git a/tools/spec-loop/specs/triage-mode.md b/tools/spec-loop/specs/triage-mode.md index 8c51684f..1c469993 100644 --- a/tools/spec-loop/specs/triage-mode.md +++ b/tools/spec-loop/specs/triage-mode.md @@ -65,13 +65,13 @@ suggestion the human signs off on. ## Acceptance criteria 1. No triage skill performs an unconfirmed state change. -2. `skill-validate` passes on all triage-family skills. +2. `skill-and-tool-validate` passes on all triage-family skills. 3. docs/modes.md Triage table matches the shipped skill set. ## Validation ```bash -uv run --project tools/skill-validator --group dev skill-validate +uv run --project tools/skill-and-tool-validator --group dev skill-and-tool-validate ``` ## Known gaps