From 9951a30785cde609273c9989231c3a7fef4cdd93 Mon Sep 17 00:00:00 2001 From: Justin McLean Date: Fri, 19 Jun 2026 23:52:13 +1000 Subject: [PATCH] docs(spec): correct stale eval-coverage Known Gap in meta-and-quality-tooling The Known Gaps section claimed "~15 suites, coverage incomplete". All 44 shipped skills now have a matching suite in tools/skill-evals/evals/; the soft eval-coverage check (check #8) in skill-and-tool-validator enforces this going forward. Remove the stale back-fill claim and state the actual coverage. Generated-by: Claude (Opus 4.7) --- tools/spec-loop/specs/meta-and-quality-tooling.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/tools/spec-loop/specs/meta-and-quality-tooling.md b/tools/spec-loop/specs/meta-and-quality-tooling.md index 9f4b0823..fd15bd79 100644 --- a/tools/spec-loop/specs/meta-and-quality-tooling.md +++ b/tools/spec-loop/specs/meta-and-quality-tooling.md @@ -82,9 +82,10 @@ uv run --project tools/skill-and-tool-validator --group dev skill-and-tool-valid ## Known gaps -- **Eval coverage is incomplete** — the harness has ~15 suites but the - repo has more skills than that; skills added before the per-skill-eval - convention have no suite. Back-filling one suite per uncovered skill is - a tracked work item. -- Other gaps appear as new quality checks worth adding (e.g. a spec - validator analogous to the skill validator) — recorded by the plan pass. +- **Eval coverage is complete.** All 44 shipped skills have a matching + suite in `tools/skill-evals/evals/`; the soft eval-coverage check in + `skill-and-tool-validator` (check #8) warns when a newly added skill + has no suite, keeping coverage complete going forward. +- Other gaps appear as new quality checks worth adding — recorded by the + plan pass. The spec validator (analogous to the skill validator) and + the ASF-coupling advisory lint are two recent additions to this surface.