Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 7 additions & 6 deletions tools/spec-loop/specs/meta-and-quality-tooling.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,9 +82,10 @@ uv run --project tools/skill-and-tool-validator --group dev skill-and-tool-valid

## Known gaps

- **Eval coverage is incomplete** — the harness has ~15 suites but the
repo has more skills than that; skills added before the per-skill-eval
convention have no suite. Back-filling one suite per uncovered skill is
a tracked work item.
- Other gaps appear as new quality checks worth adding (e.g. a spec
validator analogous to the skill validator) — recorded by the plan pass.
- **Eval coverage is complete.** All 44 shipped skills have a matching
suite in `tools/skill-evals/evals/`; the soft eval-coverage check in
`skill-and-tool-validator` (check #8) warns when a newly added skill
has no suite, keeping coverage complete going forward.
- Other gaps appear as new quality checks worth adding — recorded by the
plan pass. The spec validator (analogous to the skill validator) and
the ASF-coupling advisory lint are two recent additions to this surface.