Description
When an orchestrated development session (/orchestrate-dev) adds, removes, or renames user-facing surface (CLI flags, commands, API endpoints, configuration keys, environment variables), the generated plan's acceptance criteria do not require corresponding documentation updates. This means documentation artifacts — READMEs, help text, usage examples — can silently go stale.
Observed failure: In williamthorsen/workshop#38, the --github and --local flags were replaced by --from, but neither the showListHelp prompt nor the package README were updated. The coder, all reviewers, and the holistic reviewer missed it. A simple search for the removed flags would have caught it instantly.
Root cause: The planner never told the coder that documentation was part of the job. The coder's file-scope constraint ("only modify files that are part of the plan or directly required by it") then prevented it from touching unlisted files. Downstream reviewers verify acceptance criteria — but criteria that were never written can't be verified.
Why the existing pipeline didn't catch it: The review criteria include "Contracts: documented behaviors match implementations" and the holistic reviewer checks "Completeness," but neither instructs the agent to search for stale references. The failure is upstream: acceptance criteria are the mechanism that flows through the entire pipeline (planner → coder → reviewer), so the fix belongs there.
Proposed solution
Add a documentation coverage convention to all plan-producing skills, parallel to the existing test coverage convention. When a plan step adds, removes, or renames user-facing surface, its acceptance criteria must include corresponding updates to documentation, help text, and usage examples — including removal of references to anything that no longer exists.
Files to modify
Tier 1 — Plan producers (add documentation criterion to acceptance criteria guidance):
| File |
Location |
Pattern to follow |
~/.claude/agents/orchestrated-planner.md |
"Step design principles" (line 43) |
Add a sibling bullet after the test coverage bullet |
~/.claude/agents/planner.md |
"Step design principles" (lines 34–41) |
Add a documentation bullet (note: this file is also missing the test coverage bullet that orchestrated-planner has — add both) |
~/.claude/skills/design-and-plan/SKILL.md |
Lines 104 and 154 |
Add a "Documentation criterion convention" parallel to the "Test criterion convention," at both the ticket level and the per-task level |
Tier 2 — Plan validator (add documentation check):
| File |
Location |
Pattern to follow |
~/.claude/agents/plan-reviewer.md |
Completeness criteria table (lines 50–56) |
Add a "Documentation coverage" row |
Wording guidance
The new bullet/convention should follow this structure (adapt to each file's style):
When a step adds, removes, or renames user-facing surface (CLI flags, commands, API endpoints, configuration keys, environment variables), its acceptance criteria must include corresponding updates to documentation, help text, and usage examples — including removal of references to anything that no longer exists.
The key clause is "including removal of references to anything that no longer exists" — this directly addresses the PR #38 failure mode where new features were documented but old features weren't cleaned up.
Out of scope
superpowers:writing-plans — lives in a plugin cache and follows a different structure (TDD-focused steps without explicit acceptance criteria). A self-review checklist item could be added separately.
- Adding special-case instructions to the coder or reviewer agents — the acceptance criteria mechanism already flows through both; strengthening the source is sufficient.
- A dedicated
aspect-documentation-reviewer — premature for what is a one-line addition to existing plan-producing skills.
Acceptance criteria
Must have
Should have
Nice to have
Description
When an orchestrated development session (
/orchestrate-dev) adds, removes, or renames user-facing surface (CLI flags, commands, API endpoints, configuration keys, environment variables), the generated plan's acceptance criteria do not require corresponding documentation updates. This means documentation artifacts — READMEs, help text, usage examples — can silently go stale.Observed failure: In williamthorsen/workshop#38, the
--githuband--localflags were replaced by--from, but neither theshowListHelpprompt nor the package README were updated. The coder, all reviewers, and the holistic reviewer missed it. A simple search for the removed flags would have caught it instantly.Root cause: The planner never told the coder that documentation was part of the job. The coder's file-scope constraint ("only modify files that are part of the plan or directly required by it") then prevented it from touching unlisted files. Downstream reviewers verify acceptance criteria — but criteria that were never written can't be verified.
Why the existing pipeline didn't catch it: The review criteria include "Contracts: documented behaviors match implementations" and the holistic reviewer checks "Completeness," but neither instructs the agent to search for stale references. The failure is upstream: acceptance criteria are the mechanism that flows through the entire pipeline (planner → coder → reviewer), so the fix belongs there.
Proposed solution
Add a documentation coverage convention to all plan-producing skills, parallel to the existing test coverage convention. When a plan step adds, removes, or renames user-facing surface, its acceptance criteria must include corresponding updates to documentation, help text, and usage examples — including removal of references to anything that no longer exists.
Files to modify
Tier 1 — Plan producers (add documentation criterion to acceptance criteria guidance):
~/.claude/agents/orchestrated-planner.md~/.claude/agents/planner.mdorchestrated-plannerhas — add both)~/.claude/skills/design-and-plan/SKILL.mdTier 2 — Plan validator (add documentation check):
~/.claude/agents/plan-reviewer.mdWording guidance
The new bullet/convention should follow this structure (adapt to each file's style):
The key clause is "including removal of references to anything that no longer exists" — this directly addresses the PR #38 failure mode where new features were documented but old features weren't cleaned up.
Out of scope
superpowers:writing-plans— lives in a plugin cache and follows a different structure (TDD-focused steps without explicit acceptance criteria). A self-review checklist item could be added separately.aspect-documentation-reviewer— premature for what is a one-line addition to existing plan-producing skills.Acceptance criteria
Must have
orchestrated-planner.mdhas a documentation coverage bullet in "Step design principles"planner.mdhas both a test coverage bullet (currently missing) and a documentation coverage bullet in "Step design principles"design-and-plan/SKILL.mdhas a documentation criterion convention at both the ticket level and per-task levelplan-reviewer.mdhas a documentation coverage row in the completeness criteria tableShould have
Nice to have
--fromand default to compiled JS workshop#38 noting that this systemic fix was created in response to the oversight