Skip to content

Require documentation updates when API changes #404

Description

@williamthorsen

Description

When an orchestrated development session (/orchestrate-dev) adds, removes, or renames user-facing surface (CLI flags, commands, API endpoints, configuration keys, environment variables), the generated plan's acceptance criteria do not require corresponding documentation updates. This means documentation artifacts — READMEs, help text, usage examples — can silently go stale.

Observed failure: In williamthorsen/workshop#38, the --github and --local flags were replaced by --from, but neither the showListHelp prompt nor the package README were updated. The coder, all reviewers, and the holistic reviewer missed it. A simple search for the removed flags would have caught it instantly.

Root cause: The planner never told the coder that documentation was part of the job. The coder's file-scope constraint ("only modify files that are part of the plan or directly required by it") then prevented it from touching unlisted files. Downstream reviewers verify acceptance criteria — but criteria that were never written can't be verified.

Why the existing pipeline didn't catch it: The review criteria include "Contracts: documented behaviors match implementations" and the holistic reviewer checks "Completeness," but neither instructs the agent to search for stale references. The failure is upstream: acceptance criteria are the mechanism that flows through the entire pipeline (planner → coder → reviewer), so the fix belongs there.

Proposed solution

Add a documentation coverage convention to all plan-producing skills, parallel to the existing test coverage convention. When a plan step adds, removes, or renames user-facing surface, its acceptance criteria must include corresponding updates to documentation, help text, and usage examples — including removal of references to anything that no longer exists.

Files to modify

Tier 1 — Plan producers (add documentation criterion to acceptance criteria guidance):

File Location Pattern to follow
~/.claude/agents/orchestrated-planner.md "Step design principles" (line 43) Add a sibling bullet after the test coverage bullet
~/.claude/agents/planner.md "Step design principles" (lines 34–41) Add a documentation bullet (note: this file is also missing the test coverage bullet that orchestrated-planner has — add both)
~/.claude/skills/design-and-plan/SKILL.md Lines 104 and 154 Add a "Documentation criterion convention" parallel to the "Test criterion convention," at both the ticket level and the per-task level

Tier 2 — Plan validator (add documentation check):

File Location Pattern to follow
~/.claude/agents/plan-reviewer.md Completeness criteria table (lines 50–56) Add a "Documentation coverage" row

Wording guidance

The new bullet/convention should follow this structure (adapt to each file's style):

When a step adds, removes, or renames user-facing surface (CLI flags, commands, API endpoints, configuration keys, environment variables), its acceptance criteria must include corresponding updates to documentation, help text, and usage examples — including removal of references to anything that no longer exists.

The key clause is "including removal of references to anything that no longer exists" — this directly addresses the PR #38 failure mode where new features were documented but old features weren't cleaned up.

Out of scope

  • superpowers:writing-plans — lives in a plugin cache and follows a different structure (TDD-focused steps without explicit acceptance criteria). A self-review checklist item could be added separately.
  • Adding special-case instructions to the coder or reviewer agents — the acceptance criteria mechanism already flows through both; strengthening the source is sufficient.
  • A dedicated aspect-documentation-reviewer — premature for what is a one-line addition to existing plan-producing skills.

Acceptance criteria

Must have

  • orchestrated-planner.md has a documentation coverage bullet in "Step design principles"
  • planner.md has both a test coverage bullet (currently missing) and a documentation coverage bullet in "Step design principles"
  • design-and-plan/SKILL.md has a documentation criterion convention at both the ticket level and per-task level
  • plan-reviewer.md has a documentation coverage row in the completeness criteria table

Should have

  • Wording is consistent across all four files (adapted to each file's style, but conveying the same rule)
  • The "removal of references" clause is explicit in every instance

Nice to have

Metadata

Metadata

Labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions