Revise guidance so that code changes are always accompanied by tests

[williamthorsen]

Revise guidance so that code changes are always accompanied by tests

Problem

The orchestrated coder sometimes writes code without tests, and reviewers classify missing tests as a deferrable TODO (T-level finding). This means tests can be omitted at low and medium effort levels without blocking the workflow. The result: code ships without test coverage for new behavior.

Context

The testing requirement needs to be enforced at production time (when the coder writes code), not just at evaluation time (when the reviewer checks it). The current guidance has four gaps:

1. Ticket creation (design-and-plan) doesn't require test-related acceptance criteria
2. Planner (orchestrated-planner) doesn't include test coverage in step acceptance criteria
3. Coder (orchestrated-coder) is told to run tests but not to write them
4. Reviewers (orchestrated-reviewer, aspect-test-reviewer) receive ticket requirements but aren't told to verify acceptance criteria are satisfied

The review-criteria finding scheme (T for missing tests) remains appropriate for general-purpose PR review. This change targets the orchestrated pipeline specifically.

Solution

Shift the testing requirement left through four layers of the orchestration pipeline:

1. Ticket creation (design-and-plan skill)

Add guidance that acceptance criteria for code changes must include a test criterion (e.g., "New/modified behavior is covered by tests"). This is where the requirement originates for interactively planned work.

2. Planner (orchestrated-planner)

Add to step design principles: when a step creates or modifies testable behavior, its acceptance criteria must include test coverage. The planner already validates ticket requirements → plan coverage; this makes tests an explicit part of that validation.

3. Coder (orchestrated-coder)

Add an explicit mandate: tests are part of the deliverable. When implementing a step whose acceptance criteria include tests, write those tests as part of the step — not as a follow-up or separate step.

4. Reviewers (orchestrated-reviewer, aspect-test-reviewer)

Add to the review process: verify that ticket/plan acceptance criteria are satisfied by the implementation. For the test reviewer specifically, if test-related acceptance criteria exist and are not met, this is a contract violation (F-level), not a deferrable TODO.

5. Testing conventions (testing-conventions skill)

Add a section defining when tests are required and narrow carve-outs for genuinely untestable code (e.g., generated CSS classes, pure visual styling, static configuration). This gives the coder a reference for the rare cases where skipping tests is legitimate.

Acceptance criteria

• design-and-plan skill guidance includes test criteria in acceptance criteria conventions
• orchestrated-planner step design principles require test coverage in acceptance criteria for steps that create/modify testable behavior
• orchestrated-coder has an explicit mandate to write tests alongside code, with reference to testing-conventions for carve-outs
• orchestrated-reviewer process includes verification of ticket acceptance criteria
• aspect-test-reviewer process includes verification of test-related acceptance criteria, classifying unmet criteria as contract violations
• testing-conventions skill defines when tests are required and carve-outs for untestable code
• New/modified behavior in this change is covered by tests

[williamthorsen]

Insight: Plan-level frontmatter blind spots

The implementation plan correctly identified all six target files and the prose changes needed, but missed that orchestrated-planner needs testing-conventions in its skills frontmatter list — not just orchestrated-coder. The plan noted the coder's skills addition (Task 4) but didn't include the same for the planner, even though Task 3 references the skill in prose.

Review W1 caught this gap. Takeaway: plans can have blind spots at the frontmatter/wiring level even when the prose guidance is correct. Skills referenced in agent prose but absent from the skills frontmatter are unreachable at runtime — this is a class of error worth checking for during plan review.