Skip to content

Rewrite docs/Testing.md as a focused testing guide#199

Open
AndyTWF wants to merge 1 commit intomainfrom
testing-guidance
Open

Rewrite docs/Testing.md as a focused testing guide#199
AndyTWF wants to merge 1 commit intomainfrom
testing-guidance

Conversation

@AndyTWF
Copy link
Copy Markdown
Contributor

@AndyTWF AndyTWF commented Mar 30, 2026

Why

The old docs/Testing.md was 400 lines of unfocused guidance — heavy on running commands and debugging env vars, light on what to test and when. It didn't answer the key questions a contributor needs:

  • What kind of test should I write for this change?
  • What does a good PR test suite look like?
  • When is E2E worth the cost vs unit?

The "integration" layer was also misleading — it's 4 files all testing interactive mode subprocess behavior, not a general-purpose middle tier.

What changed

  • Replaced the 400-line emoji-heavy Testing.md with a 249-line opinionated guide that answers "I wrote a new command — what tests do I need?"
  • Defined four test layers (unit, E2E, TTY, subprocess) with clear purpose and when-to-use guidance
  • Made unit and E2E tests mandatory for every command
  • Added decision tree + explicit rules for choosing the right test layer
  • Honestly described test/integration/ as 4 interactive-mode files, not a general-purpose tier
  • Added helpers reference tables (core helpers, mock SDKs, factories, NDJSON, TTY)
  • Moved test structure/describe blocks/running tests out of AGENTS.md into Testing.md with a cross-reference

Test plan

  • Docs-only change — no code modified
  • pnpm exec eslint . passes (0 errors)
  • Read Testing.md end-to-end and confirm it answers the key contributor question
  • Verify AGENTS.md cross-reference link resolves correctly

🤖 Generated with Claude Code

Replace the 400-line emoji-heavy doc with a 249-line guide that answers
the key question: "I wrote a new command — what tests do I need?"

- Define four test layers (unit, E2E, TTY, subprocess) with clear
  purpose, speed, and "when to use" guidance
- Make unit and E2E tests mandatory for every command
- Add decision tree and explicit rules for choosing the right layer
- Honestly describe test/integration/ as 4 interactive-mode files,
  not a general-purpose integration tier
- Add helpers reference tables (core helpers, mock SDKs, factories,
  NDJSON, TTY)
- Remove decorative HTML, collapsed sections, and misleading
  integration test description
- Move test structure, describe blocks, and running tests out of
  AGENTS.md into Testing.md with a cross-reference

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@vercel
Copy link
Copy Markdown

vercel bot commented Mar 30, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
cli-web-cli Ready Ready Preview, Comment Mar 30, 2026 9:12am

Request Review

@claude-code-ably-assistant
Copy link
Copy Markdown

Walkthrough

This PR rewrites docs/Testing.md from a 400-line emoji-heavy document into a concise 249-line opinionated guide focused on answering the key contributor question: "I wrote a new command — what tests do I need?" Alongside this, 24 lines of test guidance are removed from AGENTS.md and replaced with a single cross-reference link to the new doc, consolidating all testing documentation in one place.

Changes

Area Files Summary
Docs docs/Testing.md Full rewrite: adds test layer decision table, decision tree, mandatory/optional test rules, running tests table, and helpers reference; removes decorative HTML, collapsed sections, and misleading integration tier description
Config AGENTS.md Removes 24 lines of duplicated test structure/describe block/running-tests content; replaces with a single cross-reference link to docs/Testing.md

Review Notes

  • Docs-only change — no source code or tests modified; no functional behavior affected.
  • AGENTS.md is the authoritative agent instruction file — removing content from it and pointing to an external doc means agents/LLMs must now follow the link to get describe-block rules and running-test commands. Verify the cross-reference link resolves correctly from AGENTS.md's location.
  • "integration" layer reframing — the doc now explicitly states test/integration/ is not a general-purpose tier (just 4 interactive-mode files) and hints at a future rename. This is accurate but worth confirming as a team consensus before it lands.
  • Required describe blocks rule moved out of AGENTS.md — this is a mandatory convention enforced in code review. Confirm reviewers are comfortable with it living only in Testing.md going forward.

@AndyTWF AndyTWF requested a review from umair-ably March 30, 2026 10:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant