Skip to content

Add a testscript end-to-end layer for the CLI contract - #61

Open
MPV wants to merge 1 commit into
masterfrom
claude/e2e-testscript
Open

Add a testscript end-to-end layer for the CLI contract#61
MPV wants to merge 1 commit into
masterfrom
claude/e2e-testscript

Conversation

@MPV

@MPV MPV commented Aug 2, 2026

Copy link
Copy Markdown
Owner

What

A black-box e2e layer that runs the real kir binary and asserts the actual user contract — argv + stdinstdout, stderr, exit code — using testscript. Each scenario is one self-contained .txtar file with its fixtures inlined.

  • main_test.go — registers the real program as a testscript command (TestMain + TestScripts).
  • testdata/script/*.txtar — 7 scenarios.

This is what "combining e2e + golden" looks like in practice: the script is the golden, and it natively models the exit code and stderr, which approvals.VerifyString can't.

Why here (vs. approvals in cmd/)

approvals/ tests manifest → images at the processor level; nothing covers the binary's CLI contract (exit codes, real stderr, arg/stdin wiring). This adds that top layer without touching approvals/ or the unit tests — an additive, standalone layer to get a feel for the ergonomics.

The 7 scenarios

file asserts
basic single manifest → image on stdout, clean stderr (cmp stdout)
multidoc-file ----separated docs in one file, in document order
multiple-files several file args, concatenated in order
stdin kir - reading a real stdin pipe
unsupported-kind a Service → error on stderr, nothing on stdout, exit 0
missing-file a path matching nothing → no output, exit 0
usage no args → Usage: on stderr, non-zero exit (! kir)

Cost

  • Dependency: rogpeppe/go-internal promoted to a direct require — already in the graph via go-approval-tests, so no new download.
  • Speed: each scenario builds/execs the binary; the whole e2e suite runs in ~0.05s here (7 tiny scenarios). -race, vet, gofmt, and go mod tidy are all clean.

To regenerate expectations after an intended behavior change, wire testscript's UpdateScripts behind a -update flag (not wired here; expectations are hand-written and verified against the real binary).

🤖 Generated with Claude Code

https://claude.ai/code/session_01Pc6NAURAqjU4LYJx93tgSC

The existing approvals/ suite tests manifest -> images at the processor
level; nothing exercises the actual binary's user contract (argv + stdin
-> stdout, stderr, exit code). Add a black-box e2e layer using
rogpeppe/go-internal's testscript (already in the module graph via
go-approval-tests, so no new download): main_test.go registers the real
kir program as a testscript command, and each testdata/script/*.txtar is
one self-contained scenario with its fixtures inlined.

Seven scenarios: single file, multi-document file, multiple file args,
stdin, an unsupported kind, a missing path, and no-args usage.

approvals/ and the unit tests are left untouched; this is an additive,
standalone layer.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pc6NAURAqjU4LYJx93tgSC
@MPV
MPV force-pushed the claude/e2e-testscript branch from 4e29688 to c229fb3 Compare August 2, 2026 07:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants