v1.21.1.0 test: tighten plan-ceo-review smoke (Step 0 must fire)

[garrytan]

Summary

Tightens the gate-tier /plan-ceo-review plan-mode smoke so the regression where the agent skips Step 0 entirely and goes straight to ExitPlanMode now FAILS the gate. Refactors the PTY harness so the contract is testable in 14ms of free unit tests instead of $0.50 of stochastic PTY.

• PTY runner refactor: extracts classifyVisible() pure function (silent_write → plan_ready → asked → null). Permission dialogs filtered out of 'asked' so a permission prompt cannot pose as a Step 0 skill question. Adds TAIL_SCAN_BYTES = 1500 shared constant + env? passthrough on runPlanSkillObservation.
• Tightened smoke: assertion narrows from ['asked', 'plan_ready'] to 'asked' only for plan-ceo (sibling smokes for plan-eng / plan-design / plan-devex stay loose — they legitimately reach plan_ready on certain branches without asking).
• 24 new unit tests: deterministic coverage of the classifier, permission-dialog co-trigger, env passthrough surface. Run free on every bun test in 14ms.
• Co-trigger contract: bare Do you want to proceed? no longer triggers permission detection on its own — needs a file-edit context (Edit to <path> / Write to <path>).

Test Coverage

runPlanSkillObservation('plan-ceo-review', plan)
  └── while polling:
      ├── isPlanReadyVisible       → return 'plan_ready'   [GAP→COVERED] FAIL: agent skipped Step 0
      ├── isNumberedOptionListVisible → return 'asked'      [★★★ TESTED] PASS: Step 0 fired
      ├── Write/Edit unsanctioned   → return 'silent_write' [★★  TESTED] FAIL
      ├── proc.exited               → return 'exited'        [★★  TESTED] FAIL
      └── budget elapsed            → return 'timeout'       [★★  TESTED] FAIL

COVERAGE: 5/5 outcomes asserted (was 3/5 — 'plan_ready' previously treated as PASS).

Tests: 0 → 24 unit tests added (test/helpers/claude-pty-runner.unit.test.ts).

Pre-Landing Review

13 informational findings (testing + maintainability specialists), 0 critical. 8 auto-fixed, 5 escalated to user (D5 + D7 + D8 + D9 + plus the env-passthrough honesty fix). All resolved.

Adversarial review surfaced one real issue I owe transparency on: the env: { QUESTION_TUNING: 'false', EXPLAIN_LEVEL: 'default' } passthrough is plumbing-only on current code — gstack-config reads ~/.gstack/config.yaml and not env vars. The test docstring now documents this honestly. P3 TODO captured to make gstack-config env-aware so the wiring is real, not advisory.

Plan Completion

D1 (tighten existing) ✓ · D3 (durable section names) ✓ · D4 (env passthrough) ✓ · D5 (classifier filter) ✓ · D6 (deterministic unit test) ✓ · D7 (regex co-trigger) ✓ · D8 (extract classifyVisible) ✓ · D9 (TAIL_SCAN_BYTES constant) ✓. D2 deferred to TODOS as planned (V2 per-finding count assertion).

TODOS

3 captured for v1.21.1.0:

• P2: per-finding AskUserQuestion count assertion (V2)
• P3: honor env vars in gstack-config (so QUESTION_TUNING=false actually isolates the test)
• P3: path-confusion hardening on SANCTIONED_WRITE_SUBSTRINGS

Test plan

• 24/24 unit tests pass in 14ms (free, deterministic)
• PTY gate test passes 4/4 runs (3 lock-in + 1 post-refactor verification, ~$2.00 spent)
• 339 helper + skill-validation tests pass post-commit
• No collateral on sibling plan-*-plan-mode smokes (their loose assertion is unchanged)

🤖 Generated with Claude Code

---

^{Need help on this PR? Tag @codesmith with what you need.}

• Let Codesmith autofix CI failures and bot reviews

[github-actions]

E2E Evals: ✅ PASS

0/0 tests passed | $0 total cost | 12 parallel runners

Suite	Result	Status	Cost

---

12x ubicloud-standard-2 (Docker: pre-baked toolchain + deps) | wall clock ≈ slowest suite