Have reviewers write findings incrementally for interruption resilience

[williamthorsen]

Problem

When an orchestrated reviewer is interrupted mid-analysis — most commonly by max_turns exhaustion or by the harness pausing for approval — no review artifact is produced. Each reviewer writes its single artifact at the end of the invocation, so a cutoff leaves the orchestrator with:

• no durable record of findings that had already been identified,
• no structured handoff for a continuation dispatch,
• fragile recovery limited to inspecting tool-result echoes, re-running review from scratch, or redispatching with inferred partial context.

Observed in a live orchestrated run: two reviewers paused mid-analysis (both converging on the same finding); the orchestrator detected no artifacts on disk and had to redispatch both with inferred partial context.

Context

• Same failure class as Have orchestrated-coder write change-summary incrementally for interruption resilience #448 (coder incremental writes), but the scaffold pattern differs structurally.
• The coder's review-response mode can pre-enumerate its sections because findings are input (F1, F2, W1 come from the reviewer's output). A reviewer cannot — its findings are the output, discovered during analysis. The scaffold for reviewers must be "write skeleton early, append + overwrite per finding," not "enumerate all slots upfront."
• Affects at least five agent definitions:
  • orchestrated-reviewer (Phase 4 core reviewer + Phase 4b holistic)
  • aspect-code-reviewer
  • aspect-silent-failure-reviewer
  • aspect-test-reviewer
  • code-simplification-reviewer
• Reviewer turn budgets are smaller (20–30 vs. coder's 80–150), so interruption is rarer per-reviewer but still observed.
• Reviewers' structured return blocks carry a Criticality field parsed by the orchestrator. Partial artifacts must either remain parseable or the orchestrator must tolerate a missing return block and infer criticality from the partial content.

Proposed approach (sketch — not pre-decided)

1. Scaffold write early: first tool use after reading inputs is a Write of the review skeleton — header, empty ## Findings section, ## Criticality: (pending), any reviewer-specific placeholder sections.
2. Append + overwrite per finding: after each finding crystallizes (diff citation, severity, recommendation), overwrite the file with the growing list.
3. Finalize before return block: set ## Criticality aggregate and any summary sections in the reserved final turns.
4. Apply uniformly across all five reviewer agents, adapting the skeleton to each reviewer's artifact format.

Open design questions

• Should a reviewer with zero findings still write the scaffold early (to signal "I started")? Probably yes — absence of the artifact file is currently the orchestrator's only interruption signal.
• How should the orchestrator handle a partial reviewer artifact that lacks the structured return block? Same recovery path as the coder (read what's present, treat as interruption), or reviewer-specific logic?
• Does Criticality need a "tentative" value during incremental writes, or should it stay (pending) until the end?

Out of scope

• Reviewer turn budget changes (separately decidable).
• SendMessage-based resumption for reviewers. Enable SendMessage-based resumption for interrupted orchestrated-coder dispatches #450 covers the coder; a parallel reviewer ticket can be filed if the coder work validates the approach.

Acceptance criteria

• Each of the five reviewer agent definitions specifies incremental findings writes: scaffold as first tool use, overwrite as findings accumulate, finalize aggregate before the return block.
• Example scaffold included in each agent definition.
• Orchestrator handles partial reviewer artifacts gracefully — reads what's present when the structured return block is missing.

References

• Parent/sibling: Have orchestrated-coder write change-summary incrementally for interruption resilience #448 (coder incremental writes)
• Related: Enable SendMessage-based resumption for interrupted orchestrated-coder dispatches #450 (SendMessage-based coder resumption)

[williamthorsen]

Insight from orchestrated run 20260503-191453Z (effort=low):

High-trust plan adoption worked correctly end-to-end on this ticket. The plan was authored by design-and-plan (interactive) with baseSha: f81da04, which exactly matched origin/main at run time. The orchestrator's trust-tier evaluation classified it as high, which:

• Skipped the Architecture phase (high-trust plans are presumed to have validated codebase-structure assumptions during the interactive design pass).
• Skipped the Planning phase (the orchestrator adopted the external plan as its canonical 04_orchestrator_orchestration-plan.{md,json} artifact via the high-trust conversion path).

Both downstream phases (Implementation, Review) succeeded with no quality loss observed:

• Implementation completed in a single coder dispatch (~130k tokens) with all quality gates passing.
• Phase 4 core review returned criticality: none, no fix cycle needed.
• Phase 4b holistic review confirmed all four acceptance criteria were addressed and explicitly validated each agent-definition design decision the plan called out.

Estimated savings: roughly two subagent dispatches (~30k tokens) that would have re-derived the plan's conclusions.

Validation point: the high-trust mechanism is doing what it was designed to do — when an interactively-designed plan is current with main, re-running planning is wasted compute, and the orchestrator correctly skips it without degrading downstream quality.