Skip to content

Rule 1 of next-steps-after-plan matches nearly every substantive plan, over-recommending refine-plan #740

Description

@williamthorsen

Problem

Rule 1 gives refine-plan the strongest marker for nearly every substantive plan, because both of its conjuncts are near-always true at plan time:

  • Its structural triggers (new interface or module, contract-shape change, dependency-boundary change, control-flow change) are tripped by substantive plans almost by construction.
  • "The plan has not been previously refined" is true by definition at plan time, since refine-plan runs after planning.

The rule therefore behaves as a constant rather than a heuristic, and rule 2 ("implement directly with follow-up review") is rarely reachable for non-trivial work.

The knowledge base records seven instances between 2026-06-11 and 2026-07-14 across four repositories. In five, the user challenged the recommendation and the agent withdrew it on inspection. The recommended step is often one the agent does not believe in: in one episode, the line it wrote for the recommended option read "I checked the consumer set by hand and believe it's closed — a refine pass is the cheap way to find out whether I missed one."

A prior fix (#354, 2026-03-23) narrowed the structural triggers. All seven episodes postdate it.

Context

  • packages/agents/content/skills/_partials/next-steps-after-plan.md, build-time inlined into design-and-plan, plan, refine-plan, and save-plan.
  • The spec already tells the agent to weigh caller-supplied recommendation context ("whether the plan was developed interactively"). That instruction is advisory, and it loses to the ordered first-match cascade every time: the context is passed in and has nowhere to act.
  • A refine pass re-reads the plan. It structurally cannot answer a plan's empirical unknowns, the ones resolved by running code or writing the test.
  • KB events: 01KTW96Z, 01KW657D, 01KXBGZF, 01KXBHF8, 01KXE2X0, 01KXEJ4W, 01KXGFX3.

Proposed solution

Rule 1 matches only when the agent can name something a refine pass would surface that the design work left unsettled, and the rendered output must carry that name. An agent with nothing to name cannot write the required line, so the cascade falls through on its own. This makes the judgment structural rather than advisory. The structural triggers survive as evidence to weigh, demoted from sufficient condition.

Not a narrowed trigger list, which is what #354 did. Not an override clause layered onto the existing rule, which would be a second advisory clause standing beside the one already being ignored.

The option list, the rendering shape and marker glyphs, and the HARD-GATEs stay unchanged; the output format gains a constraint and relaxes nothing. They guard the opposing failure — the agent improvising the menu, recorded five times in the KB — and are pinned by spec-inlining.test.ts.

Acceptance criteria

Must have

  • Rule 1 can return false for a substantive plan whose load-bearing decisions were settled during design; it no longer matches by construction.
  • Selecting Refine plan requires naming, in the rendered output, the specific unsettled decision a refine pass would surface. When none can be named, the cascade falls through.
  • A plan whose residual unknowns are resolvable only by running code, rather than by re-reading the plan, does not match rule 1.
  • Walked against the seven recorded episodes, the revised rule falls through to rule 2 in each.
  • Every path still yields a recommendation with reasoning; no path renders a neutral menu.
  • The option list, the rendering shape and marker glyphs, and the HARD-GATEs are unchanged, so the guards against menu improvisation are not relaxed. The output format gains a constraint and no relaxation.
  • The new rule's text is pinned against silent removal, as the spec's other rules are.

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureAdded or improved external functionalityscope:agents

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions