Add mode system and two-threshold model to orchestrate-dev

[williamthorsen]

Problem

The /orchestrate-dev skill runs a single configuration: optional architecture/planning, fix all findings, opus holistic reviewer. For well-understood tasks (small features, bug fixes, straightforward refactors), this is over-provisioned — architecture and planning phases are skipped anyway, low-criticality findings consume review budget, and the holistic reviewer uses the most expensive model.

There is no way to select a lighter-weight orchestration profile without manually passing multiple -- arguments on every invocation.

Proposed solution

1. Mode system on /orchestrate-dev

Add a --mode=<vibe|strict> argument (default = no mode, preserving current behavior). Each mode is a preset bundle of settings:

Setting	vibe	(default)	strict
architecture	absent	optional	required
planning	absent	optional	required
approval-threshold	high	low	low
budget-threshold	high	low	low
holistic_reviewer	sonnet	opus	opus
max-review-rounds	1	3	4

Mode presets are defaults — any setting can be individually overridden via explicit CLI arguments (e.g., --mode=vibe --approval-threshold=medium).

2. Two-threshold model replacing --fix-low/--no-fix-low

Replace the boolean --fix-low/--no-fix-low with two orthogonal thresholds:

• --approval-threshold=<low|medium|high>: Findings at this level or above must be fixed for code approval. Findings below are reported but don't block.
• --budget-threshold=<low|medium|high>: Remaining review-round budget is spent only on findings at this level or above. Findings below this level are not fixed even if budget remains (unless they also meet the approval threshold).

Interaction:

criticality >= approval_threshold  →  MUST fix (blocks approval)
criticality >= budget_threshold    →  MAY fix (opportunistic, if budget remains)
criticality < both thresholds      →  report only

Backward compatibility: --fix-low ≡ --approval-threshold=low --budget-threshold=low; --no-fix-low ≡ --approval-threshold=medium --budget-threshold=medium.

3. Resolution cascade

For all mode-affected settings:

1. Explicit CLI arg (highest priority)
2. Mode preset (if --mode specified)
3. orchestration.<key> in preferences.yaml
4. Legacy alias (fix_low_findings → mapped to thresholds)
5. Engine default

Files affected

File	Change
packages/agents/content/skills/orchestrate-dev/SKILL.md	Rewrite: add modes, per-mode pipeline tables, threshold arguments
packages/agents/content/skills/orchestrate/SKILL.md	Edit: thresholds replace --fix-low, update resolution cascades, run-index.json schema
packages/agents/content/skills/orchestrate/modules/review-cycle.md	Edit: threshold-based flow control in Phase 4 and Phase 4b
packages/agents/content/skills/orchestrate-review/SKILL.md	Edit: threshold arguments (no mode support — fixed pipeline)
.agents/PROJECT.md	Edit: document modes in orchestration entry points

Acceptance criteria

• /orchestrate-dev accepts --mode=vibe and --mode=strict arguments
• Default mode (no --mode) preserves identical behavior to the current /orchestrate-dev
• vibe mode: pipeline has only implementation + review-cycle (no architecture, no planning); approval-threshold=high, budget-threshold=high, holistic_reviewer=sonnet, max-review-rounds=1
• strict mode: pipeline has all four phases as required; max-review-rounds=4
• --approval-threshold and --budget-threshold arguments are accepted by the engine and review-cycle module
• Flow control in Phase 4 (parallel review) and Phase 4b (holistic review) uses the two-threshold model
• --fix-low/--no-fix-low remain as backward-compatible aliases mapping to the appropriate threshold values
• run-index.json config records approvalThreshold and budgetThreshold instead of fixLowFindings
• Explicit CLI arguments override mode preset values
• /orchestrate-review accepts the new threshold arguments (no mode support needed)
• .agents/PROJECT.md documents the mode system

[williamthorsen]

Design refinements from planning session

After critically evaluating the original proposal against the orchestration system architecture, several refinements emerged through discussion:

1. Mode system instead of separate skill

Rather than proliferating wrapper skills (orchestrate-dev-low, orchestrate-dev-fast, etc.), a --mode argument on /orchestrate-dev would be more extensible:

/orchestrate-dev mode=lite  <task>    # lightweight
/orchestrate-dev mode=max   <task>    # thorough
/orchestrate-dev            <task>    # normal (default)

Each mode would be a preset bundle of settings that can still be individually overridden.

2. Two-threshold system replacing --fix-low/--no-fix-low

The boolean --fix-low conflates two distinct decisions. A more precise model uses two thresholds:

• --approval-threshold=<level>: Findings at or above this level must be fixed for code approval. Findings below are reported but don't block. (e.g., medium = only medium+ findings block approval)
• --extra-budget-threshold=<level>: Findings below this level won't consume remaining review-round budget, even if rounds remain. (e.g., medium = don't spend budget on low findings)

These are orthogonal — you can require medium+ for approval but still opportunistically fix low findings if budget remains, or vice versa.

Mapping to current behavior:

• fix-low=true ≈ approval-threshold=low, extra-budget-threshold=none
• fix-low=false ≈ approval-threshold=medium, extra-budget-threshold=medium

3. Proposed mode presets

Setting	default	lite	max
architecture	optional	absent	required
planning	optional	absent	required
approval-threshold	low	medium	low
extra-budget-threshold	none	medium	none
holistic_reviewer model	opus	sonnet	opus
max-review-rounds	3	2	4

4. Scope considerations

This is bigger than the original "create a new skill" issue. It involves:

1. Adding mode-aware argument handling to the wrapper skill
2. Replacing --fix-low with the two-threshold system in the engine
3. Updating the review-cycle module's flow control for thresholds
4. Updating existing documentation

Could be split into:

• Phase 1: Add --mode=lite to orchestrate-dev with the original lightweight behavior (skip arch/planning, --no-fix-low, sonnet holistic, max-rounds=2)
• Phase 2: Refactor --fix-low into the two-threshold system
• Phase 3: Add --mode=max and other mode presets

[williamthorsen]

Implementation plan

Plan: Mode system and two-threshold model for /orchestrate-dev

Context

Issue #115 originally requested a lightweight /orchestrate-dev-low skill. Through design discussion, the scope evolved into a more extensible architecture:

1. Mode system on /orchestrate-dev (--mode=vibe|strict, default = no mode) — instead of proliferating wrapper skills
2. Two-threshold model replacing the boolean --fix-low/--no-fix-low — giving finer-grained control over which findings block approval and which get review budget
3. Mode-dependent pipeline — different modes configure different phases and argument defaults

Mode presets

Setting	vibe	(default)	strict
architecture	absent	optional	required
planning	absent	optional	required
approval-threshold	high	low	low
budget-threshold	high	low	low
holistic_reviewer	sonnet	opus	opus
max-review-rounds	1	3	4

Threshold semantics

--approval-threshold=<low|medium|high>: Findings at this level or above must be fixed for code approval. Findings below are reported but don't block.

--budget-threshold=<low|medium|high>: Remaining review-round budget is spent only on findings at this level or above. Findings below this level are not fixed even if budget remains, unless they also meet the approval threshold.

The two thresholds interact:

if criticality >= approval_threshold → MUST fix (blocks approval)
elif criticality >= budget_threshold → MAY fix (opportunistic, if budget remains)
else                                 → report only

Backward-compatible mapping: --fix-low ≡ --approval-threshold=low --budget-threshold=low; --no-fix-low ≡ --approval-threshold=medium --budget-threshold=medium.

Resolution cascade (for all mode-affected settings)

1. Explicit CLI arg (highest priority)
2. Mode preset (if --mode specified)
3. orchestration.<key> in preferences.yaml
4. Legacy alias (fix_low_findings → mapped to thresholds)
5. Engine default

Changes

1. Rewrite packages/agents/content/skills/orchestrate-dev/SKILL.md

Frontmatter: unchanged.

Arguments section: Add --mode argument. Replace --fix-low/--no-fix-low with --approval-threshold and --budget-threshold. Keep --fix-low/--no-fix-low as documented aliases.

New section: Mode presets with the table above and per-mode pipeline tables:

• Default (no --mode): architecture (optional) → planning (optional) → implementation (required) → review-cycle (required) — identical to current behavior
• vibe: implementation (required) → review-cycle (required)
• strict: architecture (required) → planning (required) → implementation (required) → review-cycle (required)

Process section: Updated to explain mode resolution: resolve mode → look up mode preset → apply explicit overrides → select the pipeline table for the resolved mode → invoke the engine with the resolved pipeline and arguments.

2. Update packages/agents/content/skills/orchestrate/SKILL.md

Line 11: Add orchestrate-dev modes to the wrapper list description.

Arguments section (lines 31-32): Replace --fix-low/--no-fix-low with --approval-threshold=<level> and --budget-threshold=<level>. Document the levels (low, medium, high). Add backward-compatible alias note for --fix-low/--no-fix-low.

Resolving fix-low section (lines 40-44): Rename to "Resolving thresholds". Document the resolution cascade for approval_threshold and budget_threshold, including the legacy fix_low_findings alias.

run-index.json config (lines ~178-181): Replace "fixLowFindings" with "approvalThreshold" and "budgetThreshold" in the config block.

Preferences example (lines ~89-95): Update to show the new threshold keys alongside legacy alias.

3. Update packages/agents/content/skills/orchestrate/modules/review-cycle.md

Inputs table (line 17): Replace {fix-low-findings} with {approval-threshold} and {budget-threshold}.

Phase 4 flow control (lines 143-150): Replace:

- low + fix_low_findings true + budget → fix + re-review
- low + (!fix_low || !budget) → proceed
- medium/high + budget → fix + re-review
- medium/high + !budget → needs_manual_review

With:

- criticality >= approval_threshold + budget → fix + re-review
- criticality >= approval_threshold + !budget → needs_manual_review
- criticality >= budget_threshold + budget → fix + re-review (opportunistic)
- criticality >= budget_threshold + !budget → proceed (not blocking, no budget)
- criticality < budget_threshold → proceed (report only)

Loop termination / convergence (line 186): Update convergence criteria — was "aggregated criticality is none, or low with fix_low_findings false", now "aggregated criticality is none, or below both thresholds, or below approval_threshold and above budget_threshold with no budget".

Phase 4b flow control (lines 279-287): Apply same threshold logic:

- none → converged
- criticality >= approval_threshold + budget → fix + re-review
- criticality >= approval_threshold + !budget → one fix round, needs_manual_review
- criticality >= budget_threshold + budget → fix + re-review (opportunistic)
- criticality >= budget_threshold + !budget → converged (not blocking)
- criticality < budget_threshold → converged (report only)

4. Update packages/agents/content/skills/orchestrate-review/SKILL.md

Arguments section (lines 28-29): Replace --fix-low/--no-fix-low with --approval-threshold and --budget-threshold, same as the engine. Keep --fix-low/--no-fix-low as documented aliases. No mode support — orchestrate-review's pipeline is always review-only, so mode-based pipeline changes don't apply. Thresholds are still useful here.

5. Update .agents/PROJECT.md

Orchestration entry points (around line 86): Add mode documentation:

- `/orchestrate-dev` — full workflow (default), lightweight (--mode=vibe), or thorough (--mode=strict)
- `/orchestrate-review` — review-only workflow for manually written code

Critical files

File	Action
packages/agents/content/skills/orchestrate-dev/SKILL.md	Rewrite (modes, thresholds, per-mode pipeline tables)
packages/agents/content/skills/orchestrate/SKILL.md	Edit (thresholds replace fix-low, wrapper list)
packages/agents/content/skills/orchestrate/modules/review-cycle.md	Edit (threshold-based flow control)
packages/agents/content/skills/orchestrate-review/SKILL.md	Edit (threshold arguments)
.agents/PROJECT.md	Edit (entry point docs)

Verification

1. Structural: All skill files have valid YAML frontmatter and follow the existing naming/structure conventions
2. Consistency: Threshold argument names, level values, and resolution cascades match across all four files
3. Backward compatibility: --fix-low/--no-fix-low aliases correctly map to threshold values in all documentation
4. Mode completeness: Each mode has a complete preset table entry and a corresponding pipeline table
5. Flow control correctness: Trace through each mode's thresholds against the review-cycle flow control to verify the right behavior for each criticality level
6. Build: pnpm run ws typecheck && pnpm run ws test in packages/agents/ to check for any test or type regressions
7. Cross-reference: run-index.json schema in engine matches the new field names (approvalThreshold, budgetThreshold instead of fixLowFindings)

[williamthorsen]

Implementation insights

T-level treadmill pattern: The aspect-code-reviewer exhibited a pattern where each fix to minor text prompted a new reading of adjacent text, consuming the review budget on polish rather than substance. For documentation-only changes, configuring --mode=vibe or --approval-threshold=medium avoids spending rounds on trivial refinements — this is exactly what the mode system was designed to solve.

Behavioral equivalence in model transitions: Changing from a boolean (fix_low) to a multi-valued threshold model required careful preservation of behavioral equivalence. The review caught that the Phase 4b flow control silently changed "fix + converged" to "fix + needs_manual_review" for low criticality with exhausted budget — a regression that would have altered default-mode behavior. The fix was to preserve the old semantics: when the holistic review is a final sanity check (not a gating review), exhausted budget with low findings should converge, not block.

Budget exhaustion is mode-agnostic: The needs_manual_review exit from budget exhaustion is a pre-check that fires before threshold rules. It exists in both old and new logic. The new threshold model doesn't change this behavior — but it provides better tools to manage it (e.g., --approval-threshold=medium prevents low findings from consuming budget in the first place).