Skip to content

Multi-agent review profiles for entire review - #1312

Merged
dipree merged 151 commits into
mainfrom
review-profiles
Jun 26, 2026
Merged

Multi-agent review profiles for entire review#1312
dipree merged 151 commits into
mainfrom
review-profiles

Conversation

@dipree

@dipree dipree commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

https://entire.io/gh/entireio/cli/trails/466

Adds configurable multi-agent review profiles to the hidden experimental entire review command.

User flow

1. Configure a profile

entire review --configure
entire review --configure general --local
entire review --configure security \
  --set-slot claude-code=opus \
  --set-slot codex=o4-mini \
  --set-judge claude-code=opus \
  --set-output trail \
  --set-task "Review auth and data-access risks."

Profiles live in .entire/settings.json for shared config or .entire/settings.local.json for local config. Scripted setup supports --set-agents, repeatable --set-slot agent[=model], repeatable --set-model agent=model, --set-judge agent[=model], --set-output local|trail, and --set-task.

2. Inspect and edit configuration

entire review --list
entire review --agents --profile general
entire review --models
entire review --models --agent claude-code
entire review --edit --profile general

3. Run a review

entire review general
entire review --profile general --base origin/main --prompt "Focus on regressions" --timeout 15m
entire review --profile general --agent codex --model o4-mini
entire attach --review <session-id>

Reviewers run in parallel. One final judge runs after them and emits the concise verdict. --timeout defaults to 10m per reviewer; --timeout 0 disables the bound. A single-reviewer run is available with --agent, and --model only applies with --agent.

4. Read results

entire review --findings

Local output writes the review manifest and browsable findings. --set-output trail also posts the final verdict to the branch Trail as guarded structured findings when possible, with markdown fallback.

Hardening included

  • Start and wait failures flow through terminal reviewer markers.
  • Reviewer timeouts are cause-marked and do not duplicate synthetic run errors.
  • Trail posting preserves line anchors and single-line ranges.
  • Trail output is validated at the posting boundary.

Verification

  • go build ./...
  • go test ./...
  • go test -race ./cmd/entire/cli/review/
  • golangci-lint run ./...

Copilot AI review requested due to automatic review settings June 1, 2026 19:26

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 5 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit bc13a81. Configure here.

Comment thread cmd/entire/cli/review/picker.go
Comment thread cmd/entire/cli/review/cmd.go Outdated
Comment thread cmd/entire/cli/review/cmd.go Outdated
Comment thread cmd/entire/cli/review/picker.go
Comment thread cmd/entire/cli/review/manifest.go Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR redesigns the experimental entire review command around named review profiles with fan-out to multiple worker agents and mandatory master adjudication (for multi-worker profiles), replacing the legacy per-agent review config and removing the old migration flow.

Changes:

  • Introduces review_profiles + review_default_profile settings, including worker-level agent aliases and model hints, plus a profile master used for final adjudication.
  • Updates entire review CLI surface (--profile, positional profile name, --configure, --prompt, and --agent targeting a single worker).
  • Implements profile-native master synthesis via a new AgentSynthesisProvider, and threads --model into Claude/Codex/Gemini review runner argv.

Reviewed changes

Copilot reviewed 27 out of 27 changed files in this pull request and generated 9 comments.

Show a summary per file
File Description
docs/architecture/review-command.md Updates architecture documentation for profile-based review and master adjudication.
cmd/entire/cli/settings/settings.go Adds settings schema/types for review profiles and default profile; keeps legacy fields for parsing.
cmd/entire/cli/root.go Updates root command comment to reflect profiles.
cmd/entire/cli/review/types/sink.go Extends per-run metadata to include display name vs registry name and model.
cmd/entire/cli/review/types/reviewer.go Extends RunConfig with profile/task/model context for profile-based prompts and runs.
cmd/entire/cli/review/synthesis_sink.go Adds agent-backed master synthesis provider and auto/adjudication-mode wiring.
cmd/entire/cli/review/synthesis_prompt.go Updates synthesis prompt to an adjudication-style “final report” with rules and sections.
cmd/entire/cli/review/run.go Records worker display name vs registry agent name and model in RunSummary; forwards events under display name.
cmd/entire/cli/review/run_multi.go Propagates agent registry name and model through multi-run state and summaries.
cmd/entire/cli/review/prompt.go Composes worker prompts from skills + profile name + canonical task + per-agent/per-run instructions.
cmd/entire/cli/review/profile.go New profile resolution/default-task logic, worker/master selection, and preference persistence helpers.
cmd/entire/cli/review/picker.go Adds guided setup flow and profile-aware editing; introduces master selection and model picking.
cmd/entire/cli/review/migration.go Removes legacy migration flow from project settings to clone preferences.
cmd/entire/cli/review/migration_test.go Removes tests for the deleted migration flow.
cmd/entire/cli/review/marker_fallback.go Updates terminology/docs for “no adapter yet” manual fallback flow.
cmd/entire/cli/review/manifest.go Improves manifest/session matching for worker aliases and model hints.
cmd/entire/cli/review/fix.go Updates fix-agent choice derivation to pull from profiles (including master) and dedupe aliases.
cmd/entire/cli/review/export_test.go Updates test export wrapper for changed synthesis prompt signature.
cmd/entire/cli/review/cmd.go Rebuilds entire review command flow: profile resolution, guided setup, fan-out, and master synthesis.
cmd/entire/cli/review/cmd_test.go Updates command tests for profile fan-out behavior and --prompt flow.
cmd/entire/cli/review_context_test.go Updates review-context test settings to the new profile schema.
cmd/entire/cli/review_bridge.go Removes legacy synthesis provider wiring; keeps reviewer wiring to avoid import cycles.
cmd/entire/cli/labs.go Updates labs/experimental command summary text.
cmd/entire/cli/integration_test/review_test.go Updates integration tests to configure reviews via profiles.
cmd/entire/cli/agent/geminicli/reviewer.go Threads model hint into Gemini review runner command.
cmd/entire/cli/agent/codex/reviewer.go Threads model hint into Codex review runner argv while keeping stdin ordering.
cmd/entire/cli/agent/claudecode/reviewer.go Threads model hint into Claude review runner argv.

Comment thread cmd/entire/cli/settings/settings.go
Comment thread cmd/entire/cli/review/picker.go Outdated
Comment thread docs/architecture/review-command.md Outdated
Comment thread docs/architecture/review-command.md Outdated
Comment thread cmd/entire/cli/review/synthesis_sink.go Outdated
Comment thread cmd/entire/cli/settings/settings.go
Comment thread cmd/entire/cli/review/picker.go Outdated
Comment thread docs/architecture/review-command.md Outdated
Comment thread docs/architecture/review-command.md Outdated
dipree added 19 commits June 2, 2026 12:14
- Preserve profile task/master_model when saving from --edit
- Route empty/placeholder profiles through first-run setup
- Skip re-review confirm in non-interactive runs instead of erroring
- Treat 'start review now?' abort as a clean cancel
- Fix lint: drop dead helpers, lowercase error, perfsprint, un-deprecate
  the still-used review_fix_agent field
- Doc/comment fixes (ReviewConfig prompt/skills, master adjudication)
Strip the provider prefix and thinking-level suffix when matching a
configured profile model against a session's recorded model, so hints like
'anthropic/claude-sonnet:high' link to session model 'claude-sonnet-4-5'.
Same-model/different-thinking workers fall back to start-time + used-session
disambiguation.
- Drop the --fix and --all flags and the fix-apply pathway (findings remain
  browsable via --findings).
- Add --model to override the model for a single --agent worker run
  (--model requires --agent).
- Remove now-dead fix-agent selection helpers; keep the shared default/saved
  agent-pick helpers used by the master picker.
- Update post-run footer/findings output and docs to drop --fix references.
- entire review --configure now prints available review agents (adapter-backed,
  with installed status) and current profiles every time, so it's the
  discovery entry point.
- Accept --set-agents/--set-master/--set-task/--set-model to write a profile
  non-interactively (no TUI); --set-* writes preserve untouched profile-level
  fields (task, master_model). With no --set-* flags it falls back to the
  guided wizard (interactive) or a discovery view (non-interactive).
- --agent errors now list the profile's configured workers.
- Available agents derive from the registry + ReviewerFor, so the catalog
  never drifts from what review can actually launch.
The catalog now appears only in the non-interactive discovery view; the
interactive wizard already lists selectable agents, so showing the catalog
first was redundant. Scripted --set-* writes just confirm.
- New optional agent.ModelLister capability (agent.ModelInfo + AsModelLister).
- Curated, clearly-labeled model lists for claude-code, codex, gemini (their
  CLIs have no --list-models). --model still forwards any value the CLI accepts.
- entire review --models lists advertised models per review agent (optionally
  filtered by --agent); needs no repo/profile.
- Designed so agents with a live model command (e.g. Pi's pi --list-models)
  can implement ListModels by shelling out later.
Symmetric with --models: --agents lists the worker agents valid for --agent
in the resolved profile (with hook-install status and the master marked),
falling back to the available review-agent catalog when no profile exists.
Replace the free-text 'Model for <agent>' input with a select of the agent's
advertised models (via agent.ModelLister), plus 'Default' and 'Custom…' (free
text) options. Falls back to free text when an agent advertises no models.
Pi can now be driven by `entire review` like claude-code/codex/gemini:
it runs headless via `pi --mode json <prompt>`, inherits the
ENTIRE_REVIEW_* env vars, and self-tags its session through the existing
hook adoption path. No marker file or manual attach is needed.

- Add cmd/entire/cli/agent/pi/reviewer.go: NewReviewer() (ReviewerTemplate)
  + a parser mapping pi's JSON event stream to review Events, with tests.
- Wire pi into launchableReviewerFor.
- Remove the `entire review attach` subcommand and its plumbing
  (Deps.AttachCmd, newReviewAttachCmd). Its only unique behavior —
  consuming a pending-review marker — is folded into `entire attach
  --review`, which already covers manual tagging. Tests migrated to the
  `attach --review` surface; stale doc comments updated.

Entire-Checkpoint: 86a03348156e
Replace the guided setup's multi-step flow (which agents? → choose
models? → per-agent model → add-variant loop) with a single "Build the
review crew" screen: one multiselect per launchable agent listing its
models plus Default and Custom…. Checking multiple models under an agent
creates that many workers (same agent, different models) — making the
variant concept native instead of a hidden loop. Default is pre-checked,
so pressing enter yields one worker per agent on its default model.

- Add promptForReviewCrew + resolveCrewModel; remove
  promptForSimpleReviewAgents, promptForSimpleReviewModels,
  promptForModelChoice, agentNamesToTypes.
- Workers always set cfg.Agent, so prompt/model-only agents (e.g. Pi) are
  now selectable in guided setup — previously skipped because their
  default config is empty.

Entire-Checkpoint: aaea3422dc87
…+model)

Reframe crew configuration around worker slots rather than "per agent".
A slot is an agent + model; you can define any number, including the same
agent multiple times on different OR identical models (e.g. 5 Claude
slots). The data model already supported this (profile.Agents is keyed by
an arbitrary worker name, disambiguated by workerIDForAgentModel) — this
change updates the two config surfaces that were still agent-keyed.

Picker: replace the per-agent model grid with a slot list — seeded with
one default-model slot per launchable agent, then add/remove/duplicate
slots (each add = pick agent → pick model/Default/Custom).

Scripted: add repeatable `--set-slot agent[=model]`. `--set-agents`
stays as the one-default-slot-per-agent shorthand (keeps `--set-model`
targeting). Duplicate slots are allowed and become distinct workers.

Entire-Checkpoint: c1dee195eb9b
The Add/Remove/Done action menu was clunky. Replace it with a linear
flow: configure a first worker (agent + model), print it, then ask
"Add another worker?" — repeating until Done. The crew is always
non-empty (a first slot is mandatory), so no empty-state handling, and
duplicates still work (add the same agent again).

Removes the action-menu plumbing (promptCrewAction, crewSummary,
promptCrewRemove, and the crewAction* consts); adds promptAddAnotherSlot.
Slot removal is dropped — re-run `entire review --configure` to rebuild.

Entire-Checkpoint: 8b762f4940eb
Foundation for the scout multi-agent flow: the master can now be its own
agent + model that synthesizes the workers' reports without being one of
the worker slots.

- settings: add ReviewProfileConfig.MasterAgent (standalone master);
  legacy worker `Master` still honored when MasterAgent is empty.
- profile: add profileMasterIdentity (MasterAgent wins, else legacy
  worker), masterDisplayLabel; resolveProfileMaster now delegates.
- cmd: multi-agent validation accepts a standalone master (validated for
  text generation) instead of requiring master ∈ workers; run path and
  scripted config updated accordingly.
- tests for master resolution precedence.

Entire-Checkpoint: 38a797061770
Rebrand the multi-agent command as `entire scout`, reusing the existing
review engine. `review` stays as a hidden cobra alias for back-compat, so
both invocations work. Updated command Use/Short/Long, user-facing
guidance strings ("entire review …" → "entire scout …"), the labs entry,
and affected test expectations.

Internal package, settings keys (review_profiles), and env vars
(ENTIRE_REVIEW_*) are unchanged — branding only, no behavior change.

Entire-Checkpoint: 36ead1ea2f98
- Slots: single-screen list seeded with ALL launchable agents (default
  model). Rows are selectable to Edit/Remove; "+ Add slot" adds one;
  "Done · N" finishes. Replaces the linear add-another loop.
- Master: separate step that now picks a STANDALONE master (its own
  agent + model via MasterAgent/MasterModel), chosen from installed
  text-generation-capable agents — not constrained to the worker slots.
  Only prompted when ≥2 slots.
- Task stays uniform across the whole crew (guided default); per-slot
  skills remain available via `entire scout --edit`.

Removes promptForSimpleReviewMaster + promptAddAnotherSlot; adds
promptForStandaloneMaster, promptCrewSlot, promptSlotAction.

Entire-Checkpoint: bca7cbf63255
# Conflicts:
#	cmd/entire/cli/checkpoint/open.go
#	cmd/entire/cli/labs.go
#	cmd/entire/cli/strategy/manual_commit.go
peyton-alt
peyton-alt previously approved these changes Jun 24, 2026
# Conflicts:
#	cmd/entire/cli/api/trail_types.go
dipree added 7 commits June 25, 2026 12:39
Entire-Checkpoint: ed7158e5eceb
Entire-Checkpoint: 1660a0f585ff
Entire-Checkpoint: 5c43253dac13
Entire-Checkpoint: d3623ea66317
Entire-Checkpoint: ec3ff809b3f8
# Conflicts:
#	cmd/entire/cli/review/migration.go
#	cmd/entire/cli/review/picker.go
#	cmd/entire/cli/settings/settings.go
#	cmd/entire/cli/trail_watch_cmd.go
@dipree
dipree enabled auto-merge June 25, 2026 16:28
@peyton-alt
peyton-alt self-requested a review June 26, 2026 14:42
@dipree
dipree merged commit ac6ddd0 into main Jun 26, 2026
9 checks passed
@dipree
dipree deleted the review-profiles branch June 26, 2026 14:43
peyton-alt added a commit that referenced this pull request Jul 7, 2026
…ntime-only

Product decision revisited with the A/B evidence in hand: the built-in
review brief was #1312's deliberate design, and the un-briefed variant
ran ~35% faster but missed a real finding in the measured comparison.
Depth wins — every worker (skill-bearing included) again receives the
canonical profile task, restoring #1312's reviewer behavior exactly.

What stays from the earlier change: the built-in text remains a RUNTIME
default — guided setup, first-run, and configure no longer persist it
into saved profiles (that part was a defect: baked text masquerading as
user config, freezing profiles on old wording and making the default
uneditable). Users who want a lighter pass set their own shorter task.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Entire-Checkpoint: 5ad2ee054f94
peyton-alt added a commit that referenced this pull request Jul 7, 2026
…rker, never persisted

Coverage note after review discussion: the branch no longer changes
which workers receive the canonical task (main's #1312 behavior stands:
every worker gets the user task or the built-in default). What it does
change is persistence — setup writes an empty task and the built-in
brief is applied at runtime only. Pin both halves so neither regresses:
worker briefing (skill-bearing, skill-less, user-task-verbatim) and the
runtime-only nature of the default.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Entire-Checkpoint: a457400d7876
peyton-alt added a commit that referenced this pull request Jul 7, 2026
…rker, never persisted

Coverage note after review discussion: the branch no longer changes
which workers receive the canonical task (main's #1312 behavior stands:
every worker gets the user task or the built-in default). What it does
change is persistence — setup writes an empty task and the built-in
brief is applied at runtime only. Pin both halves so neither regresses:
worker briefing (skill-bearing, skill-less, user-task-verbatim) and the
runtime-only nature of the default.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Entire-Checkpoint: a457400d7876
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants