diff --git a/docs/doperpowers/execplans/2026-07-17-implement-worker-clodex-migration.md b/docs/doperpowers/execplans/2026-07-17-implement-worker-clodex-migration.md new file mode 100644 index 0000000000..9e66d53556 --- /dev/null +++ b/docs/doperpowers/execplans/2026-07-17-implement-worker-clodex-migration.md @@ -0,0 +1,238 @@ +# Migrate implement and spike workers from the codex-CLI species to the clodex gateway route + +This ExecPlan is a living document. The sections `Progress`, `Surprises & Discoveries`, `Decision Log`, and `Outcomes & Retrospective` must be kept up to date as work proceeds. It is maintained in accordance with `skills/execplan/references/PLANS.md` from the repository root. + +## Purpose / Big Picture + +Today this repository's board pipeline runs two different kinds of unattended worker processes ("daemons"). The review side already runs every worker on one harness: a background Claude Code CLI session whose model is routed per dispatch — either through a local proxy gateway to GPT models (the "clodex" route, the default) or to plain Claude models (`engine:claude` opt-out). The implement side still runs its default workers as a *different species entirely*: detached `codex exec` processes (the OpenAI Codex CLI), with their own spawn script, their own resume mechanics, their own liveness rules, and a separate execution-doctrine text because the Codex CLI has no Skill tool and no plugin system. + +After this change, an operator dispatching an implement or spike ticket gets the same one-harness world the review loop already has: every worker is a Claude-harness daemon spawned by `daemon-spawn.sh`, and "engine" means only which *model route* the daemon rides — `codex` = the clodex gateway settings (GPT models through the local proxy), `claude` = plain Claude models. The codex CLI is no longer how implement workers run; new codex-CLI workers are not created anywhere in the pipeline. You can see it working by following the dispatch ritual in `skills/issue-tracker/SKILL.md` on a real ticket: the spawned worker appears in `claude agents`, self-reports the gateway model when probed, writes its gate verdict to the board as its first board write, and — when parked and answered — resumes through `board-answer.sh` → `daemon-resume.sh` still on the gateway (the `--settings` flag is restored from registry metadata on every resume fork). + +This was deliberately deferred scope: the 2026-07-15 review-loop rebuild (see `docs/doperpowers/execplans/2026-07-15-reviewing-prs-orchestrator-rebuild.md`, Decision Log entry six) migrated only review workers and recorded "implement workers keep their current codex-CLI species; their migration is informed by how the review loop's gateway workers behave in dog-food". That precondition is satisfied: gateway review workers were dog-fooded live on ida-solution PR #570 with sound engine/audit/wave/park behavior. The human partner confirmed proceeding on 2026-07-17, in a brainstorming grill whose decisions are all recorded in the Decision Log below. + +## Progress + +- [x] (2026-07-17 13:55Z) Brainstorming grill completed in the interactive session; roadmap decision (migration first, skill-as-protocol restructure as a separate later ExecPlan), codex-machinery decision (retire the dispatch path only, keep legacy resume/read paths, no file deletion in this change), and spawn-mechanism decision for the later restructure recorded. +- [x] (2026-07-17 14:05Z) Isolated worktree created at `.claude/worktrees/implement-worker-clodex-migration` on branch `worktree-implement-worker-clodex-migration`, branched from `origin/main` (local main == origin/main at `36bb600`). +- [x] (2026-07-17 14:20Z) Milestone 0: baseline suites green before any edit — implementing-tickets protocol suite, orchestrating-daemons daemon+codex suites, issue-tracker board suite, codex-plugin sync hermetic suite, all "all tests passed"/PASS. +- [x] (2026-07-17 14:35Z) Milestone 1: engine blocks collapsed to one `execution.md` (`git mv` of execution-claude.md; execution-codex.md removed). RED shown first (suite hard-failed on missing execution.md), then 11 single-block assertions GREEN; commit `fadf79f`. +- [x] (2026-07-17 14:45Z) Milestone 2: 6 new assertions RED first, then dispatch ritual steps 2–3 rewritten (route semantics, one `daemon-spawn.sh` command, gateway env + model `fable`, codex-spawn.sh dereferenced), implementing-tickets pieces row + spike web-reach note updated; suite GREEN; commit `35592af`. +- [x] (2026-07-17 14:50Z) Milestone 3: orchestrating-daemons SKILL.md re-framed (one-harness dispatch, codex-CLI = legacy species, codex-spawn.sh/codex-resume.sh rows marked LEGACY); daemon+codex suites still green unchanged; commit `d58cb9b`. +- [x] (2026-07-17 14:55Z) Milestone 4: dated Revision Notes appended to `2026-07-10-codex-workers-design.md` and `2026-07-09-implement-worker-autonomy-design.md`; commit `dcc1c8c`. +- [x] (2026-07-17 15:50Z) Milestone 5: live shakedown complete on SSFSKIM/doperpowers-shakedown — gateway worker spawned via the new ritual (meta: settings+effort+model=fable persisted), parked the taste fork as its FIRST board write, exposed and drove the daemon-finalize blocked-shape fix (commit `7f1814c`, test-first), resumed via `board-answer.sh` still on the gateway (in-session probe `http://localhost:8317` recorded verbatim in the PR), `[gate] re-pass — codex/direct`, PR #2 with `Closes #1` + TDD red→green Validation Evidence + `FOLLOW-UPS: none`. Evidence in Artifacts and Notes. +- [x] (2026-07-18 05:30Z) Milestone 6: reference sweep exposed the land-dispatch codex spawn (migrated test-first, commits `9d21158`/`4d4223f`/`534df62`); full verification set green (7 suites + repo shell lint + `git diff --check`); codex exit review from a disposable clone returned ONE P1 (land dedupe deadlock) — verified, fixed test-first; second independent fresh-context reviewer over the final branch returned Ready with no findings and confirmed every byte-identical constraint; branch pushed; PR opened; retrospective below. + +## Surprises & Discoveries + +- Observation: the shakedown's park→answer path failed on first try — `board-answer.sh` refused with "bound session … is mid-turn (status=working)" although the worker had cleanly parked and ended its turn. Root cause: the ended session lingered in `claude agents` as `state=blocked, status=idle`, and `daemon-finalize.sh` normalized only the `state=working, status=idle` lying shape (observed 2026-07-15); `blocked` mapped to "live", so the registry meta stayed `working` forever. The legacy codex species never hit this because it self-finalized. This would have broken EVERY gateway implement worker's park→answer flow — exactly what the live shakedown existed to catch. + Evidence: `claude agents` row `{'status': 'idle', 'state': 'blocked'}` for session c901fbcb while `daemon-finalize.sh` printed `live` and the meta read `status=working`; after the fix (commit `7f1814c`) the same invocation printed `idle` and `board-answer.sh` relayed: `relay: #1 → claude session c901fbcb (status=idle …)`, `#1: needs-human → in-progress`. +- Observation: the worker gate behaved exactly per protocol on the deliberately-unsettled taste fork: first board write was the park (`status:needs-human` label + `[gate] fail` comment naming the greeting decision, with a recommended answer and orientation summary), no code written. + Evidence: issue #1 labels `["enhancement","status:needs-human","priority:P3"]`; comment "[gate] fail — needs-human: the exact no-argument greeting is an unresolved product-wording decision required by acceptance." + +## Decision Log + +- Decision: migrate first, restructure later — this ExecPlan covers only the species migration; the skill-as-protocol restructure of `skills/implementing-tickets` (mirroring the reviewing-prs SKILL.md-is-the-protocol shape, bootstrap-file spawn) is a separate, subsequent ExecPlan. + Rationale: both work items touch the same files; migrating first collapses the two-worker-species fork so the restructure designs against a single harness instead of baking in a structure that would immediately be deleted. Rejected alternative: restructure first (double work on engine blocks); one combined plan (weaker landing points, oversized PR). + Date/Author: 2026-07-17, human-confirmed in the brainstorming grill. +- Decision: retire the codex-CLI *dispatch path* only. The dispatch ritual stops referencing `codex-spawn.sh`; no pipeline path creates new codex-CLI workers. `codex-spawn.sh`, `codex-resume.sh`, `_codex_lib.sh`, the codex branch in `board-answer.sh`, the codex legacy read/kill paths in `daemon-retire.sh` / review-loop scripts, and `tests/orchestrating-daemons/test-codex-scripts.sh` all remain untouched. Full deletion is a separate follow-up once no bound codex session exists anywhere (consumer repos may hold parked tickets bound to live codex sessions). + Rationale: mirrors how the review loop retired the species (dispatch stopped, read paths stayed); deleting resume machinery now could strand an existing parked worker. Rejected alternatives: full deletion now (risk to live bound sessions, larger test surgery); leaving the ritual pointing at codex-spawn (keeps minting the very species being retired). + Date/Author: 2026-07-17, human-confirmed in the brainstorming grill. +- Decision: engine labels keep their names and their resolution order (ticket label `engine:claude`/`engine:codex` → `$WORKER_ENGINE` → default `codex`) but are redefined as the model ROUTE of a single Claude-harness worker: `codex` = gateway settings (GPT models via the local proxy), `claude` = plain Claude models. + Rationale: exact symmetry with the review loop (`review-dispatch.sh` already implements this semantics); renaming the labels would break existing tickets and cross-repo habits for zero behavior gain. + Date/Author: 2026-07-17, human-confirmed in the brainstorming grill. +- Decision: the two implement execution blocks (`execution-claude.md`, `execution-codex.md`) collapse into ONE file, `skills/implementing-tickets/references/engine-blocks/execution.md`, based on the claude block's content (plugin-skill references like `doperpowers:execplan`, `doperpowers:test-driven-development`). The codex block's `.agents/skills` vendored-doctrine pointer is dropped from the block. + Rationale: the split existed only because the Codex CLI lacked the Skill tool and plugin system; a gateway worker IS a Claude-harness session — plugin skills resolve natively regardless of which model the session rides. Keeping a per-route doctrine fork would preserve a distinction with no referent. The file keeps its `engine-blocks/` directory (path stability for the ritual text and future blocks). Rejected alternative: keep two blocks with identical content (drift hazard, lies about a difference that no longer exists). + Date/Author: 2026-07-17, settled during grill follow-through (mechanical consequence of one harness). +- Decision: the live shakedown (Milestone 5) runs against a scratch GitHub repository with issues enabled, created for the purpose, not against the ida-solution consumer board. + Rationale: this repo's own GitHub issues are disabled, and coupling a plugin migration's first live run to a production consumer board mixes concerns; the review-loop migration validated its gateway mechanics with probe sessions first, then dog-fooded on the consumer. Consumer dog-food happens naturally on the next real ida-solution dispatch after release. Rejected alternative: shakedown directly on ida-solution (real stakes on an unproven path; a stuck worker there costs a wake-queue cleanup). + Date/Author: 2026-07-17, resolved autonomously while authoring (feasibility, not taste: both ends were sanctioned by the grill's evidence-first requirement). +- Decision: the gate-comment format `[gate] pass — {{ENGINE_NAME}}/` is unchanged; `ENGINE_NAME` now names the route (`codex` or `claude`). + Rationale: the review worker's compliance audit and the board's history parse this line; the route name carries the same operational information (which model family built this). + Date/Author: 2026-07-17, authoring. +- Decision: mid-execution scope addition — `skills/orchestrating-daemons/scripts/daemon-finalize.sh` gains a second lying-shape normalization (`state=blocked, status=idle` → ended, finalized `idle`, reply recorded through the blocked renderer so a pending AskUserQuestion still surfaces). This amends the plan's original "substrate scripts byte-identical" intent (which was about the LEGACY codex machinery; daemon-finalize is claude-species machinery). + Rationale: discovered live in Milestone 5 — without it, every gateway implement worker's park→answer relay dead-ends on a stale `working` meta, because claude-species `--no-wait` daemons have no self-finalizer and `board-answer.sh`'s built-in finalize call read the lingering `blocked` state as a live turn. Keeping the harness `status` field as the single turn signal matches the script's own documented doctrine. Rejected alternative: documenting a manual `daemon-finalize` step in the wake ritual (pushes a mechanical reconciliation onto the human; the relay already calls finalize — it just missed this shape). + Date/Author: 2026-07-17, during Milestone 5. +- Decision: second mid-execution scope addition — `skills/reviewing-prs/scripts/land-dispatch.sh` migrates in this plan too: its default engine spawned the land worker via `codex-spawn.sh`, making it a second live consumer of the codex-CLI spawn path (the grill's "implement dispatch is the last consumer" premise was wrong). The spawn branch now mirrors `review-dispatch.sh`'s gateway route (`DAEMON_CLAUDE_SETTINGS`/`DAEMON_CLAUDE_EFFORT`, model `fable`), test-first in `tests/reviewing-prs/test-land-dispatch.sh`. + Rationale: leaving it would contradict the settled decision "no new codex-CLI workers anywhere" and the rewritten orchestrating-daemons doctrine ("no pipeline path creates new ones") — and the operation manual already claimed the land worker is "same daemon machinery, not a third species", so the code was out of line with its own documentation. Rejected alternative: registering it as a follow-up (ships a doctrine the code visibly violates). + Date/Author: 2026-07-17, during Milestone 6's reference sweep. + +## Outcomes & Retrospective + +Achieved, against the original purpose: the board pipeline is now genuinely one worker species end-to-end. Implement, spike, AND land workers all spawn as Claude-harness daemons through `daemon-spawn.sh`; the engine label survives purely as a model route (gateway GPT vs plain Claude); the codex CLI has no remaining spawn path anywhere in the pipeline, while every legacy read/resume path stayed byte-identical for existing bound sessions. The migration was proven live, not just hermetically: a gateway implement worker ran the full protocol lifecycle on a scratch board — gate-park on a taste fork as its first board write, `board-answer.sh` resume that kept the gateway (in-session probe `http://localhost:8317` recorded in the PR it opened), `[gate] re-pass — codex/direct`, and a closing PR with TDD red→green evidence. + +The plan grew twice mid-flight, both times through the mechanisms designed for it: (1) the live shakedown caught that claude-species daemons finalize differently from the self-finalizing codex species — the `state=blocked/status=idle` lingering shape would have dead-ended every gateway worker's park→answer relay (fixed in `daemon-finalize.sh`, test-first); (2) the exit reviews caught that the grill's "implement dispatch is the last codex-spawn consumer" premise was false (land-dispatch was a second consumer, migrated here) and that the migrated lander needed the same finalize-before-dedupe discipline review-dispatch already had (the codex review's single P1, fixed test-first). Lesson worth keeping: when a species with self-finalizing lifecycle semantics is replaced by one without them, audit every consumer that *reads* the lifecycle fields, not just the spawn sites — both scope additions were exactly that class of gap. + +Remaining, deliberately out of scope: full deletion of the legacy codex machinery once no bound codex session exists anywhere; the skill-as-protocol restructure of `skills/implementing-tickets` (work item 2 of the 2026-07-17 roadmap, next ExecPlan); a plugin release carrying both work items to consumers. One operational leftover: the scratch repo `SSFSKIM/doperpowers-shakedown` could not be deleted (gh token lacks the `delete_repo` scope) — delete it manually or after `gh auth refresh -h github.com -s delete_repo`. + +## Context and Orientation + +All paths are relative to the repository root (`doperpowers`, a fork of obra/superpowers — a plugin of agent "skills": markdown behavior-protocols loaded by coding agents). The pieces that matter here: + +**The board pipeline.** Work is tracked as GitHub issues ("tickets") managed by the scripts in `skills/issue-tracker/scripts/` (`board-list.sh`, `board-register.sh`, `board-transition.sh`, `board-bind.sh`, `board-answer.sh`, `board-reconcile.sh`). An operator (a human's interactive Claude session) dispatches a ticket by following "The dispatch ritual" in `skills/issue-tracker/SKILL.md`: render a worker-protocol markdown file into a prompt (substituting `{{PLACEHOLDER}}` tokens), spawn a background worker with that prompt, and bind the worker's session id to the ticket (`board-bind.sh`). The worker gates the ticket, builds, and opens a PR; the review loop (`skills/reviewing-prs/`) takes it from there. + +**Daemons.** A daemon is a durable background worker session tracked in a registry at `~/.claude/orchestrating-daemons`. The substrate scripts live in `skills/orchestrating-daemons/scripts/`. Two species exist today: (a) Claude-harness daemons — background Claude Code CLI sessions spawned by `daemon-spawn.sh [cwd] [worktree] [model]`, resumed by `daemon-resume.sh` (a resume *forks* a new session that carries the conversation; the registry chains ids); (b) codex-CLI daemons — detached `codex exec --json` processes spawned by `codex-spawn.sh`, resumed by `codex-resume.sh` (one session id for life), sharing the same registry with `engine: codex` metadata. This plan retires species (b) from the *dispatch ritual*; its scripts remain as legacy machinery. + +**The gateway ("clodex") route.** `~/.claude/clodex-settings.json` is a Claude Code `--settings` file that points `ANTHROPIC_BASE_URL` at a local proxy (`http://localhost:8317`) which maps Claude model aliases (fable/opus/sonnet) to GPT models. A Claude-harness daemon spawned with these settings is a full Claude Code session (Skill tool, Task subagents, resume machinery, plugin skills) whose *model* is a GPT model. `daemon-spawn.sh` already supports this: the env vars `DAEMON_CLAUDE_SETTINGS` (a `--settings` path) and `DAEMON_CLAUDE_EFFORT` (an `--effort` value) are forwarded to the spawn AND persisted into the daemon's registry metadata (`settings`, `effort` keys); `daemon-resume.sh` reads them back and re-passes `--settings`/`--effort` on every resume fork — without that, a gateway daemon silently reverts to plain Claude models on its first resume. This substrate is proven: the review loop has dispatched gateway workers through it since 2026-07-15 (see `skills/reviewing-prs/scripts/review-dispatch.sh`, the `engine = codex` branch around line 364: `DAEMON_CLAUDE_SETTINGS="${CLODEX_SETTINGS:-$HOME/.claude/clodex-settings.json}" DAEMON_CLAUDE_EFFORT="${CLODEX_EFFORT:-xhigh}" daemon-spawn.sh --no-wait "" "${REVIEW_MODEL:-fable}"`), and it was verified live that a resume fork with `--settings` stays on the gateway while one without silently reverts. + +**The implement worker protocol.** `skills/implementing-tickets/references/implement-worker-protocol.md` is the text rendered into an implement worker's spawn prompt. It contains a `{{EXECUTION_BLOCK}}` placeholder which the dispatch ritual fills from a per-engine file in `skills/implementing-tickets/references/engine-blocks/`: currently `execution-claude.md` (references doperpowers plugin skills directly) or `execution-codex.md` (same discipline, but routes skill doctrine through a `.agents/skills` vendored copy because the codex CLI cannot load plugins). `references/spike-worker-protocol.md` is rendered instead for tickets whose category is `spike`; it has no execution block. Both protocols carry an `{{ENGINE_NAME}}` placeholder used in the worker's `[gate] pass — /` board comment. + +**Tests are prose-invariant asserts.** `tests/implementing-tickets/test-protocol-content.sh` greps the protocol and skill files for load-bearing clauses (this repo treats skill prose as behavior — the tests pin it). Its "engine blocks:" section currently asserts both per-engine files exist with specific content. `tests/orchestrating-daemons/test-codex-scripts.sh` and `test-daemon-scripts.sh` are hermetic integration tests over the substrate scripts (stub `codex`/`claude` binaries on PATH). `tests/issue-tracker/test-board-scripts.sh` covers the board scripts with a mock `gh`. `tests/codex-plugin-sync/test-sync-to-codex-plugin.sh` is a hermetic regression over the outward distribution script. + +**What already reads correctly and must NOT be touched.** The review loop's legacy codex read paths (`review-dispatch.sh` pid-liveness for old codex metas, `land-dispatch.sh` equivalents), `board-answer.sh`'s codex branch (`engine == codex → codex-resume.sh`), `daemon-retire.sh`'s codex kill path, and the whole `test-codex-scripts.sh` suite: these are the legacy machinery the grill decided to keep. `RELEASE-NOTES.md` is history — never edited retroactively. + +## Plan of Work + +Milestone 0 — baseline. In the worktree, run the four suites named in Concrete Steps and record that they pass before any edit. This distinguishes pre-existing breakage from breakage this plan introduces. + +Milestone 1 — collapse the engine blocks (test-first). Edit `tests/implementing-tickets/test-protocol-content.sh`: replace the "engine blocks:" section's two-file assertions with single-file assertions over `skills/implementing-tickets/references/engine-blocks/execution.md` — assert the file exists; assert it contains `doperpowers:execplan`, `EXECPLAN:`, `writing-plans`, `subagent-driven-development`, `claim completion on reasoning alone`, `big-but-atomic`; assert it does NOT contain `.agents/skills` (the codex-CLI-only vendored-doctrine pointer must not survive into the one-harness block), `work ALONE`, or `YOURSELF`. Run the suite — the new assertions must FAIL against the still-two-file tree (RED). Then: create `execution.md` with the content of today's `execution-claude.md` (reproduced in Artifacts and Notes below so this plan is self-contained), delete `execution-claude.md` and `execution-codex.md`, and re-run the suite (GREEN). The implement protocol itself (`implement-worker-protocol.md`) does not change — `{{EXECUTION_BLOCK}}` and `{{ENGINE_NAME}}` remain placeholders; only what the ritual substitutes into them changes. + +Milestone 2 — rewrite the dispatch ritual and the implementing-tickets doctrine. In `skills/issue-tracker/SKILL.md`, "The dispatch ritual" section: step 2 currently reads "Resolve the ENGINE — ticket label `engine:claude`/`engine:codex` → `$WORKER_ENGINE` → default `codex`" and later binds `EXECUTION_BLOCK` to "the engine's `references/engine-blocks/execution-.md`". Rewrite step 2 so the resolution order is unchanged but the meaning is stated as the model route of one Claude-harness worker (codex = clodex gateway settings / GPT models, claude = plain Claude models), and `EXECUTION_BLOCK` binds to the single `references/engine-blocks/execution.md`. Step 3 currently forks on species: "codex: `codex-spawn.sh …` (model/effort default gpt-5.6-sol/high — override with `$CODEX_MODEL`/`$CODEX_EFFORT` as args 5–6). claude: `daemon-spawn.sh …`". Rewrite it to ONE spawn command for both routes: `daemon-spawn.sh "-" "" ` — with the codex route prefixing the environment `DAEMON_CLAUDE_SETTINGS="${CLODEX_SETTINGS:-$HOME/.claude/clodex-settings.json}" DAEMON_CLAUDE_EFFORT="${CLODEX_EFFORT:-xhigh}"` and passing model `fable` (arg 5), the claude route passing no env and leaving model to inherit — exactly the review loop's convention, stated in the ritual's prose. Remove every `codex-spawn.sh` mention from this file. In `skills/implementing-tickets/SKILL.md`: rewrite the pieces-table row for `references/engine-blocks/` (currently "per-engine EXECUTION text (claude: TDD/execplan skills; codex: the same discipline via the vendored `.agents/skills` doctrine)") to describe the single execution block composed into the protocol at render time, and note that both routes are Claude-harness sessions. Update the spike-lane bullet "Research-heavy spikes often want `engine:claude` (web reach); the label mechanism is unchanged" — web reach is now harness-level (every worker has it); the engine label is purely a model-route preference. Add test-first assertions for the load-bearing new prose (see Validation) to `test-protocol-content.sh` in the same RED→GREEN discipline: at minimum, assert the issue-tracker SKILL no longer contains `codex-spawn.sh` — note the file is in a *different* skill directory, so the test must derive its path from `$REPO_ROOT/skills/issue-tracker/SKILL.md`. + +Milestone 3 — re-frame the substrate doc. `skills/orchestrating-daemons/SKILL.md`: the overview paragraph currently says "The substrate drives two engines under one registry: `claude --bg` daemons and detached `codex exec` workers (`engine: codex` in the meta) — the board pipeline picks per dispatch (label → `WORKER_ENGINE` → codex)." Rewrite: the board pipeline dispatches Claude-harness daemons only, with the engine label selecting the model route (gateway settings vs plain); detached `codex exec` workers are a legacy species — existing registry metas remain readable/resumable/retirable, no pipeline path creates new ones. Mark the `codex-spawn.sh` pieces-table row "(legacy — retired from board dispatch; kept until no bound codex sessions remain)" and leave the `codex-resume.sh` row as the legacy-session continuation path. + +Milestone 4 — living-spec revision notes. Append a dated entry to `## Revision Notes` in `docs/doperpowers/specs/2026-07-10-codex-workers-design.md` (the spec that created the codex-CLI implement-worker species and the per-engine blocks): the species is retired from dispatch as of this plan; engine labels now name model routes; the vendored-doctrine execution block is gone; legacy machinery retained pending a separate deletion follow-up; pointer to this ExecPlan. Same for `docs/doperpowers/specs/2026-07-09-implement-worker-autonomy-design.md` (its engine-block description changed). Do not rewrite historical body text — revision notes only, per the living-spec doctrine (`skills/execspec/`). + +Milestone 5 — live shakedown (evidence, not hope). Create a throwaway GitHub repo (issues enabled) with a trivial code surface, register one synthetic P3 ticket via `board-register.sh` with a fully-authored body (a change small enough for a direct-mode build, e.g. "add a --version flag to the hello script; print 0.1.0"), then perform the NEW ritual by hand exactly as written: render the protocol with the single execution block, spawn via `daemon-spawn.sh` with the gateway env, `board-bind.sh` the printed uuid. Observe and record: (1) the worker's registry meta contains `settings` and `effort` keys (inspect the meta file under `~/.claude/orchestrating-daemons`); (2) the worker's first board write is `in-progress` + a `[gate] pass — codex/` comment; (3) a deterministic gateway probe from inside the worker's turn — ask it (or have the ticket require it to log) `echo ${ANTHROPIC_BASE_URL:-unset}` — shows the local proxy, not unset (self-reported model ids are context-echo-prone; trust the env probe); (4) park/resume: post a follow-up via `board-answer.sh` and confirm the resumed fork still rides the gateway (repeat the env probe; also confirm `daemon-resume.sh` passed `--settings` by checking the new session's spawn args in the registry). (5) the PR closes the loop (`Closes #N`, `## Validation Evidence` present). If the gateway proxy is down (`curl -s http://localhost:8317` fails), restart it (`brew services restart cliproxyapi` — its credential cooldowns are in-memory) before concluding anything about the pipeline. Tear the scratch repo down afterwards; paste the key transcript lines into Artifacts and Notes. + +Milestone 6 — verification, exit review, PR. Run the full suite set (Concrete Steps). Then the exit gate: from an EMPTY temp directory, `git clone` this worktree's repo, check out the branch, and run `codex exec review --base main` there (never in-place — the codex review agent wanders the filesystem and picks up neighboring git contexts; also never on a dirty tree). Read only what follows "Final review comments:"; triage every finding (fix or record why not). Then push the branch and open a PR against `main` titled for the migration, and finish with doperpowers:finishing-a-development-branch — its retrospective step writes this plan's `Outcomes & Retrospective`. + +## Concrete Steps + +All commands run from the worktree root (`.claude/worktrees/implement-worker-clodex-migration` under the repo, itself a full checkout). + +Baseline and per-milestone verification: + + tests/implementing-tickets/test-protocol-content.sh + tests/orchestrating-daemons/test-daemon-scripts.sh + tests/orchestrating-daemons/test-codex-scripts.sh + tests/issue-tracker/test-board-scripts.sh + tests/codex-plugin-sync/test-sync-to-codex-plugin.sh + scripts/lint-shell.sh + git diff --check + +Each prints per-assert `[PASS]`/`[FAIL]` lines and exits non-zero on any failure; "all tests passed" (or the suite's equivalent summary) is the success marker. `test-codex-scripts.sh` must stay green UNCHANGED — it now covers legacy machinery, and a failure there means this plan touched something the grill said to keep. + +Milestone 1 file operations: + + git mv skills/implementing-tickets/references/engine-blocks/execution-claude.md \ + skills/implementing-tickets/references/engine-blocks/execution.md + git rm skills/implementing-tickets/references/engine-blocks/execution-codex.md + +(Then edit `execution.md` per Artifacts and Notes — the claude block is already plugin-skill-based; verify no `.agents/skills` reference remains.) + +Shakedown scaffolding (Milestone 5), sketch: + + gh repo create /doperpowers-shakedown --private --add-readme + # enable issues (on by default), clone to a scratch dir, add hello script, push + SKILLDIR=/skills/issue-tracker/scripts + "$SKILLDIR/board-register.sh" "hello: add --version flag" enhancement P3 --body-file + # render prompt per the NEW ritual, then: + DAEMON_CLAUDE_SETTINGS="$HOME/.claude/clodex-settings.json" DAEMON_CLAUDE_EFFORT=xhigh \ + /skills/orchestrating-daemons/scripts/daemon-spawn.sh --no-wait \ + "1-hello-version" "" issue-1 fable + "$SKILLDIR/board-bind.sh" 1 + +Commit at every green point with small, described commits (no Co-Authored-By lines, per repo convention). + +## Validation and Acceptance + +Acceptance is behavior, observed twice over — hermetically and live: + +1. `tests/implementing-tickets/test-protocol-content.sh` passes and now asserts: exactly one execution block exists (`references/engine-blocks/execution.md`); it routes execplan mode through `doperpowers:execplan`; it does not reference `.agents/skills`; the implement protocol's placeholder set is unchanged (`{{EXECUTION_BLOCK}}`, `{{ENGINE_NAME}}`, … — the `want=` line in the test does not change); `skills/issue-tracker/SKILL.md` contains no `codex-spawn.sh` reference; the two engine-block files `execution-claude.md`/`execution-codex.md` no longer exist. Before Milestone 1's edits, the new assertions fail (RED shown in the transcript); after, the whole suite passes. +2. `tests/orchestrating-daemons/test-codex-scripts.sh` and `test-daemon-scripts.sh` pass with zero modifications to either suite or to any script under `skills/orchestrating-daemons/scripts/`. +3. Reading `skills/issue-tracker/SKILL.md`'s dispatch ritual end-to-end, a novice operator would spawn BOTH routes through `daemon-spawn.sh`; the word `codex-spawn` does not appear in the file; the codex route's command line includes `DAEMON_CLAUDE_SETTINGS` and model `fable`. +4. The live shakedown (Milestone 5) shows, in order: a spawned worker whose registry meta carries `settings`/`effort`; `[gate] pass — codex/…` as the ticket's first non-registration board event; an in-worker env probe printing the local proxy URL; a `board-answer.sh` resume whose forked session still probes to the proxy URL; a PR with `Closes #N`. Every one of these is a transcript/board artifact pasted into this plan. +5. `scripts/lint-shell.sh` and `git diff --check` are clean; the codex-plugin sync hermetic suite passes (the sync script mirrors skills — the engine-block rename must not break it). + +## Idempotence and Recovery + +Every edit is a tracked-file change on an isolated branch; re-running a milestone means re-applying edits that are already there (no-ops) and re-running suites (read-only). The engine-block collapse is a `git mv` + `git rm` — recoverable with `git checkout` until commit, `git revert` after. The shakedown's writes land only in a throwaway repo and the daemon registry: retire the worker (`daemon-retire.sh `) and delete the scratch repo (`gh repo delete`) to clean up; a failed shakedown leaves the main repo untouched. If the gateway proxy is refusing (in-memory credential cooldown after a quota reset), `brew services restart cliproxyapi` and re-probe before drawing conclusions. If a resume fork ever probes to plain Anthropic instead of the proxy, that is the known settings-drop failure — check that the registry meta still has the `settings` key and that the resume went through `daemon-resume.sh` (not a hand-rolled `--resume`). + +## Artifacts and Notes + +The single execution block, `skills/implementing-tickets/references/engine-blocks/execution.md`, verbatim (this is today's `execution-claude.md`, which is already correct for a Claude-harness worker on either model route — carried unchanged; the codex variant's `.agents/skills` plumbing is the only content dropped from the union): + + EXECUTION (gate passed) — name the mode in the gate comment. + Every claim of done carries EVIDENCE appropriate to the change — never + claim completion on reasoning alone: + - testable logic: TDD (doperpowers:test-driven-development) — failing + test first. Green checks are what keep your PR self-merge-eligible. + - UI/visual changes: build + run it — verify the actual rendered + behavior (E2E where the repo has it); write tests only where behavior + is assertable without theater. + - config/docs/infra: the relevant check (build, lint, dry-run) passes. + Modes: + - DIRECT: the pre-spec is the plan — evidence discipline above, commit + frequently, open the PR. + - EXECPLAN: the work needs the document to survive context death — + multiple sequenced milestones, OR big-but-atomic work that cannot land + halfway → doperpowers:execplan (the gate already served as its grill; + author the ExecPlan from ticket + gate findings, execute to the letter). + Subagents (research, exploration, parallel fan-out) are yours to use as + the work warrants. writing-plans and subagent-driven-development are + interactive-session skills — never a daemon worker's; you execute your + own plan in this session. + +Current dispatch-ritual step 3 (`skills/issue-tracker/SKILL.md`), the text being replaced — kept here so a reader can locate it without git history: + + 3. codex: `codex-spawn.sh "-" "" ` + (model/effort default gpt-5.6-sol/high — override with + `$CODEX_MODEL` / `$CODEX_EFFORT` as args 5–6). claude: + `daemon-spawn.sh "-" "" `. Both + from `orchestrating-daemons` — always a worktree; workers write code. + +Shakedown evidence (Milestone 5, all from SSFSKIM/doperpowers-shakedown — repo deleted after; transcripts preserved here): + + # spawn (new ritual, gateway route): + daemon spawned (no-wait): 1-hello-version [c901fbcb / c901fbcb-...] status=working + # registry meta after spawn: + {'name': '1-hello-version', 'model': 'fable', + 'settings': '/Users/new/.claude/clodex-settings.json', 'effort': 'xhigh', ...} + # first board write = the gate verdict (park, no code): + labels: ["enhancement","status:needs-human","priority:P3"] + "[gate] fail — needs-human: the exact no-argument greeting is an + unresolved product-wording decision required by acceptance." (+ recommended + answer `hello`, + orientation summary) + # answer relay (after the daemon-finalize fix): + relay: #1 → claude session c901fbcb (status=idle, ...) + #1: needs-human → in-progress + # resumed fork still on the gateway — worker's in-session probe, verbatim + # from PR #2's Validation Evidence: + echo "${ANTHROPIC_BASE_URL:-unset}" → http://localhost:8317 + # gate re-verdict and closure: + "[gate] re-pass — codex/direct: the human approved the exact default + greeting `hello` ..." + PR #2 "Add hello.sh version flag": Closes #1; TDD red phase recorded + (test.sh exited 1 before, 0 after); FOLLOW-UPS: none; ticket → in-review. + +Exit-review verdict (Milestone 6): `codex exec review --base main` from a +disposable clone returned exactly ONE finding — + + [P1] Finalize gateway landers before deduping — land-dispatch.sh + After a default dry-run lander finishes, daemon-spawn --no-wait leaves + its registry status as `working` ... the existing dedupe path treats any + listed Claude session as active and exits with `skip active land + worker`, so the live landing never starts. + +Verified real (the retired codex lander self-finalized; the gateway lander +does not) and fixed test-first: the dedupe's `working|blocked` arm now runs +`daemon-finalize.sh` after `_is_live` passes for a claude-species lander and +skips only on a genuinely `live` verdict (commits `4d4223f`, `534df62`); the +host-awareness semantics of `_is_live` are unchanged, and codex legacy metas +keep the pid-liveness path. A second independent review (fresh-context Claude +reviewer subagent over the full final branch) covered the post-codex-review +commits; its verdict is recorded in Outcomes & Retrospective. + +## Interfaces and Dependencies + +No new code interfaces. The contract surfaces at the end of this plan: + +- `skills/implementing-tickets/references/engine-blocks/execution.md` exists; `execution-claude.md` and `execution-codex.md` do not. +- `skills/issue-tracker/SKILL.md` dispatch ritual: engine = model route; both routes spawn via `skills/orchestrating-daemons/scripts/daemon-spawn.sh`; the codex route's documented invocation carries `DAEMON_CLAUDE_SETTINGS` (default `${CLODEX_SETTINGS:-$HOME/.claude/clodex-settings.json}`), `DAEMON_CLAUDE_EFFORT` (default `${CLODEX_EFFORT:-xhigh}`), and model `fable`. +- `skills/orchestrating-daemons/scripts/*` byte-identical to `main`, with ONE exception decided mid-flight (Decision Log): `daemon-finalize.sh` normalizes the ended `state=blocked, status=idle` shape to a finalized-idle meta. All LEGACY codex machinery (`codex-spawn.sh`, `codex-resume.sh`, `_codex_lib.sh`) is byte-identical. +- `skills/implementing-tickets/references/implement-worker-protocol.md` and `spike-worker-protocol.md` byte-identical to `main` (placeholder sets unchanged). +- External dependencies, unchanged from the review loop's: the local cliproxy gateway on `localhost:8317`, `~/.claude/clodex-settings.json`, the `claude` CLI daemon supervisor, `gh`. + +## Revision Notes + +- 2026-07-17 (authoring): initial plan, authored from the brainstorming grill of the same date. diff --git a/docs/doperpowers/specs/2026-07-09-implement-worker-autonomy-design.md b/docs/doperpowers/specs/2026-07-09-implement-worker-autonomy-design.md index c493e2b227..debc64179a 100644 --- a/docs/doperpowers/specs/2026-07-09-implement-worker-autonomy-design.md +++ b/docs/doperpowers/specs/2026-07-09-implement-worker-autonomy-design.md @@ -646,3 +646,9 @@ of the plan. routes explicitly to the vendored doperpowers:execplan doctrine (one self-contained plan, executed in-thread by the worker itself) and both blocks forbid sub-agent/collab fan-out by name. +- 2026-07-17: the two per-engine execution blocks described above collapsed + into one `references/engine-blocks/execution.md` — implement/spike workers + migrated to the one-harness model (Claude-harness daemon; engine label = + model route via the clodex gateway), so the codex-CLI block and its + vendored-doctrine routing no longer exist. See + `docs/doperpowers/execplans/2026-07-17-implement-worker-clodex-migration.md`. diff --git a/docs/doperpowers/specs/2026-07-10-codex-workers-design.md b/docs/doperpowers/specs/2026-07-10-codex-workers-design.md index 1f5db86437..3b65cabe83 100644 --- a/docs/doperpowers/specs/2026-07-10-codex-workers-design.md +++ b/docs/doperpowers/specs/2026-07-10-codex-workers-design.md @@ -731,3 +731,16 @@ mechanical rather than lucky. **superseded** by `specs/2026-07-12-native-review-recovery-design.md`: one native `codex exec review` engine for both species, owned by a substrate script. +- 2026-07-17: The codex-CLI implement/spike worker species this spec created + is **retired from dispatch** (ExecPlan: + `docs/doperpowers/execplans/2026-07-17-implement-worker-clodex-migration.md`). + The dispatch ritual now spawns every implement/spike worker as a + Claude-harness daemon via `daemon-spawn.sh`; the engine label survives as + a model ROUTE (`codex` = clodex gateway settings/GPT models, + `claude` = plain Claude models), mirroring the review loop's 2026-07-15 + migration. The per-engine execution blocks collapsed into one + `references/engine-blocks/execution.md` (the `.agents/skills` + vendored-doctrine pointer was codex-CLI-only and is gone with it). + `codex-spawn.sh`/`codex-resume.sh` and the codex read/kill paths remain + as legacy machinery for existing bound sessions; full deletion is a + separate follow-up. diff --git a/skills/implementing-tickets/SKILL.md b/skills/implementing-tickets/SKILL.md index 094877ff89..46c4fea931 100644 --- a/skills/implementing-tickets/SKILL.md +++ b/skills/implementing-tickets/SKILL.md @@ -23,7 +23,7 @@ audit trail, not requests. Full design + rationale: | `references/implement-worker-protocol.md` | the Implement Worker Protocol — rendered (`{{PLACEHOLDERS}}`) into every spawn prompt | | `references/spike-worker-protocol.md` | the Spike Worker Protocol — rendered instead when the ticket's category is `spike` (the exploration lane below) | | `references/implement-decompose.md` | runtime-opened decomposition procedure — the protocol carries only a pointer (`{{DECOMPOSE_DOC}}` = absolute path); the worker opens it when Check-2 says decompose. Conditional-large protocol blocks live this way: procedure in a plugin file, instance facts in the prompt | -| `references/engine-blocks/` | per-engine EXECUTION text (claude: TDD/execplan skills; codex: the same discipline via the vendored `.agents/skills` doctrine) — both keep interactive-session skills (writing-plans, subagent-driven-development) out of daemon workers; composed into the protocol at render time (implement protocol only — spikes are exploration, not TDD) | +| `references/engine-blocks/execution.md` | the EXECUTION text — one block for both model routes, since every worker is a Claude-harness session (the engine label picks the model route: `codex` = clodex gateway/GPT, `claude` = plain Claude); keeps interactive-session skills (writing-plans, subagent-driven-development) out of daemon workers; composed into the protocol at render time (implement protocol only — spikes are exploration, not TDD) | | The Ticket Gate | the pre-code pass/park verdict (below) | | board schema + dispatch ritual | owned by doperpowers:issue-tracker (states, scripts, the mechanical ritual, the wake ritual) | | `scripts/` | empty this phase — the auto-attach trigger (`implement-dispatch.sh` + workflow template) lands here next phase | @@ -141,8 +141,9 @@ What changes and what doesn't: registered `--spawned-by ` with honest gate-triage against the IMPLEMENT gate; murkier outcomes stay a Recommendation line for the human. -- Research-heavy spikes often want `engine:claude` (web reach); the label - mechanism is unchanged. +- The engine label picks a model route only (web reach is harness-level — + every worker has it); choose per the work's model fit, mechanism + unchanged. - Category labels are plain words by design (`bug`/`enhancement` always were) — in a consumer repo that already used a descriptive `spike` label, existing tickets carrying it now read as spike-lane tickets: diff --git a/skills/implementing-tickets/references/engine-blocks/execution-codex.md b/skills/implementing-tickets/references/engine-blocks/execution-codex.md deleted file mode 100644 index 44fef98175..0000000000 --- a/skills/implementing-tickets/references/engine-blocks/execution-codex.md +++ /dev/null @@ -1,29 +0,0 @@ -EXECUTION (gate passed) — name the mode in the gate comment. -Every claim of done carries EVIDENCE appropriate to the change — never -claim completion on reasoning alone: -- testable logic (functions, routes, data handling): failing test first — - run it, watch it fail, implement the minimal code, watch it pass, - commit. Green checks are what keep your PR self-merge-eligible. -- UI/visual changes: build + run it — verify the actual rendered - behavior (E2E where the repo has it); write tests only where behavior - is assertable without theater. -- config/docs/infra: the relevant check (build, lint, dry-run) passes. -Modes: -- DIRECT: the pre-spec is the plan — implement with the evidence - discipline above, small frequent commits, open the PR with gh. -- EXECPLAN: the work needs a document to survive context death — multiple - sequenced milestones, OR big-but-atomic work that cannot land halfway → - follow the doperpowers:execplan doctrine (vendored at - .agents/skills/execplan): author ONE self-contained ExecPlan as - docs/plans/issue-{{ISSUE_NUMBER}}.md on your branch (milestones with - observable acceptance criteria, exact files per milestone), commit it, - then execute it to the letter, milestone by milestone — same - evidence discipline within each. -Sub-agents and collab threads (research, exploration, parallel fan-out) -are yours to use as the work warrants. writing-plans and -subagent-driven-development are interactive-session skills — never a -daemon worker's; you execute your own plan in this thread. -The full doperpowers skill doctrine behind this summary is vendored at -`.agents/skills/` in your workspace (test-driven-development, execplan, -verification-before-completion, systematic-debugging, …) — read the -relevant SKILL.md when the summary above is not enough. diff --git a/skills/implementing-tickets/references/engine-blocks/execution-claude.md b/skills/implementing-tickets/references/engine-blocks/execution.md similarity index 100% rename from skills/implementing-tickets/references/engine-blocks/execution-claude.md rename to skills/implementing-tickets/references/engine-blocks/execution.md diff --git a/skills/issue-tracker/SKILL.md b/skills/issue-tracker/SKILL.md index 4ed5fd7494..20a05df02b 100644 --- a/skills/issue-tracker/SKILL.md +++ b/skills/issue-tracker/SKILL.md @@ -140,7 +140,10 @@ pick by repo visibility: normal. Derived from GitHub PR state on every snapshot — never a label, never auto-closed. 2. Resolve the ENGINE — ticket label `engine:claude`/`engine:codex` → - `$WORKER_ENGINE` → default `codex`. Render the worker protocol + `$WORKER_ENGINE` → default `codex`. Every worker is ONE species — a + Claude-harness daemon; the engine names only its model route (`codex` = + the clodex gateway settings, GPT models through the local proxy; + `claude` = plain Claude models). Render the worker protocol (`doperpowers:implementing-tickets`): category `spike` → `references/spike-worker-protocol.md`, else `references/implement-worker-protocol.md`. Substitute every @@ -149,17 +152,21 @@ pick by repo visibility: issue body from `gh issue view --json body`, `ENGINE_NAME` = the engine, `REPO_FACTS` = `git show origin/:.doperpowers/repo-facts.md` (or a "(no repo-facts manifest)" note when absent), and — implement - protocol only — `EXECUTION_BLOCK` = the engine's - `references/engine-blocks/execution-.md` and `DECOMPOSE_DOC` = - the ABSOLUTE path of implementing-tickets' + protocol only — `EXECUTION_BLOCK` = implementing-tickets' + `references/engine-blocks/execution.md` (one block, both routes) and + `DECOMPOSE_DOC` = the ABSOLUTE path of implementing-tickets' `references/implement-decompose.md` (a runtime-opened procedure: the prompt carries only the pointer; the worker opens it when Check-2 says decompose). -3. codex: `codex-spawn.sh "-" "" ` - (model/effort default gpt-5.6-sol/high — override with - `$CODEX_MODEL` / `$CODEX_EFFORT` as args 5–6). claude: - `daemon-spawn.sh "-" "" `. Both +3. Spawn via `daemon-spawn.sh "-" "" ` from `orchestrating-daemons` — always a worktree; workers write code. + The codex route prefixes the gateway env and pins the gateway's model + alias as arg 5: + `DAEMON_CLAUDE_SETTINGS="${CLODEX_SETTINGS:-$HOME/.claude/clodex-settings.json}" DAEMON_CLAUDE_EFFORT="${CLODEX_EFFORT:-xhigh}" daemon-spawn.sh … fable` + (daemon-spawn persists settings/effort into the registry meta; + daemon-resume restores them on every fork — without that a gateway + worker silently reverts to plain models on its first resume). The + claude route passes no gateway env; the model inherits unless pinned. 4. `board-bind.sh `. Write NOTHING else: the worker's first board write is its gate verdict — `in-progress` (+ a `[gate]` comment) means the gate passed; a park state means it failed. diff --git a/skills/orchestrating-daemons/SKILL.md b/skills/orchestrating-daemons/SKILL.md index 3949f57053..7022b65308 100644 --- a/skills/orchestrating-daemons/SKILL.md +++ b/skills/orchestrating-daemons/SKILL.md @@ -7,7 +7,7 @@ description: Use when spawning, resuming, tracking, or debugging durable backgro ## Overview -A **daemon** is a durable background `claude` session, spawned with `claude --bg` so it runs as its own process, is visible in `claude agents`, and survives this session ending. The substrate drives two engines under one registry: `claude --bg` daemons and detached `codex exec` workers (`engine: codex` in the meta) — the board pipeline picks per dispatch (label → `WORKER_ENGINE` → codex). This skill is the *substrate*: the scripts that spawn, resume, track, and retire daemons, and the mechanics that make those operations safe. It is not an orchestration doctrine — the fleets that used to be driven from here now live in the board pipeline, where workers escalate via park states and nobody judges their turn-ends. +A **daemon** is a durable background `claude` session, spawned with `claude --bg` so it runs as its own process, is visible in `claude agents`, and survives this session ending. The board pipeline dispatches Claude-harness daemons only; its engine label picks the model ROUTE of the spawn (`codex` = the clodex gateway settings via `DAEMON_CLAUDE_SETTINGS`, GPT models through the local proxy; `claude` = plain Claude models). Detached `codex exec` workers (`engine: codex` in the meta) are a **legacy species**: existing registry metas stay readable, resumable, and retirable, but no pipeline path creates new ones. This skill is the *substrate*: the scripts that spawn, resume, track, and retire daemons, and the mechanics that make those operations safe. It is not an orchestration doctrine — the fleets that used to be driven from here now live in the board pipeline, where workers escalate via park states and nobody judges their turn-ends. **Where work goes** (decide this before spawning anything): @@ -36,8 +36,8 @@ Paths are relative to this skill's directory. The scripts hide every sharp edge | `daemon-list.sh [status]` | Fleet view; optional status filter. | | `daemon-mark.sh [note]` | Record a judgment state (`awaiting-human`, `done`) + why. | | `daemon-retire.sh [purge]` | Drop from active fleet; transcript stays resumable. | -| `codex-spawn.sh [--no-wait] [cwd] [worktree] [model] [effort]` | Spawn a detached `codex exec --json` worker into the same registry (`engine: codex`, pid liveness). Defaults gpt-5.6-sol/high. Launch in a bg shell. | -| `codex-resume.sh ` | Continue a codex daemon (`codex exec resume` — same session id, no forking). Launch in a bg shell. | +| `codex-spawn.sh [--no-wait] [cwd] [worktree] [model] [effort]` | LEGACY — retired from board dispatch (no pipeline path spawns codex-CLI workers; kept until no bound codex sessions remain). Spawns a detached `codex exec --json` worker into the same registry (`engine: codex`, pid liveness). | +| `codex-resume.sh ` | Continue a LEGACY codex daemon (`codex exec resume` — same session id, no forking). Launch in a bg shell. | ## Driving an ad-hoc daemon by hand @@ -87,7 +87,7 @@ A daemon also goes `blocked` when it calls **AskUserQuestion** — headless, nob A daemon can also block on a **harness permission prompt** that holds a tool call before it ever reaches the transcript (observed live: an AskUserQuestion call stuck at the permission layer). The recorded reply then carries a `[blocked on a harness prompt …]` marker instead of a rendered question — the daemon's last text states what it wanted; resume with your answer/instruction (the pending call is interrupted), or `claude attach` the session to approve it interactively. -Codex workers get the same posture, spelled differently: `--sandbox +Legacy codex workers get the same posture, spelled differently: `--sandbox workspace-write` plus the approvals auto-reviewer (`-c approval_policy=on-request -c approvals_reviewer=…`) — safe ops continue, genuinely unsafe escalations are declined in-flight (fail-closed). A codex diff --git a/skills/orchestrating-daemons/scripts/daemon-finalize.sh b/skills/orchestrating-daemons/scripts/daemon-finalize.sh index 50f4bae5d9..75ac4250d6 100755 --- a/skills/orchestrating-daemons/scripts/daemon-finalize.sh +++ b/skills/orchestrating-daemons/scripts/daemon-finalize.sh @@ -27,9 +27,14 @@ case "$status" in working|blocked) ;; *) echo "noop"; exit 0 ;; esac cur="$(_meta_get "$uuid" current)"; [ -n "$cur" ] || cur="$uuid" # `state` alone lies for a finished session whose harness process lingers — -# it stays "working" indefinitely. `status` is the turn signal (busy while a -# turn runs, idle after); normalize the lingering shape to done before the -# case table. Observed live 2026-07-15 on a finished review worker. +# it stays "working" (observed live 2026-07-15 on a finished review worker) +# or "blocked" (observed live 2026-07-17 on a cleanly-parked gateway implement +# worker) indefinitely. `status` is the turn signal (busy while a turn runs, +# idle after); normalize both lingering shapes before the case table. An ended +# blocked-shape turn finalizes through the blocked reply renderer: when the +# transcript ends on a pending AskUserQuestion the question surfaces in the +# reply, otherwise the recorded reply is the turn text with the harness-prompt +# marker — either way the session is over and resumable. state="$(claude agents --json --all 2>/dev/null | CUR="$cur" python3 -c ' import json, os, sys try: @@ -39,8 +44,8 @@ except Exception: for r in rows: if r.get("sessionId") == os.environ["CUR"]: st = r.get("state") or "" - if st == "working" and r.get("status") == "idle": - st = "done" + if r.get("status") == "idle" and st in ("working", "blocked"): + st = "done" if st == "working" else "done-blocked" print(st) break ')" @@ -52,6 +57,10 @@ case "$state" in _record_reply "$cur" "$uuid" "done" _meta_set "$uuid" status "idle" updated "$(_now)" echo "idle" ;; + done-blocked) + _record_reply "$cur" "$uuid" "blocked" + _meta_set "$uuid" status "idle" updated "$(_now)" + echo "idle" ;; error|stopped) _record_reply "$cur" "$uuid" "$state" _meta_set "$uuid" status "error" updated "$(_now)" diff --git a/skills/reviewing-prs/scripts/land-dispatch.sh b/skills/reviewing-prs/scripts/land-dispatch.sh index 4765dd8a53..9f3c5f99c9 100755 --- a/skills/reviewing-prs/scripts/land-dispatch.sh +++ b/skills/reviewing-prs/scripts/land-dispatch.sh @@ -23,8 +23,16 @@ # Env: # LOCAL_REPO canonical local clone of the target repo (default: $PWD) # BOARD_REPO owner/name (default: resolved from LOCAL_REPO via gh) -# WORKER_ENGINE claude|codex (default codex); an engine:* PR label wins -# LAND_MODEL optional claude model override for the land daemon +# WORKER_ENGINE which MODEL ROUTE the land daemon uses: codex|claude +# (default codex). Every worker is a Claude-harness daemon; +# "codex" rides the clodex gateway settings (GPT models via +# the local proxy), "claude" = plain Claude models. An +# engine:* PR label wins over the env. +# CLODEX_SETTINGS gateway settings file for the codex route +# (default ~/.claude/clodex-settings.json) +# CLODEX_EFFORT reasoning effort for the codex route (default xhigh) +# LAND_MODEL optional model override for the land daemon (gateway +# route defaults to fable, claude route to inherit) # LAND_ENABLED staged rollout (default false = DRY-RUN mode: the worker # analyzes and posts what it WOULD do; merges nothing) # BOARD_SCRIPTS issue-tracker scripts dir override (tests) @@ -177,7 +185,18 @@ if [ -n "$meta" ]; then case "$status" in working|blocked) if _is_live "$current" "$w_engine" "$w_pid" "$w_host" "$w_boot"; then - echo "#$pr: skip active land worker"; exit 0 + if [ "$w_engine" = "codex" ]; then + echo "#$pr: skip active land worker"; exit 0 + fi + # A claude-species lander has no self-finalizer: a finished turn + # lingers status=working while its session stays visible in + # `claude agents`. Finalize first (as review-dispatch does) — only + # a genuinely live turn is an active worker; anything else + # (idle/error/absent/noop) is a finished lander to retire. + fin="$("$DAEMON_SCRIPTS/daemon-finalize.sh" "$uuid" 2>/dev/null || true)" + if [ "$fin" = "live" ]; then + echo "#$pr: skip active land worker"; exit 0 + fi fi _retire "$uuid" ;; *) _retire "$uuid" ;; # finished/retired — an explicit dispatch is a fresh signal @@ -286,8 +305,15 @@ PY )" || { rm -rf "$control_dir"; die "#$pr: prompt render failed"; } [ -n "$prompt" ] || { rm -rf "$control_dir"; die "#$pr: empty prompt — not dispatching"; } +# ONE worker species, two model routes (mirrors review-dispatch.sh): the +# default "codex" engine is a GATEWAY worker — the same Claude-harness daemon +# pointed at the local gateway (GPT models) via --settings. engine:claude +# opts a PR into plain Claude models. No path spawns a codex-CLI worker. if [ "$engine" = "codex" ]; then - spawn_out="$("$DAEMON_SCRIPTS/codex-spawn.sh" --no-wait "land-pr-$pr" "$prompt" "$wt" "")" \ + spawn_out="$(DAEMON_CLAUDE_SETTINGS="${CLODEX_SETTINGS:-$HOME/.claude/clodex-settings.json}" \ + DAEMON_CLAUDE_EFFORT="${CLODEX_EFFORT:-xhigh}" \ + "$DAEMON_SCRIPTS/daemon-spawn.sh" --no-wait "land-pr-$pr" "$prompt" "$wt" "" \ + "${LAND_MODEL:-fable}")" \ || { rm -rf "$control_dir"; die "#$pr: land worker spawn failed"; } else spawn_out="$("$DAEMON_SCRIPTS/daemon-spawn.sh" --no-wait "land-pr-$pr" "$prompt" "$wt" "" "${LAND_MODEL:-}")" \ diff --git a/tests/implementing-tickets/test-protocol-content.sh b/tests/implementing-tickets/test-protocol-content.sh index c5b5ebb97a..5950249167 100755 --- a/tests/implementing-tickets/test-protocol-content.sh +++ b/tests/implementing-tickets/test-protocol-content.sh @@ -93,28 +93,27 @@ assert_contains "$decomp" "grants no authority beyond your prompt" "decompose do assert_not_contains "$decomp" "{{" "decompose doc: placeholder-free (opened at runtime, never rendered)" echo "engine blocks:" -EXEC_CLAUDE="$REPO_ROOT/skills/implementing-tickets/references/engine-blocks/execution-claude.md" -EXEC_CODEX="$REPO_ROOT/skills/implementing-tickets/references/engine-blocks/execution-codex.md" -[ -f "$EXEC_CLAUDE" ] || { echo "missing $EXEC_CLAUDE"; exit 1; } -[ -f "$EXEC_CODEX" ] || { echo "missing $EXEC_CODEX"; exit 1; } -exec_claude="$(cat "$EXEC_CLAUDE")" -exec_codex="$(cat "$EXEC_CODEX")" -assert_contains "$exec_claude" "doperpowers:execplan" "claude block: execplan mode wired" -assert_contains "$exec_codex" "EXECPLAN:" "codex block: execplan mode wired (not bare PLAN)" -assert_contains "$exec_codex" "doperpowers:execplan" "codex block: routes to the execplan doctrine" -assert_contains "$exec_codex" ".agents/skills" "codex block: vendored skill doctrine pointer" -assert_not_contains "$exec_codex" "work ALONE" "codex block: no blanket work-alone constraint (subagents are the worker's call)" -assert_not_contains "$exec_codex" "YOURSELF" "codex block: no solo-execution emphasis (delegation inside the thread is the worker's call)" -assert_not_contains "$exec_claude" "work ALONE" "claude block: no blanket work-alone constraint (subagents are the worker's call)" -assert_contains "$exec_codex" "writing-plans" "codex block: names writing-plans as interactive-only" -assert_contains "$exec_claude" "writing-plans" "claude block: names writing-plans as interactive-only" -assert_contains "$exec_codex" "subagent-driven-development" "codex block: names the forbidden interactive skills" -assert_contains "$exec_claude" "subagent-driven-development" "claude block: names the forbidden interactive skills" -assert_contains "$exec_claude" "never" "claude block: evidence mandate present" -assert_contains "$exec_claude" "claim completion on reasoning alone" "claude block: no-evidence-no-done clause" -assert_contains "$exec_codex" "claim completion on reasoning alone" "codex block: no-evidence-no-done clause" -assert_contains "$exec_claude" "big-but-atomic" "claude block: atomic execplan trigger" -assert_contains "$exec_codex" "big-but-atomic" "codex block: atomic plan trigger" +# One harness, one block: both model routes (gateway "codex" / plain "claude") +# are Claude-harness sessions, so a single execution block serves both. +EXEC="$REPO_ROOT/skills/implementing-tickets/references/engine-blocks/execution.md" +[ -f "$EXEC" ] || { echo "missing $EXEC"; exit 1; } +exec_block="$(cat "$EXEC")" +if [ -e "$REPO_ROOT/skills/implementing-tickets/references/engine-blocks/execution-claude.md" ] \ + || [ -e "$REPO_ROOT/skills/implementing-tickets/references/engine-blocks/execution-codex.md" ]; then + fail "per-engine execution blocks are retired (one harness, one block)" +else + pass "per-engine execution blocks are retired (one harness, one block)" +fi +assert_contains "$exec_block" "EXECPLAN:" "block: execplan mode wired (not bare PLAN)" +assert_contains "$exec_block" "doperpowers:execplan" "block: routes to the execplan doctrine" +assert_not_contains "$exec_block" ".agents/skills" "block: no vendored-doctrine pointer (plugin skills resolve natively on the Claude harness)" +assert_not_contains "$exec_block" "work ALONE" "block: no blanket work-alone constraint (subagents are the worker's call)" +assert_not_contains "$exec_block" "YOURSELF" "block: no solo-execution emphasis (delegation inside the thread is the worker's call)" +assert_contains "$exec_block" "writing-plans" "block: names writing-plans as interactive-only" +assert_contains "$exec_block" "subagent-driven-development" "block: names the forbidden interactive skills" +assert_contains "$exec_block" "never" "block: evidence mandate present" +assert_contains "$exec_block" "claim completion on reasoning alone" "block: no-evidence-no-done clause" +assert_contains "$exec_block" "big-but-atomic" "block: atomic execplan trigger" echo "skill doctrine:" [ -f "$SKILL" ] || { echo "missing $SKILL"; exit 1; } @@ -124,6 +123,18 @@ assert_contains "$skill" "references/implement-worker-protocol.md" "skill points assert_contains "$skill" "doperpowers:issue-tracker" "skill points at the board schema" assert_contains "$skill" "board-answer.sh" "skill names the answer relay (park = pause)" assert_not_contains "$skill" "status:blocked" "no retired vocabulary in doctrine" +assert_not_contains "$skill" ".agents/skills" "skill: no vendored-doctrine pointer (one Claude harness, plugin skills native)" +assert_contains "$skill" "model route" "skill: engine described as a model route, not a worker species" + +echo "dispatch ritual (issue-tracker):" +TRACKER="$REPO_ROOT/skills/issue-tracker/SKILL.md" +[ -f "$TRACKER" ] || { echo "missing $TRACKER"; exit 1; } +tracker="$(cat "$TRACKER")" +assert_not_contains "$tracker" "codex-spawn.sh" "ritual: codex-CLI spawn path retired (no new codex-CLI workers)" +assert_contains "$tracker" "DAEMON_CLAUDE_SETTINGS" "ritual: gateway route rides daemon-spawn via settings env" +assert_contains "$tracker" "daemon-spawn.sh" "ritual: one spawn command for both routes" +assert_contains "$tracker" "engine-blocks/execution.md" "ritual: EXECUTION_BLOCK binds to the single block" +assert_contains "$tracker" "model route" "ritual: engine resolution states route semantics" echo if [ "$FAILURES" -gt 0 ]; then echo "$FAILURES test(s) FAILED"; exit 1; fi diff --git a/tests/orchestrating-daemons/test-daemon-scripts.sh b/tests/orchestrating-daemons/test-daemon-scripts.sh index a81e5ca380..99a0811f18 100755 --- a/tests/orchestrating-daemons/test-daemon-scripts.sh +++ b/tests/orchestrating-daemons/test-daemon-scripts.sh @@ -411,6 +411,21 @@ sed -i '' 's/^status=busy$/status=idle/' "$STUB_STATE/agents/$FIN5_SHORT" assert_equals "$("$SCRIPTS_DIR/daemon-finalize.sh" "$FIN5_UUID")" "idle" "state=working with status=idle is a FINISHED lingering turn, not live" assert_contains "$(cat "$DAEMON_HOME/$FIN5_UUID.reply.txt")" "ANSWER:FIN-TASK-5" "lingering finished turn's reply is recorded" +# Second lying shape (observed live 2026-07-17 on a cleanly-parked gateway +# implement worker): the ended session lingers with state=blocked while +# status=idle. `status` stays the turn signal — an ended blocked-shape turn +# (clean park with a stuck flag, or a turn that ended ON a pending question) +# must finalize resumable, with the reply recorded through the blocked +# renderer so a pending question, when present, still surfaces. +FIN6_OUT="$(STUB_BG_STATE=blocked STUB_BG_STATUS=busy "$SCRIPTS_DIR/daemon-spawn.sh" --no-wait "finblk" "FIN-TASK-6" "$WORK")" +FIN6_UUID="$(printf '%s' "$FIN6_OUT" | sed -n 's/.*\[[0-9a-f]* \/ \([0-9a-f-]*\)\].*/\1/p' | head -1)" +FIN6_SHORT="$(sed -n 's/.*"short": "\([^"]*\)".*/\1/p' "$DAEMON_HOME/$FIN6_UUID.json")" +assert_equals "$("$SCRIPTS_DIR/daemon-finalize.sh" "$FIN6_UUID")" "live" "state=blocked with status=busy is a live turn" +sed -i '' 's/^status=busy$/status=idle/' "$STUB_STATE/agents/$FIN6_SHORT" +assert_equals "$("$SCRIPTS_DIR/daemon-finalize.sh" "$FIN6_UUID")" "idle" "state=blocked with status=idle is an ENDED turn — finalized resumable" +assert_contains "$(cat "$DAEMON_HOME/$FIN6_UUID.json")" '"status": "idle"' "ended blocked-shape turn's meta finalizes idle" +assert_contains "$(cat "$DAEMON_HOME/$FIN6_UUID.reply.txt")" "ANSWER:FIN-TASK-6" "ended blocked-shape turn's reply is recorded" + # The blocking-mode watcher must see the same truth: a lingering finished # session (state=working, status=idle) terminates _poll_until_done as done # instead of polling to timeout. diff --git a/tests/reviewing-prs/test-land-dispatch.sh b/tests/reviewing-prs/test-land-dispatch.sh index 598f92e68b..837853e946 100755 --- a/tests/reviewing-prs/test-land-dispatch.sh +++ b/tests/reviewing-prs/test-land-dispatch.sh @@ -66,6 +66,7 @@ cat > "$STUB_DAEMONS/daemon-spawn.sh" <<'STUB' #!/usr/bin/env bash set -euo pipefail echo "spawn:$*" >> "$SPAWN_LOG" +echo "spawn-env:settings=${DAEMON_CLAUDE_SETTINGS:-};effort=${DAEMON_CLAUDE_EFFORT:-}" >> "$SPAWN_LOG" [ "${1:-}" = "--no-wait" ] && shift name="$1"; task="$2"; cwd="${3:-}" printf '%s' "$task" > "$PROMPT_DIR/$name.prompt" @@ -285,28 +286,45 @@ assert_contains "$(cat "$PROMPT_DIR/land-pr-9.prompt")" "manual 'land' label" "p # ---- dedupe ------------------------------------------------------------------------- echo "dedupe:" -seed_lander() { # $1=status - S="$1" python3 - <<'PY' +seed_lander() { # $1=status [$2=turn_state — drives the daemon-finalize stub] + S="$1" TS="${2:-}" python3 - <<'PY' import json, os -json.dump({"uuid": "feed0000-0000-4000-8000-000000000000", - "current": "feed0000-0000-4000-8000-000000000000", - "name": "land-pr-5", "status": os.environ["S"], - "updated": "2026-07-12T00:00:00Z"}, - open(os.path.join(os.environ["DAEMON_HOME"], - "feed0000-0000-4000-8000-000000000000.json"), "w")) +m = {"uuid": "feed0000-0000-4000-8000-000000000000", + "current": "feed0000-0000-4000-8000-000000000000", + "name": "land-pr-5", "status": os.environ["S"], + "updated": "2026-07-12T00:00:00Z"} +if os.environ["TS"]: + m["turn_state"] = os.environ["TS"] +json.dump(m, open(os.path.join(os.environ["DAEMON_HOME"], + "feed0000-0000-4000-8000-000000000000.json"), "w")) PY } reset_state; seed_lander working echo '[{"id": "feedcafe", "sessionId": "feed0000-0000-4000-8000-000000000000"}]' > "$MOCK_DIR/agents.json" out="$("$DISPATCH" 5)" assert_contains "$out" "active land worker" "live ACTIVE land worker → skip" -assert_equals "$(cat "$SPAWN_LOG")" "" "live ACTIVE land worker spawns nothing" +if grep -qE "^(spawn|codex-spawn|retire):" "$SPAWN_LOG" 2>/dev/null; then + fail "live ACTIVE land worker spawns and retires nothing" +else + pass "live ACTIVE land worker spawns and retires nothing" +fi reset_state; seed_lander working # agents.json [] → session gone "$DISPATCH" 5 > /dev/null assert_contains "$(cat "$SPAWN_LOG")" "retire:feed0000" "dead land worker retired" assert_contains "$(cat "$SPAWN_LOG")" "spawn:--no-wait land-pr-5" "dead land worker respawned" +# A gateway (claude-species) lander has NO self-finalizer: after its turn ends +# the registry meta lingers status=working and the session can stay visible in +# `claude agents`. Dedupe must finalize before believing the meta — otherwise a +# finished dry-run lander blocks every later live dispatch forever. +reset_state; seed_lander working idle # visible in agents, but the turn is over +echo '[{"id": "feedcafe", "sessionId": "feed0000-0000-4000-8000-000000000000"}]' > "$MOCK_DIR/agents.json" +"$DISPATCH" 5 > /dev/null +assert_contains "$(cat "$SPAWN_LOG")" "finalize:feed0000" "lingering finished lander is finalized before dedupe" +assert_contains "$(cat "$SPAWN_LOG")" "retire:feed0000" "lingering finished lander retired, not skipped" +assert_contains "$(cat "$SPAWN_LOG")" "spawn:--no-wait land-pr-5" "lingering finished lander re-dispatched" + reset_state python3 - <<'PY' import json, os @@ -449,13 +467,25 @@ reset_state assert_equals "$(cat "$EDIT_LOG")" "" "dry-run leaves the land label in place" # ---- engine resolution ------------------------------------------------------------------- +# ONE worker species: both routes spawn via daemon-spawn.sh; the codex route +# rides the clodex gateway settings (mirrors review-dispatch.sh). echo "engine resolution:" reset_state WORKER_ENGINE=codex "$DISPATCH" 5 > /dev/null -assert_contains "$(cat "$SPAWN_LOG")" "codex-spawn:--no-wait land-pr-5" "WORKER_ENGINE=codex spawns via codex" +assert_contains "$(cat "$SPAWN_LOG")" "spawn:--no-wait land-pr-5" "WORKER_ENGINE=codex spawns the one worker species via daemon-spawn" +assert_contains "$(cat "$SPAWN_LOG")" "spawn-env:settings=$HOME/.claude/clodex-settings.json;effort=xhigh" "codex route rides the gateway DAEMON_CLAUDE_SETTINGS/EFFORT" +if grep -q "codex-spawn:" "$SPAWN_LOG"; then + fail "no codex-CLI worker is ever spawned (species retired)" +else + pass "no codex-CLI worker is ever spawned (species retired)" +fi reset_state "$DISPATCH" 12 > /dev/null # suite default WORKER_ENGINE=claude; label must win -assert_contains "$(cat "$SPAWN_LOG")" "codex-spawn:--no-wait land-pr-12" "engine:codex label overrides the env" +assert_contains "$(cat "$SPAWN_LOG")" "spawn:--no-wait land-pr-12" "engine:codex label overrides the env (gateway route)" +assert_contains "$(cat "$SPAWN_LOG")" "spawn-env:settings=$HOME/.claude/clodex-settings.json;effort=xhigh" "label-selected codex route also rides the gateway settings" +reset_state +"$DISPATCH" 5 > /dev/null # suite default WORKER_ENGINE=claude +assert_contains "$(cat "$SPAWN_LOG")" "spawn-env:settings=;effort=" "claude route spawns without the gateway settings" # ---- merge-method resolution ---------------------------------------------------------------- echo "merge method:"