refactor(reviewing-prs): separate correctness and protocol compliance - #15
Conversation
|
Closing without merge, per adversarial review of the stack (three independent reviews). The verdict: the policies in this PR survived review intact — split responsibilities, anti-anchoring audit ordering, read-only concurrency, no-retroactive-spec — but the form did not: the provenance hash machinery is fragile at its weakest link (gate-side hashes are computed by an LLM following prose against a script-computed review side) and noisy in steady state (implement-protocol-sha256 mismatches after every plugin update), SPEC FINDING and EVIDENCE FINDING route identically, and the protocol doubled in size with prose the tests then froze. This will be rebuilt from the policy record rather than patched: the execplan on this branch (docs/doperpowers/execplans/2026-07-14-split-review-responsibilities.md) is the source. The branch is kept for that record. Key changes for the rebuild: timestamp-anchored drift detection instead of hashes, one merged worker-finding class, evidence gate keyed on gate-comment presence, in-loop confident-ready expiry, and fixes for the letter-level dead-ends (ticketless TOO BIG, TECH_DEBT_ISSUE=none, engine-outage respawn cap, secondary linked issues). The trust-boundary fix and dead-template-tail fix from this stack were pulled down into PR #14 directly (commit bf4552d). |
Who is submitting this PR? (required)
claude-fable-5[1m])Installed plugin inventory
agent-harness-workspace@agent-harness0.1.0 (project, disabled)agent-harness@agent-harness0.2.0 (user, disabled)agent-sdk-dev@claude-plugins-officialunknown (user, enabled)chrome-devtools-mcp@claude-plugins-official1.5.0 (user, disabled)claude-code-setup@claude-plugins-official1.0.0 (user, enabled)claude-md-management@claude-plugins-official1.0.0 (user, enabled)cloud-sql-postgresql@claude-plugins-official0.4.0 (project, disabled)cloudflare@claude-plugins-official1.0.0 (user, disabled)code-modernization@claude-plugins-officialunknown (local, disabled)code-review@claude-plugins-officialunknown (user, disabled)code-simplifier@claude-plugins-official1.0.0 (user, enabled)codex@openai-codex1.0.6 (user, enabled)commit-commands@claude-plugins-officialunknown (project, disabled)context7@claude-plugins-officialunknown (local, enabled)context7@claude-plugins-officialunknown (user, enabled)desktop-commander@claude-plugins-official0.2.0 (user, disabled)discord@claude-plugins-official0.0.4 (local, disabled)doperpowers@doperpowers7.17.0 (user, enabled)explanatory-output-style@claude-plugins-official1.0.0 (user, enabled)feature-dev@claude-plugins-officialunknown (local, disabled)firecrawl@claude-plugins-official1.0.9 (user, disabled)frontend-design@claude-plugins-officialunknown (user, enabled)learning-output-style@claude-plugins-official1.0.0 (user, disabled)linear@claude-plugins-officialunknown (project, disabled)mcp-server-dev@claude-plugins-officialunknown (user, enabled)microsoft-docs@claude-plugins-official0.3.1 (user, disabled)playground@claude-plugins-officialunknown (project, disabled)plugin-dev@claude-plugins-officialunknown (user, enabled)pr-review-toolkit@claude-plugins-officialunknown (user, disabled)pyright-lsp@claude-plugins-official1.0.0 (user, enabled)remember@claude-plugins-official0.8.3 (user, enabled)rust-analyzer-lsp@claude-plugins-official1.0.0 (user, enabled)security-guidance@claude-plugins-official2.0.6 (user, enabled)skill-creator@claude-plugins-officialunknown (user, enabled)sourcegraph@claude-plugins-official0.1.0 (user, disabled)supabase@claude-plugins-official0.1.12 (user, enabled)typescript-lsp@claude-plugins-official1.0.0 (user, enabled)What problem are you trying to solve?
The native Codex reviewer was being asked to perform two different jobs in one invocation: its built-in correctness review and ticket-specific spec/decision-discipline review injected through
developer_instructions. In real use, the spec addendum did not behave as intended and could distract or weaken the otherwise robust native correctness review. Meanwhile the outer Review Worker waited for Codex instead of using its own ticket and process context to audit whether the Implement Worker actually passed theready-for-agentgate and stopped for human-grade product decisions.The intended split is actor-specific: native Codex should review code correctness without custom criteria, while the outer Review Worker should audit the Implement Worker's protocol compliance against the canonical ticket artifacts.
This PR is stacked on #14, which first makes
SKILL.mdthe canonical Review Worker Protocol. The changes here are only the follow-up behavior and trust-boundary work enabled by that entrypoint restructure.What does this PR change?
review-engine.shnow runs purecodex exec review --basewith no criteria file or custom developer instructions. The outer Review Worker starts that engine in the background, independently records a ticket/spec/decision-discipline audit, then joins and routes both streams.The audit uses the authorization-time issue body as primary specification, base-pinned issue-referenced documents and pre-resume human answers as authoritative supplements, and distinct
PROTOCOL BLOCKER,SPEC FINDING,AUDIT NOTE, andEVIDENCE FINDINGroutes. The dispatcher supplies trusted absolute paths and current hashes for both sides of the audit contract; Implement Worker gate comments record the original issue-body and protocol hashes, and GitHub edit history resolves later ticket-body drift.Is this change appropriate for the core library?
Yes. This is a general-purpose responsibility and trust-boundary correction for the core autonomous PR-review loop. It applies to every adopting repository and both Claude and Codex worker species, adds no dependency, and contains no project-specific policy.
What alternatives did you consider?
.agents/skills. Rejected because that path is PR-controlled in the detached review head. The dispatcher-owned canonical skill file is the only trusted protocol source.Does this PR contain multiple unrelated changes?
No. The engine simplification, worker-owned audit, source hierarchy, evidence routing, canonical-skill trust boundary, tests, and living documentation are all required by the same responsibility split. PR #14 is the structural prerequisite and is therefore this PR's base rather than bundled into its diff.
Existing PRs
#14 is the direct stack base and only changes the skill entrypoint/file ownership. #9 introduced the native review engine and the criteria/developer-instruction carrier that this PR partially supersedes while preserving its nested environment recipe. #12 established native severity as the blocker bit for native findings; this PR preserves that rule and adds separate classes only for worker-produced protocol/spec/evidence findings.
Environment tested
claude-fable-5[1m]gpt-5.6-sol, high effortNew harness support (required if this PR adds a new harness)
Not applicable. No new harness is added.
Evaluation
codex exec reviewdeveloper instructions was not working as intended and could disrupt correctness review; move that responsibility into the outer Review Worker and run both tracks concurrently.git diff --check.tests/claude-code/run-skill-tests.shhas one unrelated model-output regex instability intest-subagent-driven-development.sh: two runs produced semantically compliant answers but missed different literal patterns. The affected skill and test have no diff in this branch.Rigor
doperpowers:writing-skillsand completed adversarial pressure testingdoperpowers:writing-skillsand test-first RED/GREEN discipline were used. The first box remains unchecked because the branch has not yet run multiple live Review Worker behavior sessions; direct Codex branch review and hermetic protocol tests are not being mislabeled as that evaluation.The direct reviewer path was the bounded CLI
codex exec review --base ..., not the recursive native code-review/reviewer delegation path that previously spawned 43+ subagents.Human review
The human partner approved the design, blocker semantics, issue-source hierarchy, and stacked delivery. This PR is opened as a draft to provide the complete follow-up diff for line-by-line review; do not mark ready or merge until this box can be checked truthfully.