Skip to content

fix: harden Codex review auto-resolve flow #455

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
BigSimmo merged 2 commits into from
Jul 10, 2026
Merged

fix: harden Codex review auto-resolve flow #455

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
74c7805
fix: harden Codex review auto-resolve flow
BigSimmo Jul 10, 2026
859624f
docs: hyphenate Node 24-based policy
BigSimmo Jul 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 51 additions & 19 deletions .github/workflows/codex-autofix-review-comments.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,45 +9,59 @@ permissions:
issues: write
pull-requests: read

concurrency:
group: codex-autoresolve-${{ github.event.pull_request.number }}
cancel-in-progress: false

jobs:
request-codex-autoresolve:
name: Request Codex auto-resolve
runs-on: ubuntu-24.04
if: >
github.event.pull_request.state == 'open' &&
contains(github.event.comment.user.login, 'chatgpt-codex-connector')
github.event.comment.user.type == 'Bot' &&
(github.event.comment.user.login == 'chatgpt-codex-connector' ||
github.event.comment.user.login == 'chatgpt-codex-connector[bot]')
concurrency:
group: codex-autoresolve-${{ github.event.pull_request.number }}
cancel-in-progress: false
steps:
- name: Ask Codex to resolve review comments
uses: actions/github-script@v7
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
github-token: ${{ github.token }}
script: |
const pr = context.payload.pull_request;
const reviewComment = context.payload.comment;
const allowedCodexBotLogins = new Set([
"chatgpt-codex-connector",
"chatgpt-codex-connector[bot]",
]);
const scopedResolveCommand = "@codex resolve actionable Codex review findings for this pull request and current head";

if (
reviewComment.user?.type !== "Bot" ||
!allowedCodexBotLogins.has(reviewComment.user.login)
) {
core.warning("Skipping Codex auto-resolve request because the review comment author is not the trusted Codex connector bot.");
return;
}

if (reviewComment.in_reply_to_id) {
core.notice("Skipping Codex auto-resolve request because the comment is a review-thread reply.");
return;
}

const sourceBody = reviewComment.body || "";
if (
sourceBody.includes("codex-autoresolve:") ||
sourceBody.includes("codex-autoresolve-pr:") ||
sourceBody.includes("@codex resolve actionable Codex review findings for this pull request and current head")
) {
const sourceBody = (reviewComment.body || "").trimStart();
const hasAutoResolveMarker =
sourceBody.startsWith("<!-- codex-autoresolve:") ||
sourceBody.startsWith("<!-- codex-autoresolve-pr:");
if (hasAutoResolveMarker && sourceBody.includes(scopedResolveCommand)) {
core.notice("Skipping Codex auto-resolve request because the source comment already looks like an auto-resolve request.");
return;
}

const owner = context.repo.owner;
const repo = context.repo.repo;
const issue_number = pr.number;
const marker = `<!-- codex-autoresolve-pr:${pr.number} -->`;
const marker = `<!-- codex-autoresolve:${pr.head.sha} -->`;
const maxAutoResolveRequests = 3;

let existingComments;
try {
Expand All @@ -59,23 +73,39 @@ jobs:
});
} catch (error) {
if (error.status === 403) {
core.warning("Skipping Codex auto-resolve request because GitHub denied permission to list issue comments.");
core.notice("Grant issues: read/write permission or allow the workflow token to read issue comments to enable duplicate request checks.");
const message = "Codex auto-resolve failed because GitHub denied permission to list issue comments.";
core.warning(message);
core.setFailed(message);
return;
}

throw error;
}

if (existingComments.some((comment) => (comment.body || "").includes(marker))) {
core.notice("Skipping duplicate Codex auto-resolve request for this pull request.");
const trustedExistingRequests = existingComments.filter(
(comment) =>
comment.user?.type === "Bot" &&
comment.user.login === "github-actions[bot]" &&
(comment.body || "").trimStart().startsWith("<!-- codex-autoresolve"),
);
const hasTrustedExistingRequest = trustedExistingRequests.some((comment) =>
(comment.body || "").includes(marker),
);

if (hasTrustedExistingRequest) {
core.notice("Skipping duplicate Codex auto-resolve request for this pull request head.");
return;
}

if (trustedExistingRequests.length >= maxAutoResolveRequests) {
core.warning(`Skipping Codex auto-resolve request because this pull request reached the automatic repair limit of ${maxAutoResolveRequests}.`);
return;
}

const prompt = [
marker,
`<!-- codex-autoresolve-source-review-comment:${reviewComment.id} -->`,
"@codex resolve actionable Codex review findings for this pull request and current head using the repository instructions. Always fix P0 and P1 findings. For P2 and lower findings, decide whether each is worth fixing automatically. Fix clear, scoped, low-risk issues with the best minimal change; otherwise reply explaining why the issue is deferred or not actionable. Do not update the branch from main, address unrelated reviews, broaden scope, or create more than one scoped fix commit unless explicitly asked. After each fix or decision, resolve the review conversation if supported. Do not use external APIs, paid services, credentials, dependency changes, or broad refactors unless explicitly authorized. Add targeted tests where behavior changes and run the narrowest relevant validation.",
`${scopedResolveCommand} using the repository instructions. Always fix P0 and P1 findings. For P2 and lower findings, decide whether each is worth fixing automatically. Fix clear, scoped, low-risk issues with the best minimal change; otherwise reply explaining why the issue is deferred or not actionable. Do not update the branch from main, address unrelated reviews, broaden scope, or create more than one scoped fix commit unless explicitly asked. After each fix or decision, resolve the review conversation if supported. Do not use external APIs, paid services, credentials, dependency changes, or broad refactors unless explicitly authorized. Add targeted tests where behavior changes and run the narrowest relevant validation.`,
].join("\n\n");

try {
Expand All @@ -87,7 +117,9 @@ jobs:
});
} catch (error) {
if (error.status === 403) {
core.warning("Skipping Codex auto-resolve request because this workflow token cannot comment on the pull request.");
const message = "Codex auto-resolve failed because this workflow token cannot comment on the pull request.";
core.warning(message);
core.setFailed(message);
return;
}

Expand Down
7 changes: 6 additions & 1 deletion AGENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -389,9 +389,14 @@ When explicitly asked to fix or resolve review findings:
Automatic Codex review is review-only by default. This repository includes `.github/workflows/codex-autofix-review-comments.yml`, which requests the resolve task automatically after Codex posts a PR review or inline review comment.

- The workflow must only trigger from Codex review bot reviews or comments on open pull requests.
- Match the trusted Codex connector bot by exact login and bot type; do not use substring login checks.
- Keep per-pull-request concurrency on the authorized job, not the whole workflow, so unrelated review comments cannot displace a pending Codex request.
- Pin the supported Node 24-based `actions/github-script` release to its reviewed immutable commit SHA.
- The workflow must skip review-thread replies and auto-resolve request comments so Codex fix summaries do not re-trigger the workflow.
- The workflow must ask Codex to resolve all review comments using these repository instructions.
- The workflow must avoid duplicate requests for the same pull request, even after follow-up commits change the head SHA.
- The workflow may request one repair per pull-request head SHA, capped at three automatic repair cycles per pull request.
- Only trust a pull-request deduplication marker when it was posted by the GitHub Actions bot.
- Permission failures while reading or creating pull-request comments must fail the workflow visibly, not return a successful soft-skip.
- The workflow must not run Codex directly with API credentials.
- P0 and P1 findings should always be fixed.
- P2 and lower findings should be fixed only when clear, scoped, low-risk, and testable; otherwise explain the decision and resolve or mark ready for human resolution.
Expand Down
9 changes: 6 additions & 3 deletions docs/branch-review-ledger.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,9 @@ Use this ledger to prevent repeated branch and PR reviews when the reviewed HEAD

## Review Records

| Date | Branch or ref | Reviewed HEAD | Scope | Outcome | Checks |
| ---------- | -------------- | ------------- | -------------- | ------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- |
| 2026-07-09 | example/branch | abc1234 | branch-cleanup | Example: already merged into `main`; no unique patch content. | `git log --right-only --cherry-pick main...example/branch`; `git diff --name-status main...example/branch` |
| Date | Branch or ref | Reviewed HEAD | Scope | Outcome | Checks |
| ---------- | ------------------------- | ---------------------------------------- | ---------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 2026-07-09 | example/branch | abc1234 | branch-cleanup | Example: already merged into `main`; no unique patch content. | `git log --right-only --cherry-pick main...example/branch`; `git diff --name-status main...example/branch` |
| 2026-07-10 | codex/review-autofix-flow | 155c801cd58f797037d8aaa8b885405a1c599249 | codex-autofix-flow | Fixed exact connector authorization, trusted-marker deduplication, and strict self-trigger matching; added regression coverage. | `npm run check:codex-autofix-workflow`; focused Vitest (4 passed); `npm run verify:cheap` pre-test stages passed before tool timeout; `npm test` (1,415 passed, 1 skipped); focused Prettier check; `npm run check:github-actions` |
| 2026-07-10 | codex/review-autofix-flow | 155c801cd58f797037d8aaa8b885405a1c599249 | codex-autofix-flow-followup | Fixed untrusted workflow-level concurrency interference and migrated the bridge from the Node 20 action runtime to `actions/github-script@v9`; added direct embedded-script execution coverage. | Focused Vitest (13 passed); targeted ESLint; `tsc --noEmit`; `npm run check:codex-autofix-workflow`; `npm run check:github-actions`; focused Prettier check; `git diff --check` |
| 2026-07-10 | codex/review-autofix-flow | 155c801cd58f797037d8aaa8b885405a1c599249 | codex-autofix-residual-fixes | Replaced one-shot PR deduplication with a three-cycle head-SHA cap, made comment permission failures fail visibly, and pinned `github-script` v9.0.0 to its verified immutable commit. | TDD red run (7 expected failures); focused Vitest green run (15 passed); `npm run verify:cheap` (152 files passed, 1 skipped; 1,426 tests passed, 1 skipped); focused Prettier check; `git diff --check`; official `git ls-remote` tag verification |
Loading