Skip to content

fix: harden Codex review auto-resolve flow#455

Merged
BigSimmo merged 2 commits into
mainfrom
codex/review-autofix-flow
Jul 10, 2026
Merged

fix: harden Codex review auto-resolve flow#455
BigSimmo merged 2 commits into
mainfrom
codex/review-autofix-flow

Conversation

@BigSimmo

Copy link
Copy Markdown
Owner

Summary

  • Require the exact trusted Codex connector bot identity.
  • Move per-PR concurrency onto the authorized job so unrelated review comments cannot displace it.
  • Trust deduplication markers only from GitHub Actions and allow one repair per head SHA, capped at three cycles.
  • Fail visibly when comment permissions are denied.
  • Pin actions/github-script v9.0.0 to its verified immutable commit.
  • Add workflow guards, direct embedded-script tests, policy documentation, and review-ledger records.

Why

The previous bridge could be triggered by substring-matching identities, suppressed by spoofed markers, displaced by unrelated workflow runs, and silently succeed when GitHub denied comment permissions. It also allowed only one repair request for the entire PR and used a mutable action tag.

Areas changed

  • .github/workflows/codex-autofix-review-comments.yml
  • scripts/check-codex-autofix-workflow.mjs
  • scripts/check-github-action-pins.mjs
  • tests/codex-autofix-workflow.test.ts
  • AGENTS.md
  • docs/branch-review-ledger.md

Verification

  • npm run verify:cheap
    • runtime check passed
    • GitHub Actions guard passed
    • sitemap check passed
    • lint passed
    • typecheck passed
    • 152 test files passed, 1 skipped
    • 1,426 tests passed, 1 skipped
  • Focused auto-resolve tests: 15 passed
  • Prettier check passed
  • git diff --check passed
  • Official git ls-remote verified the pinned github-script v9.0.0 commit.

Checks not run

  • npm run verify:release was not run because it includes provider-backed Supabase/OpenAI governance and evaluation gates; this change does not touch application, clinical, database, or provider behavior.
  • The live GitHub event path was not manually triggered before opening this PR; hosted PR checks provide the integration signal.

Risk and limitations

  • Automatic repair is intentionally capped at three head-specific cycles per PR to prevent an unbounded review/fix loop.
  • Permission failures now make the workflow red instead of silently returning success.
  • No application, database, migration, clinical-output, privacy, or production-environment behavior changed.

@supabase

supabase Bot commented Jul 10, 2026

Copy link
Copy Markdown

This pull request has been ignored for the connected project sjrfecxgysukkwxsowpy because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

@coderabbitai

coderabbitai Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The PR hardens Codex auto-resolve review-comment handling with exact bot checks, scoped deduplication, repair limits, explicit permission failures, immutable action validation, updated instructions, and end-to-end workflow tests.

Changes

Codex auto-resolve hardening

Layer / File(s) Summary
Workflow runtime filtering and repair flow
.github/workflows/codex-autofix-review-comments.yml, AGENTS.md, docs/branch-review-ledger.md
The workflow and instructions require trusted request comments, skip replies, scope markers to PR heads, limit repairs, and fail visibly on comment permission errors; review records document updated flow runs.
Static workflow and action pin enforcement
scripts/check-codex-autofix-workflow.mjs, scripts/check-github-action-pins.mjs
Guard checks enforce workflow identity, permission, concurrency, deduplication, prompt, and immutable action-pinning requirements; github-script majors 8–9 are supported.
Workflow extraction and behavioral validation
tests/codex-autofix-workflow.test.ts
Tests execute the embedded workflow with stubbed GitHub APIs and validate guard failures, trust checks, deduplication, repair limits, quoted markers, and permission errors.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Reviewer
  participant CodexAutofixWorkflow
  participant GitHubAPI
  Reviewer->>CodexAutofixWorkflow: post trusted review request
  CodexAutofixWorkflow->>GitHubAPI: list PR comments
  GitHubAPI-->>CodexAutofixWorkflow: comments or permission failure
  CodexAutofixWorkflow->>CodexAutofixWorkflow: check head marker and repair limit
  CodexAutofixWorkflow->>GitHubAPI: create scoped auto-resolve comment
Loading

Caution

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

  • Ignore

❌ Failed checks (1 error, 1 warning)

Check name Status Explanation Resolution
Verification Claims ❌ Error The generated summary says verification completed successfully without tying it to exact commands/checks. Rewrite vague success lines as explicit check results, e.g. 'Ran npm run verify:cheap: passed' and 'Not run: verify:release (reason)'; avoid generic 'verified/successfully' claims.
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 70.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (9 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: hardening the Codex review auto-resolve flow.
Description check ✅ Passed The description covers summary, rationale, affected areas, verification, and limitations, with only non-critical template sections omitted.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Generated And Sensitive Files ✅ Passed Touched files are workflow/script/test/doc changes only; scans found no secrets, keys, tokens, .env files, caches, build outputs, or local artifacts.
Risky Git Or Deployment Actions ✅ Passed No modified file recommends force-push/reset/deploy; the added AGENTS/workflow text explicitly forbids destructive git/deployment actions without confirmation.
Supabase Project And Schema Safety ✅ Passed Touched files are workflow/docs/scripts/tests only; no Supabase env, schema, RLS, policy, migration, or stale project-ref changes appear in the diff.
Runtime And Package Manager Integrity ✅ Passed PASS: the PR only changes workflow guards/docs/tests; package.json/.npmrc/.nvmrc still pin npm@11.17.0, Node 24, and engine-strict=true, with no lockfile or install-flag changes.
Api Route Failure Handling ✅ Passed The only runtime integration here is the GitHub comment bridge, and it now fails visibly on 403 list/create errors while rethrowing unexpected exceptions; no other API/server-action changes exist.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/review-autofix-flow
✨ Simplify code
  • Create PR with simplified code
  • Commit simplified code in branch codex/review-autofix-flow

Comment @coderabbitai help to get the list of available commands.

coderabbitai[bot]
coderabbitai Bot previously requested changes Jul 10, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@AGENTS.md`:
- Line 394: Update the wording in the AGENTS.md bullet to hyphenate the compound
adjective, changing “Node 24 based” to “Node 24-based.”
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 2cc1b2f4-229b-439d-b6f8-69afe52f9b27

📥 Commits

Reviewing files that changed from the base of the PR and between 155c801 and 74c7805.

📒 Files selected for processing (6)
  • .github/workflows/codex-autofix-review-comments.yml
  • AGENTS.md
  • docs/branch-review-ledger.md
  • scripts/check-codex-autofix-workflow.mjs
  • scripts/check-github-action-pins.mjs
  • tests/codex-autofix-workflow.test.ts

Comment thread AGENTS.md Outdated
@BigSimmo
BigSimmo dismissed coderabbitai[bot]’s stale review July 10, 2026 14:48

The sole finding was fixed in 859624f, its thread is resolved, and the refreshed CodeRabbit plus all required CI/security checks passed on the updated head.

@BigSimmo
BigSimmo merged commit 609a96f into main Jul 10, 2026
9 checks passed
@BigSimmo
BigSimmo deleted the codex/review-autofix-flow branch July 10, 2026 21:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant