feat(pr-reviewer): label + author gates and already-merged skip#21
Conversation
|
CodeAnt AI is reviewing your PR. |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughThis PR adds a review gate mechanism to the PR review agent that evaluates skip conditions before reviewing. It loads PR state/labels from materialized metadata, checks for disabling labels and merged status, optionally enforces an author allowlist, and notifies Slack before skipping or proceeding to review. ChangesPR Review Gate with Skip Rules
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
codeant-ai
Bot
added
the
size:L
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request refactors several agents to use high-level client helpers from '@relayfile/relay-helpers' instead of low-level JSON file operations. It also introduces a review gate mechanism in the review agent to skip reviews based on PR status, configured labels, or author allowlists. Feedback is provided to defensively handle 'meta.labels' and label names in the review gate logic to prevent potential runtime errors.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| const prLabels = (meta?.labels ?? []) | ||
| .map((l) => l?.name?.trim().toLowerCase()) | ||
| .filter((n): n is string => Boolean(n)); |
There was a problem hiding this comment.
To prevent potential runtime errors if meta.labels is not an array or if a label's name is not a string, we should defensively check that meta.labels is an array and ensure l.name is a string before calling .trim().
| const prLabels = (meta?.labels ?? []) | |
| .map((l) => l?.name?.trim().toLowerCase()) | |
| .filter((n): n is string => Boolean(n)); | |
| const prLabels = (Array.isArray(meta?.labels) ? meta.labels : []) | |
| .map((l) => typeof l?.name === 'string' ? l.name.trim().toLowerCase() : undefined) | |
| .filter((n): n is string => Boolean(n)); |
![codeant-ai[bot]](https://avatars.githubusercontent.com/in/646884?s=60&v=4){kind=small}
| interface PrMeta { | ||
| state?: string; // 'open' | 'closed' | ||
| merged?: boolean; | ||
| author?: { login?: string }; | ||
| labels?: Array<{ name?: string }>; | ||
| [key: string]: unknown; | ||
| } |
There was a problem hiding this comment.
Suggestion: The PR metadata author shape is modeled incorrectly, so the authoritative author from meta.json is never read. In this codebase the metadata author is a string login (as used by repo-hygiene), but here it is typed as an object with login; that makes meta?.author?.login resolve to undefined and forces fallback to payload-derived author values, which can be wrong on check_run.completed. Update the meta contract to match the real schema and read the login from that field directly. [api mismatch]
Severity Level: Major ⚠️
- ❌ CI-triggered PR reviews skipped for allowlisted human authors.
- ⚠️ REVIEW_AUTHORS ignores authoritative author from PR metadata.Steps of Reproduction ✅
1. Configure the pr-reviewer to use an author allowlist by setting `REVIEW_AUTHORS`
(consumed via `input()` at `review/agent.ts:52-55` and used in `reviewAuthorAllowlist()`
at `review/agent.ts:155-159`).
2. Note that `shouldSkipReview()` at `review/agent.ts:101-132` reads PR metadata via
`loadPrMeta()` (`review/agent.ts:135-146`) and expects `PrMeta.author` to be an object
with `login` (`review/agent.ts:36-42`) while the same `meta.json` endpoint is typed as
`author?: string` in `GithubPrMeta` at `repo-hygiene/agent.ts:28-35`.
3. Open a PR in a repo where CI emits `check_run.completed` events; when such an event
arrives, the handler in `review/agent.ts:50-92` calls `readPr()`
(`review/agent.ts:262-282`), which sets `pr.author` from the webhook payload
(`p?.pull_request?.user?.login ?? p?.sender?.login ?? 'unknown'`).
4. For `check_run.completed` payloads, `p.pull_request` is often absent and
`p.sender.login` is typically the app (for example, `github-actions[bot]`), so `pr.author`
is not the human PR opener, while the `meta.json` `author` field (typed as `string` in
`GithubPrMeta` at `repo-hygiene/agent.ts:28-35`) contains the correct opener login.
5. When `shouldSkipReview()` runs for that event, `loadPrMeta()` successfully returns a
`PrMeta` whose runtime `author` value is a string, making `meta?.author?.login`
(`review/agent.ts:126`) evaluate to `undefined`; the expression `(meta?.author?.login ??
pr.author)` therefore falls back to `pr.author` (the app login), causing the allowlist
check `!allow.has(author)` (`review/agent.ts:127`) to skip the review even though the
correct human author in `meta.json` would have passed `REVIEW_AUTHORS`.Fix in Cursor | Fix in VSCode Claude
(Use Cmd/Ctrl + Click for best experience)
Prompt for AI Agent 🤖
This is a comment left during a code review.
**Path:** review/agent.ts
**Line:** 36:42
**Comment:**
*Api Mismatch: The PR metadata author shape is modeled incorrectly, so the authoritative author from `meta.json` is never read. In this codebase the metadata author is a string login (as used by `repo-hygiene`), but here it is typed as an object with `login`; that makes `meta?.author?.login` resolve to `undefined` and forces fallback to payload-derived author values, which can be wrong on `check_run.completed`. Update the meta contract to match the real schema and read the login from that field directly.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix| const allow = reviewAuthorAllowlist(ctx); | ||
| if (allow.size > 0) { | ||
| const author = (meta?.author?.login ?? pr.author).trim().toLowerCase(); | ||
| if (!allow.has(author)) { | ||
| return { reason: `author @${author} is not in REVIEW_AUTHORS` }; | ||
| } |
There was a problem hiding this comment.
Suggestion: The author allowlist gate is not actually fail-open when metadata loading fails: it still enforces REVIEW_AUTHORS against fallback payload author values (often sender/unknown for non-PR payload shapes), which can incorrectly skip eligible PRs during transient meta read failures. Only enforce this gate when a reliable PR opener is available from metadata, or explicitly bypass the gate when metadata is unavailable. [logic error]
Severity Level: Major ⚠️
- ❌ Transient meta read failures can block allowed PR reviews.
- ⚠️ REVIEW_AUTHORS behaves fail-closed on metadata errors.Steps of Reproduction ✅
1. Enable an author allowlist for pr-reviewer by setting `REVIEW_AUTHORS` so that
`reviewAuthorAllowlist()` at `review/agent.ts:155-159` returns a non-empty `Set`,
activating the gate in `shouldSkipReview()` at `review/agent.ts:124-130`.
2. Observe that `loadPrMeta()` at `review/agent.ts:135-146` reads `meta.json` via
`readJsonFile<PrMeta>()` and catches all errors, returning `undefined` on any failure
(e.g., transient VFS or adapter issues).
3. When a GitHub event (for example, `check_run.completed` listed in `triggers` at
`review/agent.ts:55-60`) arrives during such a transient failure, `loadPrMeta()` returns
`undefined`, so `meta` is falsy in `shouldSkipReview()` (`review/agent.ts:101-132`): the
merged/closed and label gates effectively no-op, but the allowlist gate still executes.
4. In this meta-failure case, the allowlist gate computes `author` as
`(meta?.author?.login ?? pr.author).trim().toLowerCase()` at `review/agent.ts:126`, which
reduces to `pr.author` because `meta` is `undefined`; `pr.author` comes from `readPr()`
(`review/agent.ts:262-282`) and, for `check_run.completed`, is often the app or `sender`
login (e.g., `github-actions[bot]`) or `'unknown'`, not the actual PR opener.
5. Because `REVIEW_AUTHORS` is defined in terms of human opener logins, the
`allow.has(author)` check at `review/agent.ts:127` fails when `author` is this fallback
value, causing `shouldSkipReview()` to return a skip reason and the handler at
`review/agent.ts:79-85` to skip `reviewAndFix()` despite the PR having an allowlisted
opener; this violates the intended best-effort/fail-open behavior when metadata loading is
unreliable.Fix in Cursor | Fix in VSCode Claude
(Use Cmd/Ctrl + Click for best experience)
Prompt for AI Agent 🤖
This is a comment left during a code review.
**Path:** review/agent.ts
**Line:** 124:129
**Comment:**
*Logic Error: The author allowlist gate is not actually fail-open when metadata loading fails: it still enforces `REVIEW_AUTHORS` against fallback payload author values (often `sender`/`unknown` for non-PR payload shapes), which can incorrectly skip eligible PRs during transient meta read failures. Only enforce this gate when a reliable PR opener is available from metadata, or explicitly bypass the gate when metadata is unavailable.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
|
CodeAnt AI finished reviewing your PR. |
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
3 issues found across 11 files
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
Add three agent-side gates to the pr-reviewer before it reviews/fixes a PR: - Label gate: skip entirely when the PR carries a disabling label (configurable via the new SKIP_LABELS input; defaults to "no-agent-relay-review"). - Author allowlist: new REVIEW_AUTHORS input — when set, only review/fix PRs opened by those logins (e.g. "only my own PRs"). Unset = every author, so existing deployments are unaffected. - Already-merged/closed skip: don't post a stale review when the PR has already merged/closed by the time the handler runs (the cheap, agent-side half of the merge-race; recovery-PR preservation is tracked in AgentWorkforce/cloud#1659 / #1660). Author and labels are read from the live PR meta.json via readJsonFile, because the webhook payload doesn't carry them on every trigger (check_run.completed has neither). The read is best-effort: on failure it falls back to the payload author and proceeds. Gates apply only to the review/fix path; merge-on-approval still flows through APPROVERS unchanged. Review feedback addressed: - meta.author accepted as either a login string or { login } object, so a shape mismatch can't silently bypass the allowlist. - labels validated with Array.isArray + a string check on label.name before trimming, to avoid runtime errors on malformed metadata. - author allowlist fails open: an undeterminable author ('unknown'/empty) no longer blocks the review. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
khaliqgant
force-pushed
the
feat/pr-reviewer-gates
branch
from
d35c65a to
0696b7b
Compare
|
✅ pr-reviewer applied fixes — committed and pushed Fixed the PR gate behavior in What changed:
Validation:
|
![agent-relay-code[bot]](https://avatars.githubusercontent.com/in/3624277?s=60&v=4){kind=small}
There was a problem hiding this comment.
✅ pr-reviewer applied fixes — committed and pushed e7527d8 to this PR. The notes below describe what changed.
Fixed the PR gate behavior in review/agent.ts.
What changed:
readPrnow preservesstate,merged, andlabelsfrompull_requestpayloads.shouldSkipReviewnow falls back to webhook payload data whenmeta.jsonis unavailable.- Closed-state checks are case-insensitive.
- Skip-label parsing is centralized and still validates label shape defensively.
Validation:
npm run typecheckpassed.npm audit --omit=optional --jsonreported 0 vulnerabilities.- I did not keep a local
agentworkforceCLI dependency: the current CLI made compile pass but introduced high-severity transitive audit findings; the older CLI did not supportpersona compile.
1 participant
User description
What
Adds three agent-side gates to the pr-reviewer, evaluated before it reviews/fixes a PR:
SKIP_LABELSinput (comma-separated); defaults tono-agent-relay-review.REVIEW_AUTHORSinput. When set, only review/fix PRs opened by those GitHub logins (e.g. only my own PRs). Unset = every author, so existing deployments are unaffected (mirrors theAPPROVERSpattern).How
meta.jsonviareadJsonFile(the same patternrepo-hygieneuses), because the webhook payload doesn't carry them on every trigger —check_run.completedhas neither author nor labels.APPROVERS.tsc --noEmitpasses.Scope / follow-ups
The already-merged skip is the cheap, agent-side half of the merge-race. Preserving the unpushed fixes via a recovery PR + Slack confirm needs platform work that isn't possible in this repo (no durable cross-turn session, and inbound Slack messages aren't delivered to agents) — tracked in AgentWorkforce/cloud#1659 (Slack delivery) and AgentWorkforce/cloud#1660 (CF continuation runtime).
Also drops a stale Daytona/Codex sandbox comment in
persona.tsthat was already modified in the working tree.🤖 Generated with Claude Code
Summary by cubic
Add agent-side gates to the PR reviewer to skip runs based on labels, author allowlist, and already-merged state, reducing noisy or stale reviews.
SLACK_CHANNEL.SKIP_LABELS(default:no-agent-relay-review).REVIEW_AUTHORS(comma-separated). Unset = review everyone.pulls/{n}/meta.jsonwith fail-open fallback; acceptsmeta.authoras a string or{ login }, validates label names, and fails open when author is unknown; gates only affect review/fix and do not change merge-on-approval viaAPPROVERS.REVIEW_AUTHORSandSKIP_LABELS.Written for commit e7527d8. Summary will update on new commits.
CodeAnt-AI Description
Skip review runs for closed PRs and PRs that should not be reviewed
What Changed
Impact
✅ Fewer stale PR reviews✅ Fewer unwanted review runs✅ Clearer skip messages in Slack💡 Usage Guide
Checking Your Pull Request
Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.
Talking to CodeAnt AI
Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:
This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.
Example
Preserve Org Learnings with CodeAnt
You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:
This helps CodeAnt AI learn and adapt to your team's coding style and standards.
Example
Retrigger review
Ask CodeAnt AI to review the PR again, by typing:
Check Your Repository Health
To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.