diff --git a/README.md b/README.md index 68844e3..68683cd 100644 --- a/README.md +++ b/README.md @@ -1,159 +1,420 @@ -# crq β€” CodeRabbit review queue +

🐰 crq β€” CodeRabbit review queue

-**An account-wide, cross-host coordinator so many parallel coding agents don't compete for one shared CodeRabbit PR-review rate limit.** +

Stop your AI agents from fighting over one CodeRabbit rate limit.

-If you run several autonomous agents (Claude Code, etc.), each looping on its own -pull request and periodically posting `@coderabbitai review`, they will fight over -your **per-developer / per-organization** CodeRabbit quota. On the Pro plan at the -lowest adaptive refill rate that quota is effectively *one review at a time, hours -apart* β€” so the agents (a) keep firing review requests while globally rate-limited -(noise), and (b) stampede the moment the window reopens. - -`crq` fixes this. Agents call `crq wait ` instead of posting the review -command directly. `crq` then: - -- knows the **true, account-wide** rate-limit state (it *asks* CodeRabbit, it doesn't - guess from one PR's stale countdown), -- **queues** review requests and fires them **FIFO, one at a time**, the instant the - window opens β€” no stampede, -- keeps a **live GitHub issue dashboard** of the queue and status, -- works **across machines** (laptop, WSL, a server) because all shared state lives on - GitHub. - -There is no off-the-shelf tool for this; community "rate-limit" helpers are -single-agent. `crq` is built for the multi-agent, multi-host case. +

+One shared queue for your whole CodeRabbit account, so parallel agents request reviews +in an orderly line β€” one at a time, only when there's capacity β€” instead of stampeding. +

--- -## How it works +## The problem (in plain English) + +You've got several AI coding agents working at once β€” each on its own pull request, each looping: +*fix code β†’ push β†’ ask CodeRabbit to review β†’ read feedback β†’ repeat.* To ask for a review, an +agent posts `@coderabbitai review` on its PR. + +Here's the catch: **CodeRabbit's review limit is per *account/organization*, not per PR.** On the +Pro plan, once you've been reviewing a lot, the refill rate drops β€” at the slowest tier it's +effectively **one review at a time, hours apart**, shared across *all* your PRs. + +So your agents collide: + +- πŸ” **They spam while blocked.** Agent A's PR is "available in 3 hours," but Agent B has no idea β€” + it keeps posting `@coderabbitai review` on *its* PR and getting rate-limited too. Noise everywhere. +- 🐎 **They stampede when the window opens.** The moment one review slot frees up, every agent fires + at once. One wins; the rest waste the slot and trip the limit again. +- 🀷 **Nobody knows the real state.** Each agent only sees its own PR's stale countdown, but the + limit is account-wide β€” so that countdown is usually wrong. + +The result: wasted quota, redundant requests, and reviews landing in a random order. + +## What crq does + +`crq` puts **one queue in front of your whole account.** Agents don't post `@coderabbitai review` +themselves anymore β€” they ask `crq`, and `crq`: + +- 🧠 **Knows the real limit** β€” it *asks CodeRabbit directly* (`@coderabbitai rate limit`, which + doesn't cost a review) instead of guessing from a stale comment. +- 🚦 **Serializes everything** β€” a single GitHub-native lock means reviews fire **one at a time, in + FIFO order, only when the account is actually unblocked.** No stampede, no spam. +- πŸ“Š **Shows you the line** β€” a live GitHub issue is the dashboard: who's queued, what's in flight, + and when the next slot opens. +- 🌍 **Works across machines** β€” all state lives on GitHub, so agents on your laptop, a server, and + a CI box all share the same queue with zero extra infrastructure. + +One agent changes one line β€” `gh pr comment ... @coderabbitai review` becomes `crq wait ` β€” +and the chaos is gone. + +> ## ⚠️ Required: turn OFF CodeRabbit auto-review +> +> crq can only control *when* reviews happen if CodeRabbit isn't reviewing on its own. +> **If auto-review is enabled, CodeRabbit reviews every push automatically β€” bypassing crq's +> queue entirely and spending your shared rate limit outside it.** That defeats the whole point: +> your agents would still stampede and rate-limit themselves, crq or not. +> +> So crq's model is **pull, not push** β€” reviews fire *only* when crq (or you) explicitly post +> `@coderabbitai review`. Disable auto-review before relying on crq: +> +> - **Account / org-wide (recommended):** CodeRabbit dashboard β†’ your organization β†’ +> **Settings β†’ Review β†’ Automatic Review** β†’ turn it **off** (also disable incremental/auto +> reviews). This is the setting that matters most. +> - **Or per repository:** commit a `.coderabbit.yaml` with: +> +> ```yaml +> reviews: +> auto_review: +> enabled: false +> ``` +> +> To have reviews happen automatically across your PRs, use **`crq autoreview`** instead β€” it does +> the same job rate-coordinated. See [Review all your PRs automatically](#review-all-your-prs-automatically). -All coordination state lives in one small **gate repo** on GitHub (private is fine): +## How it works -| Piece | What it is | -|-------|------------| -| **Lock** | An atomic git ref (`refs/heads/crq-lock`). Created via the GitHub *create-if-not-exists* ref API; the holder is proven by a nonce in the lock commit. This is a real distributed mutex β€” every state mutation and every review-fire happens under it, so there are no lost updates and **at most one** review is ever fired at a time across all hosts. | -| **Dashboard issue** | Its body holds the queue/status as JSON (in a hidden HTML comment) plus a rendered table. The issue **title** shows a one-glance status. | -| **Calibration PR** | A throwaway draft PR where `crq` posts `@coderabbitai rate limit` to read your remaining quota + refill time **without consuming a review**. Auto-review is disabled on the gate repo so this PR costs nothing. | +```text + agent A ─┐ + agent B ─┼─► crq wait + agent C β”€β”˜ β”‚ + β–Ό + β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” asks "any capacity?" + β”‚ global lock (a git ref) β”‚ ───────────────────────► CodeRabbit + β”‚ + FIFO queue (in an issue) β”‚ ◄─────────────────────── "available now" / "in 3h" + β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ + β”‚ when unblocked, fire the FIFO head β€” exactly one + β–Ό + posts "@coderabbitai review" on the next PR in line +``` -The rate-limit reading is **calibration-first** (ask CodeRabbit, cached and throttled -so command volume stays tiny) and corroborated by scanning your open PRs for -CodeRabbit's "rate limited… available in X" warning comments. The max of the two is -the account-wide `blocked_until`. +Everything lives in one small **gate repo** (private is fine): -FIFO order uses a monotonic integer `seq` assigned under the lock (not timestamps, so -it is immune to clock skew between hosts). +| Piece | What it is | +|-------|-----------| +| πŸ”’ **Lock** | An atomic git ref. GitHub's "create ref only if it doesn't exist" gives a real cross-machine mutex, so only one agent acts at a time and the queue never corrupts. | +| πŸ“‹ **Dashboard** | Published to the gate repo's **`README.md`** β€” status, queue, and a "recently reviewed" history, every PR linked (committed only when something material changes, not on every tick). A tracking **issue** holds the machine-readable state (its **title** is a one-glance status: `crq Β· 🟑 2 queued`). | +| 🐰 **Calibration PR** | A throwaway draft PR where crq asks `@coderabbitai rate limit` to read your real quota *without spending a review*. (crq disables auto-review on this repo so the PR itself costs nothing.) | --- -## Install +## Quick start -Requires [`gh`](https://cli.github.com/) (authenticated) and [`jq`](https://jqlang.github.io/jq/). +**1. Install** (needs [`gh`](https://cli.github.com/) logged in, and [`jq`](https://jqlang.github.io/jq/)): ```bash curl -fsSL https://raw.githubusercontent.com/kristofferR/coderabbit-queue/main/install.sh | bash ``` -or manually: +
+Manual install (if you'd rather not pipe a script from the net) + +`crq` is a single self-contained bash script β€” read it first if you like, then drop it on your PATH: ```bash +# clone (or just download the one file) and review it git clone https://github.com/kristofferR/coderabbit-queue.git -install -m 0755 coderabbit-queue/crq ~/.local/bin/crq # ensure ~/.local/bin is on PATH +less coderabbit-queue/crq + +# install it onto your PATH (make sure ~/.local/bin is on $PATH) +install -m 0755 coderabbit-queue/crq ~/.local/bin/crq + +# or without cloning β€” fetch just the CLI: +# curl -fsSL https://raw.githubusercontent.com/kristofferR/coderabbit-queue/main/crq -o ~/.local/bin/crq +# chmod +x ~/.local/bin/crq + +crq help # verify (needs gh + jq installed) ``` +
-## Setup +**2. Create your queue** (one private repo holds the lock + dashboard + calibration PR): ```bash -export CRQ_REPO=youruser/crq-state # the gate repo crq will create/use (private OK) -crq init # creates the repo, dashboard issue, calibration PR +export CRQ_REPO=YOURUSER/crq-state +crq init ``` -`crq init` prints the `export CRQ_*` lines to save (e.g. in `~/.config/crq/env`, which -`crq` sources automatically). Put the same config on every host that runs agents. +`crq init` creates the repo, opens the calibration PR and dashboard issue, publishes the dashboard +to the repo's README, prints the `export CRQ_*` lines to save, and opens the gate repo in your +browser (set `CRQ_NO_OPEN=1` to skip that β€” e.g. on a headless box). Drop them into `~/.config/crq/env` β€” crq sources that file +automatically, so every machine just needs the same four lines: ```bash mkdir -p ~/.config/crq -crq init >> ~/.config/crq/env # then trim it to just the export lines +cat > ~/.config/crq/env <<'EOF' +export CRQ_REPO=YOURUSER/crq-state +export CRQ_ISSUE=2 +export CRQ_CAL_PR=1 +export CRQ_SCOPE=YOURUSER +EOF +``` + +> **One-time:** make sure CodeRabbit is installed on the gate repo (so it can answer +> `@coderabbitai rate limit` on the calibration PR). If your CodeRabbit covers "all repositories" +> you're already done; otherwise add `crq-state` in the CodeRabbit dashboard. +> +> `crq init` also sets the gate repo to **Watch β†’ Ignore**, so its calibration PR (where crq +> posts `@coderabbitai rate limit`) never emails you β€” the repo is machine-only and you never +> read it. This is essential: crq posts those comments _as you_, which re-subscribes you, so a +> per-PR unsubscribe wouldn't hold; ignoring the whole repo is the durable fix. If `crq init` +> can't set it (rare), do it by hand: the repo's **Watch β–Ύ β†’ Ignore**. + +**3. Use it.** In any review loop, replace this: + +```bash +gh pr comment "$PR" --repo "$REPO" --body "@coderabbitai review" # ❌ competes with other agents ``` -## Usage +with this: ```bash -crq wait # enqueue and BLOCK until OUR review is fired β€” use this in agent loops -crq enqueue # add to the FIFO queue (idempotent) -crq pump # fire <=1 queued review if the window is open (safe to call anywhere) -crq status # show the dashboard -crq refresh # force a fresh rate-limit calibration, then show status -crq cancel # remove from queue / clear a stuck in-flight entry -crq gc # drop closed/merged PRs; clear a timed-out in-flight -crq unlock [--force] # inspect / break the global lock +crq wait "$REPO" "$PR" # βœ… joins the queue, blocks until YOUR review is actually fired ``` -`` is `owner/name`, `` is the number. +That's the whole integration. `crq wait` gets in line, waits its turn, and posts the review command +for you β€” exactly once, only when CodeRabbit has capacity. -### Integrate into an agent review loop +--- + +## Review all your PRs automatically -Replace any direct `@coderabbitai review` posting with a single `crq wait` call: +`crq autoreview` reviews all your open PRs automatically, rate-coordinated. Run it as a background watcher: ```bash -while review_cycle_should_continue; do - crq wait "$REPO" "$PR" # blocks until OUR review is fired β€” FIFO, no competition - # crq has now posted the review command exactly once, only while unblocked. - wait_for_review_to_land "$REPO" "$PR" # your logic: read CodeRabbit's review - apply_fixes_and_push # do the work, then loop to re-enqueue +crq autoreview # auto-review every open PR + re-review on each push (FIFO, rate-aware) +crq autoreview --no-incremental # auto-review each PR ONCE only β€” no re-review on later pushes +crq autoreview --once # a single pass (e.g. from cron or your monitor) +``` + +By default `autoreview` covers every open PR in `CRQ_SCOPE`. To limit it to specific repos, set an +allowlist (`CRQ_REPOS=owner/a,owner/b`) β€” or exclude a few with a denylist (`CRQ_EXCLUDE=owner/c`). + +Each pass enqueues any open PR in scope (minus deny / outside allow) whose latest commit CodeRabbit hasn't reviewed yet +(a brand-new PR β†’ its first review; new commits β†’ an incremental review), then fires them FIFO until +**every** PR is reviewed. The two flags mirror CodeRabbit's own toggles: default = *Automatic + +Incremental review*; `--no-incremental` = *Automatic review* only. (The gate repo itself is never auto-reviewed.) + +The guarantee is that **all your PRs get auto-reviewed** β€” none are skipped. The queue isn't a cap; +it only orders the reviews so they never exceed your rate limit. (`crq wait` is the on-demand version +for when an agent needs *its* PR reviewed right now.) `autoreview` and `crq wait` never double-post: +crq records the commit it requested a review for, so the same commit is never reviewed twice. + +
+Run it persistently (macOS launchd / Linux systemd) + +So it survives reboots and always keeps your PRs reviewed, run `crq autoreview` as a service. + +**macOS β€” a LaunchAgent** at `~/Library/LaunchAgents/
+ +--- + +## ⭐ The recommended PR-review loop + +Here's the autonomous review loop this tool was built for. Run one per PR β€” on as many PRs and +machines as you like β€” and they'll all share the queue without competing. + +```bash +#!/usr/bin/env bash +# review-loop.sh β€” autonomously address CodeRabbit feedback until a PR converges. +# REPO=owner/name PR=123 ./review-loop.sh +set -uo pipefail +REPO="${REPO:?set REPO=owner/name}"; PR="${PR:?set PR=}" + +# Keep looping unless the PR is *explicitly* CLOSED/MERGED β€” a transient gh/API failure returns an +# empty state, which must NOT be mistaken for a real closure (it would exit the loop early). +still_open() { local s; s="$(gh pr view "$PR" --repo "$REPO" --json state -q .state 2>/dev/null)"; [ "$s" != "CLOSED" ] && [ "$s" != "MERGED" ]; } + +while still_open; do + # 1. Ask for a review β€” coordinated, FIFO, never fires while rate-limited. Exit codes: + # 0 = our review was fired 3 = deduped (this HEAD was already reviewed) other = timeout/error + crq wait "$REPO" "$PR"; rc=$? + case "$rc" in + 0) ;; # fired -> wait for feedback below + 3) process_review_and_push "$REPO" "$PR"; sleep "${LOOP_IDLE_SLEEP:-60}"; continue ;; # already reviewed -> process existing, back off + *) sleep "${LOOP_IDLE_SLEEP:-60}"; continue ;; # timeout/error -> back off, skip + esac + + # Start the feedback window AFTER crq fires β€” otherwise a delayed response from a previous round + # that lands while we were blocked in `crq wait` would be newer than $since and falsely satisfy + # the poll below before our just-requested review actually runs. + since="$(date -u +%Y-%m-%dT%H:%M:%SZ)" + + # 2. Wait for CodeRabbit's review to land (give it ~20 min). It can land as a conversation comment + # OR a formal PR review, so check both. A rate-limit WARNING is NOT feedback β€” exclude it. + got="" + for _ in $(seq 1 40); do + # --slurp + a standalone jq with `add`: combine all pages before counting (gh forbids + # --slurp with --jq, so pipe to jq; plain --paginate --jq counts per page). + new=$(gh api "repos/$REPO/issues/$PR/comments" --paginate --slurp \ + | jq "add | map(select(.user.login==\"coderabbitai[bot]\" and .created_at > \"$since\" and (.body|contains(\"rate limited by coderabbit.ai\")|not))) | length") + rev=$(gh api "repos/$REPO/pulls/$PR/reviews" --paginate --slurp \ + | jq "add | map(select(.user.login==\"coderabbitai[bot]\" and .submitted_at > \"$since\")) | length") + { [ "${new:-0}" -gt 0 ] || [ "${rev:-0}" -gt 0 ]; } && { got=1; break; } + sleep 30 + done + [ -n "$got" ] || continue # nothing landed within the cap β€” don't push a round on stale feedback + + # 3. Do the work: read the findings, fix the real ones, run your tests/linters, commit & push. + # (Pushing a new commit is what makes the next CodeRabbit review meaningful.) + process_review_and_push "$REPO" "$PR" # <- your logic here + + # 4. Loop: the next round re-enters the queue and waits again. done +echo "βœ… $REPO#$PR converged or closed." ``` -See [`examples/agent-loop.sh`](examples/agent-loop.sh) for a fuller version. +**Running fully unattended?** Add the lightweight **monitor** ([`examples/monitor.sh`](examples/monitor.sh)): +a background watcher that wakes you when new bot feedback lands and, between rounds, keeps the PR in +the queue with the non-blocking `crq enqueue` + `crq pump`. It never posts `@coderabbitai review` +directly β€” `crq` owns that, account-wide. + +> πŸ’‘ **Watching the line:** run `crq status` any time to see the queue, what's in flight, and the +> next slot. Or just open your gate repo on GitHub β€” its README **is** the live dashboard. --- +## Commands + +```bash +crq wait # ⭐ enqueue + block until OUR review is fired (use this in loops) +crq autoreview # ⭐ review ALL open PRs automatically, rate-coordinated + # (--no-incremental = first review only; --once = single pass for cron) +crq enqueue # add to the queue and return immediately (idempotent) +crq pump # fire the next review if the window is open (safe to call from anywhere) +crq status # show the dashboard: queue, in-flight, quota, next slot +crq refresh # re-check the account quota right now, then show status +crq cancel # take a PR out of the line +crq gc # tidy up: drop closed/merged PRs, clear a stuck in-flight entry +crq unlock [--force] # inspect (or break) the global lock if something wedged +crq init # first-time setup of the gate repo +crq version # print the version +crq help # this list +``` + +`` is `owner/name`; `` is the number. **Exit codes:** `crq wait` returns `0` once your +review is fired, `3` if this commit was already reviewed (deduped β€” no new review fired, so process +the existing feedback), or `2` if `CRQ_WAIT_TIMEOUT` is hit; other commands return `0` on success. + ## Configuration -All via environment (or `~/.config/crq/env`, sourced automatically): +Set these in `~/.config/crq/env` (sourced automatically) or as environment variables: -| Var | Default | Meaning | -|-----|---------|---------| -| `CRQ_REPO` | *(required)* | gate repo holding lock ref + dashboard issue + calibration PR | +| Variable | Default | What it does | +|----------|---------|--------------| +| `CRQ_REPO` | *(required)* | the gate repo (`owner/name`) holding the lock, dashboard, calibration PR | | `CRQ_ISSUE` | from `init` | dashboard issue number | | `CRQ_CAL_PR` | from `init` | calibration PR number | -| `CRQ_SCOPE` | owner of `CRQ_REPO` | comma-list of owners/orgs = rate-limit buckets (one gate each) | -| `CRQ_CALIBRATE_TTL` | `120` | max age (s) of the cached calibration before re-asking | -| `CRQ_LOCK_TTL` | `180` | seconds before a held lock is considered stale and stealable | -| `CRQ_MIN_INTERVAL` | `90` | minimum seconds between fires | -| `CRQ_INFLIGHT_TIMEOUT` | `900` | backstop to clear a stuck in-flight review | -| `CRQ_POLL` | `15` | `crq wait` poll interval (+ jitter) | -| `CRQ_WAIT_TIMEOUT` | `0` | give up `wait` after N seconds (0 = never) | -| `CRQ_WARNING_SCAN` | `1` | also corroborate via open-PR warning comments (`0` to disable) | -| `CRQ_BOT` | `coderabbitai[bot]` | review-bot login to watch | -| `CRQ_REVIEW_CMD` | `@coderabbitai review` | command posted to fire a review | -| `CRQ_RATELIMIT_CMD` | `@coderabbitai rate limit` | non-consuming quota-check command | -| `CRQ_RL_MARKER` | `rate limited by coderabbit.ai` | substring identifying a rate-limit warning comment | - -### Other review bots / multiple orgs - -`crq` is bot-agnostic: point `CRQ_BOT`, `CRQ_REVIEW_CMD`, `CRQ_RATELIMIT_CMD`, and -`CRQ_RL_MARKER` at any bot with a similar command surface. - -CodeRabbit's quota is per **organization**, so PRs in different orgs draw from -*separate* buckets. Run one independent gate per org (a distinct `CRQ_REPO` / -`CRQ_SCOPE`); don't lump unrelated orgs into one queue or you'll serialize reviews -that don't actually compete. +| `CRQ_SCOPE` | owner of `CRQ_REPO` | which owners/orgs share this quota (comma-separated) | +| `CRQ_REPOS` | _(all in scope)_ | `autoreview` allowlist β€” only review these `owner/name` repos (comma-separated) | +| `CRQ_EXCLUDE` | _(none)_ | `autoreview` denylist β€” never review these `owner/name` repos (comma-separated) | +| `CRQ_TZ` | `UTC` | dashboard display timezone (IANA name, e.g. `Europe/Oslo`) | +| `CRQ_NO_OPEN` | _(unset)_ | set to skip opening the gate repo in a browser after `crq init` | +| `CRQ_CALIBRATE_TTL` | `120` | how long (s) to trust a quota reading before re-asking CodeRabbit | +| `CRQ_MIN_INTERVAL` | `90` | minimum seconds between fired reviews | +| `CRQ_POLL` | `15` | how often (s) `crq wait` checks its place in line | +| `CRQ_WAIT_TIMEOUT` | `0` | give up `crq wait` after N seconds (`0` = never) | +| `CRQ_LOCK_TTL` | `180` | when a crashed lock holder is considered gone and stealable | +| `CRQ_INFLIGHT_TIMEOUT` | `900` | backstop to release a stuck in-flight review | +| `CRQ_WARNING_SCAN` | `1` | also corroborate via open-PR "rate limited" comments (`0` to disable) | + +**Other review bots:** crq isn't CodeRabbit-specific. Point `CRQ_BOT`, `CRQ_REVIEW_CMD`, +`CRQ_RATELIMIT_CMD`, and `CRQ_RL_MARKER` at any bot with a similar command surface. + +**Multiple orgs:** CodeRabbit's quota is per-org, so PRs in different orgs draw from *different* +buckets. Run a separate gate (its own `CRQ_REPO`) per org rather than mixing them β€” otherwise you'd +serialize reviews that don't actually compete. + +--- + +## πŸ€– For AI agents (LLM-friendly cheat sheet) + +If you're an autonomous agent running a PR-review loop, here's everything you need: + +- **The one rule:** never post `@coderabbitai review` yourself. To request a review, run + `crq wait "" ""`. It blocks until the review is actually fired, then returns `0`. +- **Check state without acting:** `crq status` (human-readable) shows the queue, what's in flight, + remaining quota, and when the next slot opens. +- **Don't busy-wait or retry on your own** β€” `crq wait` already handles the waiting, ordering, and + rate-limit backoff. Call it once per review round. +- **Setup check:** if `crq` says `CRQ_REPO is not set`, run the Quick Start above (install + `crq init`) + once, then retry. + +A drop-in **[Claude Code skill](skills/coderabbit-queue/SKILL.md)** is included β€” copy +`skills/coderabbit-queue/` into your agent's skills directory and it'll know when and how to use `crq`. --- -## Notes & caveats +## Troubleshooting + +| Symptom | Fix | +|---------|-----| +| `crq status` shows `source: warning` and never `calibrate` | CodeRabbit isn't answering on the calibration PR β€” make sure it's **installed on the gate repo**. crq still works via the warning-scan fallback. | +| A PR is stuck "in flight" forever | `crq gc` clears it after `CRQ_INFLIGHT_TIMEOUT`; or `crq cancel `. | +| "could not acquire lock" | A crashed holder. `crq unlock` shows who holds it; `crq unlock --force` breaks it. | +| Reviews fire slower than expected | That's the point β€” you're rate-limited. `crq status` shows the real countdown from CodeRabbit. | + +## How the lock works (for the curious) -- **`@coderabbitai rate limit` is PR-scoped and non-consuming**, but subject to a - commands-per-minute cap and a chat rate limit β€” hence the centralized, cached - `CRQ_CALIBRATE_TTL` throttle (only the lock holder ever asks; everyone else reads the - cached value). -- The exact wording of CodeRabbit's `rate limit` reply and its review-acknowledgement - reaction may change; `crq`'s parser and in-flight detector are deliberately tolerant - and fall back to a warning-scan + timeout. Tune `parse_quota` / `inflight_status` in - `crq` if your account shows different wording. -- The gate repo must have CodeRabbit **auto-review disabled** (`crq init` commits a - `.coderabbit.yaml` that does this) so the calibration PR never burns a real review. +GitHub's `POST /git/refs` atomically creates a ref *only if it doesn't already exist* β€” a genuine +distributed mutex. The lock ref points at a tiny commit whose message carries a random **nonce**; +after creating the ref, crq reads it back and checks the nonce, so even under a thundering herd +exactly one agent ever owns the lock (and on release it only deletes a lock that's still *its own*). +A crashed holder is reclaimed after `CRQ_LOCK_TTL`. Every queue change happens under this lock, so +the dashboard has never lost updates, and FIFO order uses a monotonic counter (not timestamps) so +it's immune to clock differences between machines. ## License -MIT Β© Kristoffer Risanger. See [LICENSE](LICENSE). +MIT Β© Kristoffer Risanger. See [LICENSE](LICENSE). Contributions welcome. diff --git a/crq b/crq index 53add32..85f7a80 100755 --- a/crq +++ b/crq @@ -12,6 +12,8 @@ # Docs: https://github.com/kristofferR/coderabbit-queue set -uo pipefail +CRQ_VERSION="1.0.0" + # ---------------------------------------------------------------------------- # Config (env, overridable). A config file is sourced first if present. # ---------------------------------------------------------------------------- @@ -22,22 +24,36 @@ CRQ_REPO="${CRQ_REPO:-}" # owner/name holding lock + dashboard + CRQ_ISSUE="${CRQ_ISSUE:-}" # dashboard issue number CRQ_CAL_PR="${CRQ_CAL_PR:-}" # calibration PR number CRQ_SCOPE="${CRQ_SCOPE:-${CRQ_REPO%%/*}}" # comma-list of owners/orgs = rate-limit buckets +CRQ_REPOS="${CRQ_REPOS:-}"; CRQ_REPOS="${CRQ_REPOS// /}" # autoreview allowlist (owner/name,…); empty = all repos in scope +CRQ_EXCLUDE="${CRQ_EXCLUDE:-}"; CRQ_EXCLUDE="${CRQ_EXCLUDE// /}" # autoreview denylist (owner/name,…) +# lowercased copies for case-insensitive matching (GitHub owner/repo names fold case) +CRQ_REPO_LC="$(printf '%s' "$CRQ_REPO" | tr 'A-Z' 'a-z')" +CRQ_REPOS_LC="$(printf '%s' "$CRQ_REPOS" | tr 'A-Z' 'a-z')" +CRQ_EXCLUDE_LC="$(printf '%s' "$CRQ_EXCLUDE" | tr 'A-Z' 'a-z')" +CRQ_TZ="${CRQ_TZ:-}" # dashboard display timezone (IANA, e.g. Europe/Oslo); empty = UTC CRQ_BOT="${CRQ_BOT:-coderabbitai[bot]}" CRQ_REVIEW_CMD="${CRQ_REVIEW_CMD:-@coderabbitai review}" CRQ_RATELIMIT_CMD="${CRQ_RATELIMIT_CMD:-@coderabbitai rate limit}" CRQ_RL_MARKER="${CRQ_RL_MARKER:-rate limited by coderabbit.ai}" CRQ_CAL_REPLY_MARKER="${CRQ_CAL_REPLY_MARKER:-auto-generated reply by CodeRabbit}" +CRQ_REVIEW_DONE_MARKER="${CRQ_REVIEW_DONE_MARKER:-summarize by coderabbit.ai}" # walkthrough/summary marker = a review happened (incl. comment-only) CRQ_CALIBRATE_TTL="${CRQ_CALIBRATE_TTL:-120}" CRQ_LOCK_TTL="${CRQ_LOCK_TTL:-180}" CRQ_LOCK_TRIES="${CRQ_LOCK_TRIES:-40}" # max acquire attempts before giving up -CRQ_LOCK_BACKOFF_MAX="${CRQ_LOCK_BACKOFF_MAX:-20}" # cap (s) on exponential backoff +CRQ_LOCK_BACKOFF_MAX="${CRQ_LOCK_BACKOFF_MAX:-8}" # cap (s) on exponential backoff CRQ_MIN_INTERVAL="${CRQ_MIN_INTERVAL:-90}" CRQ_INFLIGHT_TIMEOUT="${CRQ_INFLIGHT_TIMEOUT:-900}" CRQ_POLL="${CRQ_POLL:-15}" CRQ_WAIT_TIMEOUT="${CRQ_WAIT_TIMEOUT:-0}" # 0 = no timeout CRQ_LOCK_REF="${CRQ_LOCK_REF:-crq-lock}" CRQ_WARNING_SCAN="${CRQ_WARNING_SCAN:-1}" # 1 = corroborate with open-PR WARNING comments +CRQ_WARNING_SCAN_LIMIT="${CRQ_WARNING_SCAN_LIMIT:-40}" # max PRs the under-lock WARNING scan probes (keep well under CRQ_LOCK_TTL) +CRQ_FIRED_MAX="${CRQ_FIRED_MAX:-500}" # cap on retained fired-markers (bounds dashboard JSON growth) CRQ_GC_TTL="${CRQ_GC_TTL:-120}" # min seconds between closed-PR queue prunes +CRQ_AUTOREVIEW_POLL="${CRQ_AUTOREVIEW_POLL:-60}" # `crq autoreview` poll interval (s) +CRQ_AUTOREVIEW_MAX_SCAN="${CRQ_AUTOREVIEW_MAX_SCAN:-400}" # max PRs that incur API lookups per autoreview pass (REST secondary-limit guard) +CRQ_LEADER_REF="${CRQ_LEADER_REF:-crq-leader}" # ref electing the single account-wide autoreview daemon +CRQ_LEADER_TTL="${CRQ_LEADER_TTL:-180}" # s before a leader's stale heartbeat can be stolen (> AUTOREVIEW_POLL) CRQ_HOST="${CRQ_HOST:-$(hostname 2>/dev/null || echo unknown)}" EMPTY_TREE="4b825dc642cb6eb9a060e54bf8d69288fbee4904" # universal empty-tree object @@ -96,17 +112,32 @@ parse_available_in() { # Ownership is proven by a nonce read back from the lock commit message, which # defeats steal races (a loser never deletes the winner's fresh lock). # ---------------------------------------------------------------------------- -_lock_commit_msg() { gh api "repos/$CRQ_REPO/git/commits/$1" --jq .message 2>/dev/null; } -_lock_ref_sha() { gh api "repos/$CRQ_REPO/git/refs/heads/$CRQ_LOCK_REF" --jq .object.sha 2>/dev/null; } +_lock_commit_msg() { gh api "repos/$CRQ_REPO/git/commits/$1" --jq '.message // empty' 2>/dev/null; } +# Returns the lock commit SHA, or nothing if the ref is absent. NOTE: `gh api` dumps +# the error body to stdout and skips --jq on HTTP 404, so we must validate that the +# output is actually a 40-char hex sha (else a missing ref looks "held"). +_lock_ref_sha() { + local out + out="$(gh api "repos/$CRQ_REPO/git/refs/heads/$CRQ_LOCK_REF" --jq '.object.sha // empty' 2>/dev/null)" + case "$out" in *[!0-9a-f]*|"") return 0 ;; esac + printf '%s' "$out" +} + +# Read the acquire-epoch we embed in the lock commit message (our format, one fetch). +_lock_epoch() { printf '%s' "$1" | sed -n 's/.*epoch=\([0-9][0-9]*\).*/\1/p'; } acquire() { # echoes nonce on success - local nonce msg sha cur age cdate tries=0 backoff=1 - nonce="$(uuidgen)"; msg="crq-lock host=$CRQ_HOST pid=$$ nonce=$nonce" + local nonce msg sha cur hepoch cdate tries=0 backoff=1 + nonce="$(uuidgen 2>/dev/null)" + [ -n "$nonce" ] || nonce="$(LC_ALL=C tr -dc 'a-f0-9' /dev/null | head -c 32)" + [ -n "$nonce" ] || nonce="crq-$$-$(date -u +%s)-${RANDOM}${RANDOM}" + [ -n "$nonce" ] || return 1 # never proceed with an empty nonce β€” it would match any lock while [ "$tries" -lt "$CRQ_LOCK_TRIES" ]; do tries=$(( tries + 1 )) cur="$(_lock_ref_sha)" if [ -z "$cur" ]; then - # ref is free: create our lock commit (server-stamps committer.date) and claim it + # ref is free: create our lock commit (acquire epoch embedded for TTL) and claim it + msg="crq-lock host=$CRQ_HOST pid=$$ nonce=$nonce epoch=$(now)" sha="$(gh api -X POST "repos/$CRQ_REPO/git/commits" -f message="$msg" -f tree="$EMPTY_TREE" --jq .sha 2>/dev/null)" if [ -n "$sha" ] && gh api -X POST "repos/$CRQ_REPO/git/refs" \ -f ref="refs/heads/$CRQ_LOCK_REF" -f sha="$sha" >/dev/null 2>&1; then @@ -116,11 +147,18 @@ acquire() { # echoes nonce on success fi fi else - # held: steal only if stale; delete AT MOST ONCE then retry (ownership proven by read-back) - cdate="$(gh api "repos/$CRQ_REPO/git/commits/$cur" --jq .committer.date 2>/dev/null)" - if [ -n "$cdate" ] && [ $(( $(now) - $(iso2epoch "$cdate") )) -gt "$CRQ_LOCK_TTL" ]; then - age=$(( $(now) - $(iso2epoch "$cdate") )) - log "stealing stale lock (age ${age}s > ${CRQ_LOCK_TTL}s)" + # held: steal only if demonstrably stale; delete AT MOST ONCE then retry + # (ownership is proven by the nonce read-back above, so a steal race is safe). + # Age the lock by GitHub's own commit timestamp β€” authoritative server time, immune to any + # single host's clock skew β€” NOT the client epoch embedded in the message (a host whose + # server-time sample fell back to a wrong local clock could otherwise mis-judge staleness + # and delete a live lock). epoch= stays in the message for debugging only. + hepoch="" + cdate="$(gh api "repos/$CRQ_REPO/git/commits/$cur" --jq '.committer.date // empty' 2>/dev/null)" + [ -n "$cdate" ] && hepoch="$(iso2epoch "$cdate")" + [ -n "$hepoch" ] || hepoch="$(_lock_epoch "$(_lock_commit_msg "$cur")")" # fallback if the commit fetch failed + if [ -n "$hepoch" ] && [ $(( $(now) - hepoch )) -gt "$CRQ_LOCK_TTL" ]; then + log "stealing stale lock (age $(( $(now) - hepoch ))s > ${CRQ_LOCK_TTL}s)" gh api -X DELETE "repos/$CRQ_REPO/git/refs/heads/$CRQ_LOCK_REF" >/dev/null 2>&1 continue fi @@ -135,6 +173,7 @@ acquire() { # echoes nonce on success release() { # $1 = nonce; only deletes if we still own it local cur + [ -n "${1:-}" ] || return 0 # never delete on an empty nonce β€” it would match any lock cur="$(_lock_ref_sha)" || return 0 [ -n "$cur" ] || return 0 if _lock_commit_msg "$cur" | grep -q "nonce=$1"; then @@ -142,12 +181,83 @@ release() { # $1 = nonce; only deletes if we still own it fi } +# ---------------------------------------------------------------------------- +# Singleton: exactly ONE autoreview daemon should drive an account (CRQ_REPO). +# - per host: a pidfile so a second `crq autoreview` on the same machine exits. +# - account-wide: a leader ref ($CRQ_LEADER_REF) in the gate repo claimed via +# atomic create-if-not-exists, heartbeated each pass, and stealable only when +# its commit timestamp goes stale (> CRQ_LEADER_TTL). Across all hosts exactly +# one daemon leads; the rest stand by and fail over if the leader dies. +# (Agent `crq wait`/`enqueue`/`pump` are NOT daemons β€” they stay parallel, serialized +# only by the state lock.) +# ---------------------------------------------------------------------------- +_LEADER_ID="host=$CRQ_HOST pid=$$" + +_host_singleton() { # 0 = we hold this host's autoreview slot; 1 = another live daemon has it + local dir tag old + dir="${TMPDIR:-/tmp}"; dir="${dir%/}/crq"; mkdir -p "$dir" 2>/dev/null || dir="/tmp" + tag="$(printf '%s' "$CRQ_REPO" | tr -c 'A-Za-z0-9' '_')" + CRQ_HOST_PIDFILE="$dir/autoreview-$tag.pid" + if [ -f "$CRQ_HOST_PIDFILE" ]; then + old="$(cat "$CRQ_HOST_PIDFILE" 2>/dev/null)" + if [ -n "$old" ] && [ "$old" != "$$" ] && kill -0 "$old" 2>/dev/null \ + && ps -p "$old" -o command= 2>/dev/null | grep -q 'crq'; then return 1; fi + fi + printf '%s\n' "$$" > "$CRQ_HOST_PIDFILE" 2>/dev/null || true + return 0 +} +_host_singleton_drop() { + [ -n "${CRQ_HOST_PIDFILE:-}" ] && [ "$(cat "$CRQ_HOST_PIDFILE" 2>/dev/null)" = "$$" ] && rm -f "$CRQ_HOST_PIDFILE" 2>/dev/null + return 0 +} + +_leader_ref_sha() { + local o; o="$(gh api "repos/$CRQ_REPO/git/refs/heads/$CRQ_LEADER_REF" --jq '.object.sha // empty' 2>/dev/null)" + case "$o" in *[!0-9a-f]*|"") return 0 ;; esac; printf '%s' "$o" +} +_leader_commit() { gh api -X POST "repos/$CRQ_REPO/git/commits" -f message="crq-leader $_LEADER_ID epoch=$(now)" -f tree="$EMPTY_TREE" --jq .sha 2>/dev/null; } +_leader_beat() { # refresh our heartbeat; 0 = still ours + advanced, 1 = lost/failed (caller re-contends) + local cur sha + # Re-confirm WE still hold the ref immediately before the force-update: a beat delayed past + # CRQ_LEADER_TTL must NOT clobber a host that legitimately stole the stale ref (split-brain). + cur="$(_leader_ref_sha)"; [ -n "$cur" ] || return 1 + case "$(_lock_commit_msg "$cur")" in *"$_LEADER_ID epoch="*) ;; *) return 1 ;; esac + sha="$(_leader_commit)"; [ -n "$sha" ] || return 1 + gh api -X PATCH "repos/$CRQ_REPO/git/refs/heads/$CRQ_LEADER_REF" -f sha="$sha" -F force=true >/dev/null 2>&1 # 0/1 propagated +} +_leader_try() { # 0 = we are (now) leader; 1 = a fresh foreign leader holds it -> stand by + local cur msg cdate cep age sha + cur="$(_leader_ref_sha)" + if [ -n "$cur" ]; then + msg="$(_lock_commit_msg "$cur")" + case "$msg" in *"$_LEADER_ID epoch="*) _leader_beat; return $? ;; esac # already ours -> heartbeat (fail -> re-contend) + cdate="$(gh api "repos/$CRQ_REPO/git/commits/$cur" --jq '.committer.date // empty' 2>/dev/null)" + [ -n "$cdate" ] || return 1 # can't read its timestamp (API blip) -> assume ALIVE, stand by (crq:234) + cep="$(iso2epoch "$cdate")" + [ -n "$cep" ] && [ -z "${cep//[0-9]/}" ] || return 1 # unparsable timestamp (gh dumped an error body) -> assume ALIVE (crq:236) + age=$(( $(now) - cep )) + [ "$age" -lt "$CRQ_LEADER_TTL" ] && return 1 # foreign + fresh -> stand by + # foreign + stale -> steal, but only if the ref STILL points at the same stale commit we just + # measured; if another standby already refreshed/replaced it, don't delete its fresh leader (crq:236). + [ "$(_leader_ref_sha)" = "$cur" ] || return 1 + gh api -X DELETE "repos/$CRQ_REPO/git/refs/heads/$CRQ_LEADER_REF" >/dev/null 2>&1 + fi + sha="$(_leader_commit)"; [ -n "$sha" ] || return 1 # claim via atomic create-if-not-exists + gh api -X POST "repos/$CRQ_REPO/git/refs" -f ref="refs/heads/$CRQ_LEADER_REF" -f sha="$sha" >/dev/null 2>&1 || return 1 + cur="$(_leader_ref_sha)"; [ "$cur" = "$sha" ] # confirm we won the create +} +_leader_drop() { + local cur; cur="$(_leader_ref_sha)"; [ -n "$cur" ] || return 0 + case "$(_lock_commit_msg "$cur")" in *"$_LEADER_ID epoch="*) gh api -X DELETE "repos/$CRQ_REPO/git/refs/heads/$CRQ_LEADER_REF" >/dev/null 2>&1 ;; esac + return 0 +} + # ---------------------------------------------------------------------------- # State (JSON inside the dashboard issue body, between markers). # ---------------------------------------------------------------------------- empty_state() { jq -cn --arg scope "$CRQ_SCOPE" \ - '{v:1,rev:0,next_seq:0,queue:[],in_flight:null,last_fired:null,warn:null, + '{v:1,rev:0,next_seq:0,queue:[],in_flight:null,last_fired:null,warn:null,fired:{},history:[], blocked:{scope:$scope,blocked_until:null,remaining:null,source:"init",checked_at:null,calib_asked_at:null}}' } @@ -158,39 +268,100 @@ state_read() { # echoes compact JSON from the dashboard issue } render_human() { - jq -r ' - def ts(x): if x==null then "β€”" else x end; - "## 🐰 crq dashboard Β· rev \(.rev) Β· updated \(ts(.wrote_at))", + TZ="${CRQ_TZ:-UTC}" jq -r ' + def ts(x): if x==null then "β€”" else (x | fromdateiso8601 | strflocaltime("%Y-%m-%d %H:%M %Z")) end; + def link(repo; pr): "[\(repo)#\(pr)](https://github.com/\(repo)/pull/\(pr))"; + def mins(x): ((x | fromdateiso8601) - now) / 60 | floor; + (if (.blocked.blocked_until != null) and ((.blocked.blocked_until | fromdateiso8601) > now) + then "πŸ”΄ Blocked β€” next review in ~\(mins(.blocked.blocked_until))m" + elif (.in_flight != null) then "🟑 Reviewing \(.in_flight.repo)#\(.in_flight.pr)" + elif ((.queue | length) > 0) then "🟑 \(.queue|length) queued" + else "🟒 Idle β€” all caught up" end) as $status | + "# 🐰 crq β€” CodeRabbit review queue", + "", + "### \($status)", "", - "**Scope:** \(.blocked.scope // "β€”") Β· **Remaining:** \(.blocked.remaining // "?") Β· **Source:** \(.blocked.source // "β€”")", - "**Blocked until:** \(ts(.blocked.blocked_until)) Β· **Last fired:** \(ts(.last_fired))", - "**In flight:** " + (if .in_flight==null then "β€”" else "\(.in_flight.repo)#\(.in_flight.pr) β€” fired \(.in_flight.fired_at) by \(.in_flight.by_host)" end), + "| | |", + "|---|---|", + "| **Scope** | `\(.blocked.scope // "β€”")` |", + "| **Reviews remaining** | \(if .blocked.remaining != null then (.blocked.remaining|tostring) elif (.blocked.blocked_until != null and ((.blocked.blocked_until|fromdateiso8601) > now)) then "0 β€” rate-limited" else "available now" end) _(via \(.blocked.source // "β€”"))_ |", + "| **Rate limit** | \(if (.blocked.blocked_until != null and ((.blocked.blocked_until|fromdateiso8601) > now)) then "β›” resets \(ts(.blocked.blocked_until)) (~\(mins(.blocked.blocked_until))m)" else "βœ… not currently limited" end) |", + "| **Last review fired** | \(ts(.last_fired)) |", + "| **In flight** | \(if .in_flight==null then "β€”" else "\(link(.in_flight.repo; .in_flight.pr)) Β· fired \(ts(.in_flight.fired_at)) Β· `\(.in_flight.by_host)`" end) |", (if (.warn // null) != null then "", "> ⚠️ \(.warn)" else empty end), "", - "### Queue (FIFO)", - (if (.queue|length)==0 then "_empty_" - else ("| seq | PR | repo | host | enqueued |", "|--:|--:|---|---|---|"), - (.queue | sort_by(.seq)[] | "| \(.seq) | #\(.pr) | \(.repo) | \(.host) | \(.enqueued_at) |") + "## ⏳ Queue β€” \(.queue|length) waiting", + "", + (if (.queue|length)==0 then "_Nothing queued._" + else ("| # | PR | enqueued | host |", "|--:|---|---|---|"), + (.queue | sort_by(.seq) | to_entries[] | + "| \(.key + 1) | \(link(.value.repo; .value.pr)) | \(ts(.value.enqueued_at)) | `\(.value.host)` |") + end), + "", + "## βœ… Recently reviewed β€” last \((.history // []) | length)", + "", + (if ((.history // []) | length)==0 then "_None yet._" + else ("| PR | commit | reviewed | host |", "|---|---|---|---|"), + ((.history // [])[] | + "| \(link(.repo; .pr)) | `\(.commit // "?")` | \(ts(.at)) | `\(.host)` |") end), "", - "Managed by crq β€” do not edit by hand; state lives in the hidden block above." + "---", + "πŸ€– Managed by [crq](https://github.com/kristofferR/coderabbit-queue) Β· rev \(.rev) Β· updated \(ts(.wrote_at)) Β· do not edit by hand (machine state is in the hidden block at the top)." ' <<<"$1" } render_title() { jq -r ' - (.queue|length) as $q | - (if .in_flight==null then "idle" else "inflight=\(.in_flight.repo|split("/")|last)#\(.in_flight.pr)" end) as $f | - "crq: q=\($q) Β· \($f) Β· remaining=\(.blocked.remaining // "?")" + (if (.blocked.blocked_until!=null) and ((.blocked.blocked_until|fromdateiso8601)>now) then "πŸ”΄ blocked" + elif .in_flight!=null then "🟑 reviewing #\(.in_flight.pr)" + elif (.queue|length)>0 then "🟑 \(.queue|length) queued" + else "🟒 idle" end) as $s | + "crq Β· \($s) Β· queue \(.queue|length)" ' <<<"$1" } -state_write() { # $1 = JSON; bumps rev, stamps wrote_at, re-renders issue - local json body title +# Publish the human dashboard to the gate repo's README.md (via the Contents API). +update_readme() { # $1 = json + local body b64 sha + body="$(render_human "$1")" + b64="$(printf '%s\n' "$body" | base64 | tr -d '\n')" + sha="$(gh api "repos/$CRQ_REPO/contents/README.md" --jq '.sha // empty' 2>/dev/null)" + case "$sha" in *[!0-9a-f]*|"") sha="" ;; esac # guard: gh dumps the 404 body to stdout + if [ -n "$sha" ]; then + gh api -X PUT "repos/$CRQ_REPO/contents/README.md" \ + -f message="crq: dashboard update" -f content="$b64" -f sha="$sha" >/dev/null 2>&1 + else + gh api -X PUT "repos/$CRQ_REPO/contents/README.md" \ + -f message="crq: dashboard" -f content="$b64" >/dev/null 2>&1 + fi +} + +state_write() { # $1 = JSON; bumps rev, updates the issue (machine state) + README dashboard + local json title body prev_sig new_sig sig + prev_sig="$(jq -r '.readme_sig // empty' <<<"$1")" + # signature of the user-visible content only β€” so the README is committed on material changes, + # not on every internal tick (rev/checked_at/countdown). + new_sig="$(jq -c '[.queue,.in_flight,.blocked.blocked_until,.blocked.remaining,.blocked.source,.warn,.history]' <<<"$1")" json="$(jq -c --arg now "$(now_iso)" '.rev=((.rev//0)+1) | .wrote_at=$now' <<<"$1")" - body="$(printf '%s\n%s\n%s\n\n%s\n' "$STATE_BEGIN" "$(jq -c . <<<"$json")" "$STATE_END" "$(render_human "$json")")" + # Publish the README dashboard FIRST when content changed; only adopt the new signature if + # that succeeds β€” otherwise keep the old one so the next material write retries the publish + # (don't let a failed README update leave the dashboard stale forever). + sig="$prev_sig" + if [ "$new_sig" != "$prev_sig" ]; then + if update_readme "$json"; then sig="$new_sig"; else log "WARN: README dashboard publish failed β€” will retry next change"; fi + fi + json="$(jq -c --arg sig "$sig" '.readme_sig=$sig' <<<"$json")" + # issue body holds the machine state; the dashboard lives in the repo README + body="$(printf '%s\n%s\n%s\n\n_Machine state for crq. Live dashboard: the [README](https://github.com/%s#readme) of this repo._\n' \ + "$STATE_BEGIN" "$(jq -c . <<<"$json")" "$STATE_END" "$CRQ_REPO")" title="$(render_title "$json")" - printf '%s' "$body" | gh issue edit "$CRQ_ISSUE" --repo "$CRQ_REPO" --title "$title" --body-file - >/dev/null + # Persist machine state to the issue. Propagate failure so callers don't proceed on stale state. + if ! printf '%s' "$body" | gh issue edit "$CRQ_ISSUE" --repo "$CRQ_REPO" --title "$title" --body-file - >/dev/null 2>&1; then + log "ERROR: failed to persist state to issue #$CRQ_ISSUE β€” not releasing as success" + return 1 + fi + return 0 } # ---------------------------------------------------------------------------- @@ -205,8 +376,8 @@ _calib_post() { # Latest bot rate-limit *reply* on the calibration PR with updated_at > $1 (ISO). # Filters to the reply marker so it ignores the PR-open summary/skip comment. _calib_latest_reply() { - gh api "repos/$CRQ_REPO/issues/$CRQ_CAL_PR/comments" --paginate \ - --jq "[.[]|select(.user.login==\"$CRQ_BOT\")|select(.body|contains(\"$CRQ_CAL_REPLY_MARKER\"))|select(.updated_at > \"$1\")]|last|{u:.updated_at,b:.body}" 2>/dev/null + gh api "repos/$CRQ_REPO/issues/$CRQ_CAL_PR/comments" --paginate --slurp 2>/dev/null \ + | jq -c "add | map(select(.user.login==\"$CRQ_BOT\" and (.body|contains(\"$CRQ_CAL_REPLY_MARKER\")) and .updated_at > \"$1\")) | last // empty | {u:.updated_at,b:.body}" 2>/dev/null } # Parse a CodeRabbit reply/body for (remaining, reset_epoch). Tolerant; refined @@ -226,14 +397,19 @@ parse_quota() { # override a fresh "available now" reply. The open-PR WARNING scan is only a # fallback used when no fresh calibration reply is available. Echoes updated JSON. calibrate_json() { - local json="$1" n checked src reply cutoff rtext rbase pair remaining reset at w wreset + local json="$1" n checked src reply cutoff rtext rbase pair remaining reset at w wreset bu bu_e n="$(now)" checked="$(jq -r '.blocked.checked_at // empty' <<<"$json")" src="$(jq -r '.blocked.source // empty' <<<"$json")" - # Trust a recent calibration reading. - if [ "$src" = "calibrate" ] && [ -n "$checked" ] \ - && [ $(( n - $(iso2epoch "$checked") )) -lt "$CRQ_CALIBRATE_TTL" ]; then - echo "$json"; return 0 + if [ -n "$checked" ] && [ $(( n - $(iso2epoch "$checked") )) -lt "$CRQ_CALIBRATE_TTL" ]; then + # Trust a recent calibration reading... + [ "$src" = "calibrate" ] && { echo "$json"; return 0; } + # ...and a recent WARNING-derived block (e.g. a fired review was just rate-limited) β€” don't + # let a stale "available now" calibration override it. + if [ "$src" = "warning" ]; then + bu="$(jq -r '.blocked.blocked_until // empty' <<<"$json")"; bu_e=0; [ -n "$bu" ] && bu_e="$(iso2epoch "$bu")" + [ "$bu_e" -gt "$n" ] && { echo "$json"; return 0; } + fi fi if [ -n "${CRQ_CAL_PR:-}" ]; then # Reuse a fresh prior reply (avoids spamming the calibration PR). @@ -286,14 +462,14 @@ _warning_latest() { for owner in ${CRQ_SCOPE//,/ }; do while read -r repo num; do [ -n "$repo" ] || continue - line="$(gh api "repos/$repo/issues/$num/comments" --paginate \ - --jq "[.[]|select(.user.login==\"$CRQ_BOT\")|select(.body|contains(\"$CRQ_RL_MARKER\"))]|last|{u:.updated_at,b:.body}" 2>/dev/null)" + line="$(gh api "repos/$repo/issues/$num/comments" --paginate --slurp 2>/dev/null \ + | jq -c "add | map(select(.user.login==\"$CRQ_BOT\" and (.body|contains(\"$CRQ_RL_MARKER\")))) | last // empty | {u:.updated_at,b:.body}" 2>/dev/null)" [ -n "$line" ] && [ "$line" != "null" ] || continue u="$(jq -r .u <<<"$line")"; b="$(jq -r .b <<<"$line")" ue="$(iso2epoch "$u")"; reset="$(parse_available_in "$b" "$ue")" || continue [ -n "$reset" ] || continue if [ "$ue" -gt "$best_u" ]; then best_u="$ue"; best_reset="$reset"; found=1; fi - done < <(gh search prs --owner "$owner" --state open --json number,repository \ + done < <(gh search prs --owner "$owner" --state open --archived=false --limit "$CRQ_WARNING_SCAN_LIMIT" --json number,repository \ --jq '.[]|"\(.repository.nameWithOwner) \(.number)"' 2>/dev/null) done [ "$found" = "1" ] && echo "${best_u}|${best_reset}" @@ -303,20 +479,49 @@ _warning_latest() { # In-flight detection: can pump fire the NEXT review? # Returns 0 (clear) / 1 (still in flight). Echoes a reason on stderr. # ---------------------------------------------------------------------------- -inflight_status() { # $1 = json - local inf repo pr fired_at fired_id fe n rx newer +inflight_status() { # $1 = json; echoes timeout | warning: | done ; 0=cleared 1=still in flight + local inf repo pr fired_at fired_id fe n rx newer wl wu wb wreset revd inf="$(jq -c '.in_flight // empty' <<<"$1")" [ -n "$inf" ] || return 0 repo="$(jq -r .repo <<<"$inf")"; pr="$(jq -r .pr <<<"$inf")" fired_at="$(jq -r .fired_at <<<"$inf")"; fired_id="$(jq -r .fired_comment_id <<<"$inf")" fe="$(iso2epoch "$fired_at")"; n="$(now)" - if [ $(( n - fe )) -gt "$CRQ_INFLIGHT_TIMEOUT" ]; then echo "timeout" ; return 0; fi + # 1) Rate-limited? A fresh WARNING on this PR β€” surface its reset so the gate backs off NOW + # instead of trusting a stale "available now" calibration. (The review did NOT run.) + # (--slurp then a standalone jq with `add`: combine ALL pages before filtering β€” plain + # --paginate --jq runs the filter once PER PAGE, so on long PRs counts/last arrive multi-line + # and the numeric/object tests misfire. gh forbids --slurp together with --jq, hence the pipe.) + wl="$(gh api "repos/$repo/issues/$pr/comments" --paginate --slurp 2>/dev/null \ + | jq -c "add | map(select(.user.login==\"$CRQ_BOT\" and (.body|contains(\"$CRQ_RL_MARKER\")) and .updated_at > \"$fired_at\")) | last // empty | {u:.updated_at,b:.body}" 2>/dev/null)" + if [ -n "$wl" ] && [ "$wl" != "null" ]; then + wu="$(jq -r '.u // empty' <<<"$wl" 2>/dev/null)"; wb="$(jq -r '.b // empty' <<<"$wl" 2>/dev/null)" + # Only a fully-parsed warning counts β€” a transient gh error body must NOT trip the rate-limit path. + if [ -n "$wu" ] && [ -n "$wb" ]; then + wreset="$(parse_available_in "$wb" "$(iso2epoch "$wu")")" + # A rate-limit warning means the review did NOT run, so ALWAYS requeue β€” never let it fall + # through to a "done" signal. Future reset -> back off until then; already-expired or + # unparseable -> requeue with no backoff (retry now; a still-limited retry just re-warns with + # a fresh future reset, so this can't spin). Bounded either way. + if [ -n "$wreset" ] && [ "$wreset" -gt "$n" ]; then echo "warning:${wreset}"; else echo "warning:0"; fi + return 0 + fi + fi + # 2) Completed? A formal CRQ_BOT review submitted after we fired (autoreview's source of truth) β€” + # catches completions that leave no reaction/comment, so we don't wrongly time out + requeue. + revd="$(gh api "repos/$repo/pulls/$pr/reviews" --paginate --slurp 2>/dev/null \ + | jq "add | map(select(.user.login==\"$CRQ_BOT\" and .submitted_at > \"$fired_at\")) | length" 2>/dev/null)" + [ "${revd:-0}" -gt 0 ] && { echo "done"; return 0; } + # 3) Completed via a reaction on our trigger comment, or any newer bot comment. rx="$(gh api "repos/$repo/issues/comments/$fired_id/reactions" \ --jq "[.[]|select(.user.login==\"$CRQ_BOT\")]|length" 2>/dev/null)" - [ "${rx:-0}" -gt 0 ] && { echo "reaction"; return 0; } - newer="$(gh api "repos/$repo/issues/$pr/comments" --paginate \ - --jq "[.[]|select(.user.login==\"$CRQ_BOT\")|select((.id|tostring)!=\"$fired_id\")|select(.updated_at > \"$fired_at\")]|length" 2>/dev/null)" - [ "${newer:-0}" -gt 0 ] && { echo "bot-comment"; return 0; } + [ "${rx:-0}" -gt 0 ] && { echo "done"; return 0; } + # Exclude rate-limit WARNING comments here β€” a warning is NOT a completion signal (the warning + # branch above handles it). Without this, an expired-reset warning would be miscounted as "done". + newer="$(gh api "repos/$repo/issues/$pr/comments" --paginate --slurp 2>/dev/null \ + | jq "add | map(select(.user.login==\"$CRQ_BOT\" and (.id|tostring)!=\"$fired_id\" and .updated_at > \"$fired_at\" and (.body|contains(\"$CRQ_RL_MARKER\")|not))) | length" 2>/dev/null)" + [ "${newer:-0}" -gt 0 ] && { echo "done"; return 0; } + # 4) Backstop: only now, if nothing landed in time, treat as a timed-out review. + [ $(( n - fe )) -gt "$CRQ_INFLIGHT_TIMEOUT" ] && { echo "timeout"; return 0; } return 1 } @@ -324,17 +529,39 @@ inflight_status() { # $1 = json # gc: drop closed/merged PRs from queue; clear timed-out in-flight. # ---------------------------------------------------------------------------- gc_json() { # $1 = json - local json="$1" entry repo pr state inf reason gat n + local json="$1" entry repo pr state inf reason gat n wreset n="$(now)" # in-flight: clear as soon as a completion signal appears (responsive β€” every call). inf="$(jq -c '.in_flight // empty' <<<"$json")" if [ -n "$inf" ]; then reason="$(inflight_status "$json")" && { - if [ "$reason" = "timeout" ]; then - json="$(jq -c '.warn="in_flight \(.in_flight.repo)#\(.in_flight.pr) timed out without a completion signal β€” check detector" | .in_flight=null' <<<"$json")" - else - json="$(jq -c '.in_flight=null | .warn=null' <<<"$json")" - fi + case "$reason" in + warning:*) + # The fired review was rate-limited β€” it never actually ran. Requeue the PR (so it's + # retried once unblocked) and clear its fired-marker, else dedup/autoreview would skip + # this exact commit forever. Set blocked_until from the WARNING so we back off now. + wreset="${reason#warning:}" + json="$(jq -c --arg ca "$(now_iso)" \ + --arg bu "$([ "${wreset:-0}" -gt "$n" ] && epoch2iso "$wreset")" ' + (.in_flight.repo) as $r | (.in_flight.pr) as $p | ($r + "#" + ($p|tostring)) as $k + | ((.in_flight.seq) // ((.next_seq//0)+1)) as $sq + | .queue += [{seq:$sq, owner:($r|split("/")[0]), repo:$r, pr:$p, host:(.in_flight.by_host // "?"), enqueued_at:$ca}] + | del(.fired[$k]) | .in_flight=null | .warn=null + | (if $bu=="" then . else (.blocked.blocked_until=$bu | .blocked.source="warning" + | .blocked.remaining=0 | .blocked.checked_at=$ca) end)' <<<"$json")" + log "rate-limited on fired PR β€” requeued for retry" ;; + timeout) + # No completion signal in time. The review may never have run, so requeue it and + # clear its fired-marker (else dedup/autoreview skip the commit forever). + json="$(jq -c --arg ca "$(now_iso)" ' + (.in_flight.repo) as $r | (.in_flight.pr) as $p | ($r + "#" + ($p|tostring)) as $k + | .warn="in_flight \($r)#\($p) timed out without a completion signal β€” requeued (check detector)" + | ((.in_flight.seq) // ((.next_seq//0)+1)) as $sq + | .queue += [{seq:$sq, owner:($r|split("/")[0]), repo:$r, pr:$p, host:(.in_flight.by_host // "?"), enqueued_at:$ca}] + | del(.fired[$k]) | .in_flight=null' <<<"$json")" ;; + *) + json="$(jq -c '.in_flight=null | .warn=null' <<<"$json")" ;; + esac } fi # queue prune (drop non-open PRs): gated by CRQ_GC_TTL β€” the per-PR scan is costly. @@ -343,14 +570,27 @@ gc_json() { # $1 = json while read -r entry; do [ -n "$entry" ] || continue repo="$(jq -r .repo <<<"$entry")"; pr="$(jq -r .pr <<<"$entry")" - state="$(gh api "repos/$repo/pulls/$pr" --jq '.state // "unknown"' 2>/dev/null)" - if [ "$state" != "open" ] && [ "$state" != "unknown" ]; then - log "gc: dropping $repo#$pr ($state)" - json="$(jq -c --arg r "$repo" --argjson p "$pr" '.queue|=map(select(.repo!=$r or .pr!=$p))' <<<"$json")" + state="$(gh api "repos/$repo/pulls/$pr" --jq '.state // empty' 2>/dev/null)" + # Only drop on an explicit "closed". A transient API error returns a JSON body on + # stdout (gh skips --jq on errors), so anything that isn't a known state = keep. + if [ "$state" = "closed" ]; then + log "gc: dropping $repo#$pr (closed)" + json="$(jq -c --arg r "$repo" --argjson p "$pr" --arg k "$repo#$pr" \ + '.queue|=map(select(.repo!=$r or .pr!=$p)) | del(.fired[$k])' <<<"$json")" fi done < <(jq -c '.queue[]?' <<<"$json") json="$(jq -c --arg t "$(now_iso)" '.gc_at=$t' <<<"$json")" fi + # Normalize fired-marker keys (and history repo names) to lowercase. Case folding here means a + # repo-name casing change or mixed-case caller can't orphan a marker and trigger a re-review; + # merges any duplicate-cased keys (from_entries keeps the last value, same head in practice). + # Fold all repo names to lowercase (case-insensitive matching) and cap the fired-marker dict so + # markers for long-closed PRs can't grow the dashboard JSON without bound (keep the most-recent). + json="$(jq -c --argjson max "$CRQ_FIRED_MAX" ' + .fired = ((.fired // {}) | to_entries | map(.key |= ascii_downcase) | from_entries | to_entries | .[-$max:] | from_entries) + | .history = ((.history // []) | map(if .repo then .repo |= ascii_downcase else . end)) + | .queue = ((.queue // []) | map(if .repo then .repo |= ascii_downcase else . end)) + | (if (.in_flight|type) == "object" and .in_flight.repo then .in_flight.repo |= ascii_downcase else . end)' <<<"$json")" echo "$json" } @@ -360,6 +600,7 @@ gc_json() { # $1 = json cmd_enqueue() { local repo="$1" pr="$2" nonce json present need CRQ_REPO; need CRQ_ISSUE + repo="$(printf '%s' "$repo" | tr 'A-Z' 'a-z')" # store normalized so all state lookups match (GitHub folds case) nonce="$(acquire)" || die "could not acquire lock" # shellcheck disable=SC2064 trap "release '$nonce'" EXIT @@ -371,7 +612,7 @@ cmd_enqueue() { json="$(jq -c --arg r "$repo" --argjson p "$pr" --arg h "$CRQ_HOST" --arg t "$(now_iso)" --arg o "${repo%%/*}" \ '.next_seq=((.next_seq//0)+1) | .queue += [{seq:.next_seq, owner:$o, repo:$r, pr:$p, host:$h, enqueued_at:$t}]' <<<"$json")" - state_write "$json" + state_write "$json" || { release "$nonce"; trap - EXIT; return 1; } log "enqueued $repo#$pr (seq $(jq -r '.next_seq' <<<"$json"))" else log "$repo#$pr already queued/in-flight" @@ -380,7 +621,7 @@ cmd_enqueue() { } cmd_pump() { - local nonce json n bu bu_epoch lf lf_epoch qn cfresh infp head repo pr seq cid + local nonce json n bu bu_epoch lf lf_epoch qn cfresh infp head repo pr seq cid hsha key lastsha crrev_p crrev_raw need CRQ_REPO; need CRQ_ISSUE # Lockless fast path: avoid taking the global lock on idle ticks. Re-checked @@ -395,8 +636,15 @@ cmd_pump() { fi qn="$(jq '.queue|length' <<<"$json")" infp="$(jq -r 'if .in_flight==null then 0 else 1 end' <<<"$json")" - if [ "$cfresh" = "1" ] && { [ "$n" -lt "$bu_epoch" ] || { [ "$qn" -eq 0 ] && [ "$infp" = "0" ]; }; }; then - return 0 # blocked (and known), or idle with nothing to do β€” no lock needed + # Idle: nothing queued and nothing in flight -> nothing to fire, so DON'T calibrate or lock. + # (Calibration posts `@coderabbitai rate limit`; doing it on idle ticks is what floods the + # calibration PR. We only need a quota reading when there's actually a review waiting to fire.) + if [ "$qn" -eq 0 ] && [ "$infp" = "0" ]; then + return 0 + fi + # Blocked and the reading is still fresh -> nothing to do until it expires; skip the lock. + if [ "$cfresh" = "1" ] && [ "$n" -lt "$bu_epoch" ]; then + return 0 fi nonce="$(acquire)" || die "could not acquire lock" @@ -427,41 +675,101 @@ cmd_pump() { # window open, nothing in flight, interval ok, queue non-empty -> fire FIFO head head="$(jq -c '.queue|sort_by(.seq)|.[0]' <<<"$json")" repo="$(jq -r .repo <<<"$head")"; pr="$(jq -r .pr <<<"$head")"; seq="$(jq -r .seq <<<"$head")" - if [ "${CRQ_DRY_RUN:-0}" = "1" ]; then - log "DRY-RUN: would fire $CRQ_REVIEW_CMD on $repo#$pr (seq $seq)" - state_write "$json"; release "$nonce"; trap - EXIT; return 0 - fi - cid="$(gh api "repos/$repo/issues/$pr/comments" -f body="$CRQ_REVIEW_CMD" --jq .id 2>/dev/null)" - if [ -n "$cid" ]; then - json="$(jq -c --argjson s "$seq" --arg r "$repo" --argjson p "$pr" \ - --arg fa "$(now_iso)" --argjson cid "$cid" --arg h "$CRQ_HOST" \ - '.queue|=map(select(.seq!=$s)) - | .in_flight={seq:$s,repo:$r,pr:$p,fired_at:$fa,fired_comment_id:$cid,by_host:$h} - | .last_fired=$fa | .warn=null' <<<"$json")" - log "FIRED $repo#$pr (seq $seq, comment $cid)" + hsha="$(gh api "repos/$repo/pulls/$pr" --jq '.head.sha // empty' 2>/dev/null | cut -c1-9)" + key="$repo#$pr" + lastsha="$(jq -r --arg k "$key" '.fired[$k] // empty' <<<"$json")" + # A failed head lookup makes gh dump an error body to stdout (--jq skipped), so cut yields a + # non-empty NON-hex string β€” validate it's a real short SHA, not just non-empty (crq:676/679). + if [ -z "$hsha" ] || [ -n "${hsha//[0-9a-f]/}" ]; then + # Couldn't read a valid head SHA (transient/secondary-limit) β€” DON'T fire with a bogus SHA; + # leave the PR queued so the next pump retries once the API recovers. + log "skip: could not read a valid $key head SHA (transient) β€” leaving it queued" + elif [ "$hsha" = "$lastsha" ]; then + # DEDUP: we already requested a review for this exact commit (e.g. autoreview and an + # agent both queued it). Drop it without posting a second @coderabbitai review. + json="$(jq -c --argjson s "$seq" '.queue|=map(select(.seq!=$s))' <<<"$json")" + log "dedup: review already requested for $key@$hsha β€” not re-posting" + elif ! crrev_raw="$(gh api "repos/$repo/pulls/$pr/reviews" --paginate --slurp 2>/dev/null)"; then + # Reviews lookup FAILED β€” don't fall through to a live post (that would reopen the duplicate + # window on a transient error); leave queued so the next pump retries (crq:560/699). + log "skip: could not read reviews for $key (transient) β€” leaving it queued" + elif ! crrev_p="$(printf '%s' "$crrev_raw" | jq -r --arg bot "$CRQ_BOT" --arg h "$hsha" 'add|map(select(.user.login==$bot and ((.commit_id//"")[0:9])==$h))|length' 2>/dev/null)"; then + log "skip: could not parse reviews for $key β€” leaving it queued" + elif [ "${crrev_p:-0}" -gt 0 ]; then + # The bot has already reviewed THIS exact commit even though we hold no fired-marker β€” e.g. a + # slow review landed right after a timeout-requeue cleared the marker. Record it and drop + # without posting a duplicate @coderabbitai review (crq:560). + json="$(jq -c --argjson s "$seq" --arg k "$key" --arg hs "$hsha" '.queue|=map(select(.seq!=$s)) | .fired[$k]=$hs' <<<"$json")" + log "dedup: $CRQ_BOT already reviewed $key@$hsha β€” recording, not re-posting" + elif [ "${CRQ_DRY_RUN:-0}" = "1" ]; then + log "DRY-RUN: would fire $CRQ_REVIEW_CMD on $repo#$pr@${hsha:-?} (seq $seq)" + state_write "$json" || { release "$nonce"; trap - EXIT; return 1; } # propagate write failure like the live paths + release "$nonce"; trap - EXIT; return 0 else - log "failed to post review command on $repo#$pr" + # Require a successful POST AND a numeric id β€” on errors (perms, locked PR, secondary + # rate limit) gh exits non-zero and may print an error body to stdout. + if cid="$(gh api "repos/$repo/issues/$pr/comments" -f body="$CRQ_REVIEW_CMD" --jq '.id // empty' 2>/dev/null)" \ + && [ -n "$cid" ] && [ -z "${cid//[0-9]/}" ]; then + json="$(jq -c --argjson s "$seq" --arg r "$repo" --argjson p "$pr" \ + --arg fa "$(now_iso)" --argjson cid "$cid" --arg h "$CRQ_HOST" \ + --arg k "$key" --arg hs "$hsha" \ + '.queue|=map(select(.seq!=$s)) + | .in_flight={seq:$s,repo:$r,pr:$p,fired_at:$fa,fired_comment_id:$cid,by_host:$h} + | .last_fired=$fa | .warn=null + | (if $hs=="" then . else (.fired=(.fired|del(.[$k]))) | .fired[$k]=$hs end) + | .history = ([{repo:$r,pr:$p,commit:(if $hs=="" then null else $hs end),at:$fa,host:$h}] + (.history // []))[0:20]' <<<"$json")" + log "FIRED $repo#$pr@${hsha:-?} (seq $seq, comment $cid)" + else + log "failed to post review command on $repo#$pr (no valid comment id) β€” leaving it queued" + fi fi fi - state_write "$json" + state_write "$json" || { release "$nonce"; trap - EXIT; return 1; } release "$nonce"; trap - EXIT } cmd_wait() { - local repo="$1" pr="$2" start json queued + local repo="$1" pr="$2" start json queued inflight enqueued=0 whead wfired need CRQ_REPO; need CRQ_ISSUE - cmd_enqueue "$repo" "$pr" + repo="$(printf '%s' "$repo" | tr 'A-Z' 'a-z')" # normalize so state lookups match (GitHub folds case) start="$(now)" while true; do - cmd_pump >/dev/null 2>&1 || true + # Timeout FIRST, so even a persistently-failing enqueue (which loops via `continue` below) + # still honours CRQ_WAIT_TIMEOUT instead of hanging the caller forever. + if [ "$CRQ_WAIT_TIMEOUT" -gt 0 ] && [ $(( $(now) - start )) -gt "$CRQ_WAIT_TIMEOUT" ]; then + log "wait timeout for $repo#$pr"; return 2 + fi + # Pump FIRST: its gc clears a COMPLETED in-flight from a previous round, so a stale same-PR + # in_flight isn't mistaken for this waiter's freshly-fired review (it also fires the FIFO head). + # Then check completion BEFORE (re)enqueuing, so another runner firing our PR doesn't get a + # duplicate queue entry recreated here; only treat "absent" as fired once we actually enqueued. + ( cmd_pump ) >/dev/null 2>&1 || true json="$(state_read)" || { sleep "$CRQ_POLL"; continue; } + inflight="$(jq -r --arg r "$repo" --argjson p "$pr" '(.in_flight // {}) | select(.repo==$r and .pr==$p) | 1' <<<"$json" 2>/dev/null)" queued="$(jq --arg r "$repo" --argjson p "$pr" '[.queue[]?|select(.repo==$r and .pr==$p)]|length' <<<"$json")" - if [ "${queued:-1}" -eq 0 ]; then + if [ "${inflight:-}" = "1" ]; then log "fired $repo#$pr"; return 0; fi # genuinely in flight now (gc didn't clear it) + if [ "${queued:-0}" -gt 0 ]; then + enqueued=1 # in the queue, awaiting our turn + elif [ "$enqueued" = "1" ]; then + # Left the queue: either we fired it (in_flight, caught above) or pump DEDUPED it because this + # exact commit was already reviewed. Report accurately instead of always claiming "fired". + whead="$(gh api "repos/$repo/pulls/$pr" --jq '.head.sha // empty' 2>/dev/null | cut -c1-9)" + wfired="$(jq -r --arg k "$repo#$pr" '.fired[$k] // empty' <<<"$json" 2>/dev/null)" + if [ -n "$whead" ] && [ "$whead" = "$wfired" ]; then + log "already reviewed $repo#$pr@$whead (deduped)"; return 3 # distinct: no NEW review fired + fi log "fired $repo#$pr"; return 0 - fi - if [ "$CRQ_WAIT_TIMEOUT" -gt 0 ] && [ $(( $(now) - start )) -gt "$CRQ_WAIT_TIMEOUT" ]; then - log "wait timeout for $repo#$pr"; return 2 + else + ( cmd_enqueue "$repo" "$pr" ) || { sleep "$CRQ_POLL"; continue; } # first enqueue; retry if the write failed + enqueued=1 + ( cmd_pump ) >/dev/null 2>&1 || true # fire NOW β€” don't wait a full poll for a fresh enqueue + # If that pump fired OUR review, report it immediately β€” before the next iteration's pump runs + # gc, which could clear a fast completion and make our just-fired review look deduped (crq:740). + json="$(state_read)" || { sleep "$CRQ_POLL"; continue; } + [ "$(jq -r --arg r "$repo" --argjson p "$pr" '(.in_flight // {}) | select(.repo==$r and .pr==$p) | 1' <<<"$json" 2>/dev/null)" = "1" ] \ + && { log "fired $repo#$pr"; return 0; } + continue fi sleep $(( CRQ_POLL + (RANDOM % 5) )) done @@ -475,14 +783,19 @@ cmd_status() { cmd_cancel() { local repo="$1" pr="$2" nonce json + repo="$(printf '%s' "$repo" | tr 'A-Z' 'a-z')" # match normalized state (GitHub folds case) nonce="$(acquire)" || die "could not acquire lock" # shellcheck disable=SC2064 trap "release '$nonce'" EXIT json="$(state_read)" || { release "$nonce"; trap - EXIT; die "cannot read state"; } + # Case-insensitive match β€” a dashboard upgraded from an older version may still hold mixed-case + # repo names/keys that gc hasn't normalized yet, and a manual cancel must still clear them (crq:761). json="$(jq -c --arg r "$repo" --argjson p "$pr" \ - '.queue|=map(select(.repo!=$r or .pr!=$p)) - | (if (.in_flight!=null and .in_flight.repo==$r and .in_flight.pr==$p) then .in_flight=null else . end)' <<<"$json")" - state_write "$json" + '($r + "#" + ($p|tostring)) as $k + | .queue |= map(select((.repo|ascii_downcase)!=$r or .pr!=$p)) + | (if (.in_flight!=null and (.in_flight.repo|ascii_downcase)==$r and .in_flight.pr==$p) then .in_flight=null else . end) + | .fired = ((.fired // {}) | with_entries(select((.key|ascii_downcase)!=$k)))' <<<"$json")" + state_write "$json" || { release "$nonce"; trap - EXIT; return 1; } release "$nonce"; trap - EXIT log "cancelled $repo#$pr" } @@ -493,7 +806,7 @@ cmd_gc() { # shellcheck disable=SC2064 trap "release '$nonce'" EXIT json="$(state_read)" || { release "$nonce"; trap - EXIT; die "cannot read state"; } - json="$(gc_json "$json")"; state_write "$json" + json="$(gc_json "$json")"; state_write "$json" || { release "$nonce"; trap - EXIT; return 1; } release "$nonce"; trap - EXIT } @@ -504,19 +817,18 @@ cmd_refresh() { trap "release '$nonce'" EXIT json="$(state_read)" || { release "$nonce"; trap - EXIT; die "cannot read state"; } json="$(jq -c '.blocked.checked_at=null' <<<"$json")" # force re-calibrate - json="$(calibrate_json "$json")"; state_write "$json" + json="$(calibrate_json "$json")"; state_write "$json" || { release "$nonce"; trap - EXIT; return 1; } release "$nonce"; trap - EXIT cmd_status } cmd_unlock() { - local cur msg age cdate + local cur msg hepoch age cur="$(_lock_ref_sha)" if [ -z "$cur" ]; then log "no lock held"; return 0; fi - msg="$(_lock_commit_msg "$cur")" - cdate="$(gh api "repos/$CRQ_REPO/git/commits/$cur" --jq .committer.date 2>/dev/null)" - age=$(( $(now) - $(iso2epoch "$cdate") )) - log "current holder: $msg (age ${age}s)" + msg="$(_lock_commit_msg "$cur")"; hepoch="$(_lock_epoch "$msg")" + if [ -n "$hepoch" ]; then age="$(( $(now) - hepoch ))s"; else age="?"; fi + log "current holder: ${msg:-} (age $age)" if [ "${1:-}" = "--force" ]; then gh api -X DELETE "repos/$CRQ_REPO/git/refs/heads/$CRQ_LOCK_REF" >/dev/null 2>&1 && log "lock deleted" else @@ -524,9 +836,28 @@ cmd_unlock() { fi } +# Open a URL in the user's default browser (best-effort, cross-platform). CRQ_NO_OPEN disables it. +_open_url() { + local url="$1" + [ -n "${CRQ_NO_OPEN:-}" ] && return 1 + if command -v open >/dev/null 2>&1; then open "$url" >/dev/null 2>&1 # macOS + elif command -v wslview >/dev/null 2>&1; then wslview "$url" >/dev/null 2>&1 # WSL (wslu) + elif command -v xdg-open >/dev/null 2>&1; then ( xdg-open "$url" >/dev/null 2>&1 & ) # Linux + elif command -v powershell.exe >/dev/null 2>&1; then powershell.exe -NoProfile -Command "Start-Process '$url'" >/dev/null 2>&1 # WSL/Windows + else return 1; fi +} + +# Ignore ALL notifications from the gate repo. It's a machine-only state repo the user never +# reads, and crq posts `@coderabbitai rate limit` on the calibration PR AS the user β€” which +# re-subscribes them, so a per-PR unsubscribe does NOT hold and CodeRabbit's replies keep +# emailing them. Repo-level "ignore" is the durable fix (needs only the 'repo' scope). +_mute_repo() { + gh api --method PUT "repos/$CRQ_REPO/subscription" -F ignored=true --jq '.ignored' 2>/dev/null | grep -q true +} + cmd_init() { need CRQ_REPO - local owner branch base_sha pr_url issue_url + local owner branch base_sha pr_url issue_url repo_url owner="${CRQ_REPO%%/*}" if ! gh repo view "$CRQ_REPO" >/dev/null 2>&1; then log "creating private repo $CRQ_REPO" @@ -557,6 +888,11 @@ cmd_init() { CRQ_CAL_PR="$(printf '%s' "$pr_url" | grep -oE '[0-9]+$')" log "calibration PR: ${pr_url:-#$CRQ_CAL_PR}" fi + # Ignore ALL notifications from this machine-only gate repo, so the calibration PR's repeated + # CodeRabbit replies never email the user. (Per-PR unsubscribe doesn't hold β€” crq's own + # rate-limit comments re-subscribe the author; repo-level Ignore is the durable fix.) + if _mute_repo; then log "muted all notifications from gate repo $CRQ_REPO (Watch β†’ Ignore)" + else log "note: couldn't ignore $CRQ_REPO notifications β€” set Watch β†’ Ignore on the repo by hand to stop emails"; fi # dashboard issue if [ -z "${CRQ_ISSUE:-}" ] || ! gh issue view "$CRQ_ISSUE" --repo "$CRQ_REPO" >/dev/null 2>&1; then local json body @@ -566,12 +902,134 @@ cmd_init() { --title "crq: q=0 Β· idle Β· remaining=?" --body-file - 2>/dev/null)" CRQ_ISSUE="$(printf '%s' "$issue_url" | grep -oE '[0-9]+$')" log "dashboard issue: ${issue_url:-#$CRQ_ISSUE}" + state_write "$json" # normalize the issue + publish the initial README dashboard (first time only) fi printf '\n# Add these to %s (or your shell profile):\n' "${CRQ_CONFIG:-$HOME/.config/crq/env}" printf 'export CRQ_REPO=%q\n' "$CRQ_REPO" printf 'export CRQ_ISSUE=%q\n' "$CRQ_ISSUE" printf 'export CRQ_CAL_PR=%q\n' "$CRQ_CAL_PR" printf 'export CRQ_SCOPE=%q\n' "$CRQ_SCOPE" + # open the gate repo (its README is the live dashboard) in the default browser + repo_url="https://github.com/$CRQ_REPO" + if _open_url "$repo_url"; then log "opened your gate dashboard in the browser: $repo_url" + else log "your gate dashboard (the repo README): $repo_url"; fi +} + +# Emulate CodeRabbit's native auto-review (+ incremental review), but rate-coordinated: +# enqueue open PRs in scope that need a review, then let the FIFO pump fire them when the +# account is unblocked. Keep CodeRabbit's own auto-review OFF so this is the only driver. +# default : auto-review + incremental β€” review new PRs AND re-review on every push +# (enqueue whenever HEAD isn't the last CRQ_BOT-reviewed commit) +# --no-incremental : auto-review only β€” review each PR once (when it has no review yet), +# never re-review on later pushes +# --once : single pass (e.g. from cron / a monitor); otherwise loops forever +cmd_autoreview() { + local once=0 incremental=1 a + for a in "$@"; do case "$a" in + --once) once=1 ;; --no-incremental) incremental=0 ;; + *) die "usage: crq autoreview [--once] [--no-incremental]" ;; + esac; done + need CRQ_REPO; need CRQ_ISSUE; need CRQ_SCOPE # empty scope -> the owner loop would silently no-op + # Singleton β€” only one autoreview daemon per host, and one leader per account (CRQ_REPO). + if ! _host_singleton; then log "another crq autoreview is already running on $CRQ_HOST β€” exiting"; return 0; fi + # Clean up on any exit; on SIGINT/SIGTERM actually exit (bash would otherwise resume the loop, + # re-acquire leadership, and become an untracked daemon that's hard to stop). + trap '_host_singleton_drop; _leader_drop' EXIT + trap 'log "crq autoreview received stop signal β€” exiting"; exit 0' INT TERM + while ! _leader_try; do + [ "$once" = "1" ] && { log "another host leads crq autoreview for $CRQ_REPO β€” nothing to do (--once)"; return 0; } + log "standing by β€” another host leads crq autoreview for $CRQ_REPO" + sleep "$CRQ_AUTOREVIEW_POLL" + done + log "crq autoreview leader for $CRQ_REPO ($_LEADER_ID)" + local repo pr head crrev crrev_raw reviewed reviewed_raw json present key firedsha rlc scanned idx deferred scan_cursor=0 searched + local search_targets search_flag target + while true; do + # Re-assert leadership each pass (also our heartbeat). If a stale period let another host + # steal it, step down and stand by until we can reclaim β€” never two leaders scanning at once. + if ! _leader_try; then + log "lost crq autoreview leadership β€” standing by" + while ! _leader_try; do [ "$once" = "1" ] && return 0; sleep "$CRQ_AUTOREVIEW_POLL"; done + log "reclaimed crq autoreview leadership" + fi + json="$(state_read 2>/dev/null)" || json="" + scanned=0; idx=0; deferred=0 + # With an allowlist, search each repo directly (--repo) so an owner with >1000 open PRs can't let + # the owner-wide 1000-result cap hide allowlisted PRs; otherwise scan per owner in scope (crq:978). + if [ -n "$CRQ_REPOS" ]; then search_targets="${CRQ_REPOS//,/ }"; search_flag="--repo" + else search_targets="${CRQ_SCOPE//,/ }"; search_flag="--owner"; fi + for target in $search_targets; do + searched=0 + while read -r repo pr; do + [ -n "$repo" ] || continue + searched=$((searched+1)) + rlc="$(printf '%s' "$repo" | tr 'A-Z' 'a-z')" # case-insensitive matching (GitHub folds case) + [ "$rlc" = "$CRQ_REPO_LC" ] && continue # never auto-review the gate repo (calibration PR) + # repo allowlist / denylist (comma-separated owner/name; allowlist empty = all in scope) + if [ -n "$CRQ_REPOS_LC" ]; then case ",$CRQ_REPOS_LC," in *",$rlc,"*) ;; *) continue ;; esac; fi + if [ -n "$CRQ_EXCLUDE_LC" ]; then case ",$CRQ_EXCLUDE_LC," in *",$rlc,"*) continue ;; esac; fi + key="$rlc#$pr" # normalized key β€” matches the lowercased repo cmd_enqueue/pump store + # skip PRs already queued / in-flight (lockless pre-check avoids lock churn) + if [ -n "$json" ]; then + present="$(jq --arg r "$rlc" --argjson p "$pr" \ + '[ (.queue[]?|select(.repo==$r and .pr==$p)), + (.in_flight|select(.!=null and .repo==$r and .pr==$p)) ]|length' <<<"$json" 2>/dev/null)" + [ "${present:-0}" -gt 0 ] && continue + fi + # Bound REST GETs per pass (the head + reviews lookups below cost ~2 points each) so a huge + # scope can't exceed GitHub's ~900-pt/min secondary limit. ROTATE the budget across passes + # instead of always stopping at the same point: skip PRs covered earlier this rotation, scan + # up to the cap, defer the rest, then advance the cursor below so the next pass resumes where + # this one stopped (no tail starvation). Daemon rotates in-process; --once starts at the front. + idx=$((idx+1)) + [ "$idx" -le "$scan_cursor" ] && continue + if [ "$scanned" -ge "$CRQ_AUTOREVIEW_MAX_SCAN" ]; then deferred=1; continue; fi + scanned=$((scanned+1)) + head="$(gh api "repos/$repo/pulls/$pr" --jq '.head.sha // empty' 2>/dev/null | cut -c1-9)" + # Require a real short SHA β€” a failed lookup leaves a non-empty NON-hex error body (gh dumps + # it to stdout, --jq skipped), which must not be treated as a head and drive an enqueue (crq:975). + [ -n "$head" ] && [ -z "${head//[0-9a-f]/}" ] || continue + # DEDUP: skip if we already requested a review for this exact commit (pump records it). + firedsha="$(jq -r --arg k "$key" '.fired[$k] // empty' <<<"${json:-{}}" 2>/dev/null)" + [ "$firedsha" = "$head" ] && continue + # --paginate so the LATEST review is considered (not just page 1 on long-lived PRs). + # If the reviews lookup itself FAILS (rate-limit/transient), skip this PR β€” don't mistake an + # unreadable review history for "never reviewed" and enqueue a redundant review (crq:954). + if ! crrev_raw="$(gh api "repos/$repo/pulls/$pr/reviews" --paginate \ + --jq ".[]|select(.user.login==\"$CRQ_BOT\" and .commit_id!=null)|.commit_id" 2>/dev/null)"; then continue; fi + crrev="$(printf '%s' "$crrev_raw" | tail -1 | cut -c1-9)" + # enqueue in a subshell so a transient cmd_enqueue die can't kill the long-running watcher + if [ "$incremental" = "0" ]; then + # auto-review only: enqueue if never reviewed. crrev catches FORMAL reviews; also honor a + # comment-only completion (a CodeRabbit walkthrough/summary comment) so we don't re-review (crq:956). + if [ -z "$crrev" ]; then + # If the comments lookup FAILS, skip β€” don't treat unreadable comments as "not reviewed" + # and enqueue a redundant review (the comment scan is the only guard here) (crq:972). + if ! reviewed_raw="$(gh api "repos/$repo/issues/$pr/comments" --paginate --slurp 2>/dev/null)"; then continue; fi + reviewed="$(printf '%s' "$reviewed_raw" | jq --arg bot "$CRQ_BOT" --arg m "$CRQ_REVIEW_DONE_MARKER" 'add | map(select(.user.login==$bot and ((.body//"")|contains($m)))) | length' 2>/dev/null)" + [ "${reviewed:-0}" -gt 0 ] || { ( cmd_enqueue "$repo" "$pr" ) || true; } + fi + else + [ "$crrev" != "$head" ] && { ( cmd_enqueue "$repo" "$pr" ) || true; } # + incremental: HEAD not reviewed + fi + done < <(gh search prs "$search_flag" "$target" --state open --archived=false --limit 1000 --json number,repository \ + --jq '.[]|"\(.repository.nameWithOwner) \(.number)"' 2>/dev/null) + # gh search caps at 1000 results β€” if we hit it, some open PRs in this target are unseen. + # Surface it (no silent truncation) rather than partition; narrowing CRQ_REPOS is the fix. + [ "$searched" -ge 1000 ] && log "autoreview: '$target' returned the 1000-result search cap β€” some open PRs may be unseen; narrow CRQ_REPOS/CRQ_SCOPE" + done + # Advance the rotation cursor when we deferred PRs (resume there next pass); reset to the front + # once a pass covers everything within budget. Keeps large scopes progressing, not starving. + if [ "$deferred" = "1" ]; then + scan_cursor=$(( scan_cursor + scanned )) + log "autoreview: scanned $scanned this pass (cap $CRQ_AUTOREVIEW_MAX_SCAN); rotating to offset $scan_cursor next pass" + else + scan_cursor=0 + fi + ( cmd_pump ) >/dev/null 2>&1 || true # subshell: a transient pump die can't kill the watcher + [ "$once" = "1" ] && break + sleep "$CRQ_AUTOREVIEW_POLL" + done } usage() { @@ -580,7 +1038,11 @@ crq β€” CodeRabbit review queue (account-wide coordinator for parallel agents) USAGE crq init provision the gate repo (lock, dashboard issue, calibration PR) - crq wait enqueue and block until OUR review is fired (use this in loops) + crq wait enqueue + block until OUR review fires (exit 0=fired, 3=already reviewed, 2=timeout) + crq autoreview [--once] [--no-incremental] + emulate CodeRabbit auto-review (+ incremental) review, rate-coordinated: + enqueue open PRs needing review; loops unless --once. + --no-incremental = review each PR once only (no re-review on push) crq enqueue add to the FIFO queue (idempotent) crq pump fire <=1 queued review if the window is open (safe to call anywhere) crq calibrate refresh the account-wide rate-limit reading @@ -589,6 +1051,8 @@ USAGE crq gc drop closed PRs; clear timed-out in-flight crq refresh force a fresh calibration and print status crq unlock [--force] inspect / break the global lock + crq version print the crq version + crq help show this help is owner/name; is a number. Configure via env / \$HOME/.config/crq/env (CRQ_REPO is required). See 'crq init' output and the README. @@ -596,21 +1060,38 @@ EOF } main() { - command -v gh >/dev/null 2>&1 || die "gh (GitHub CLI) is required" - command -v jq >/dev/null 2>&1 || die "jq is required" local cmd="${1:-}"; shift || true + # These need nothing β€” handle before the gh/jq/auth preflight. + case "$cmd" in + ""|-h|--help|help) usage; return 0 ;; + -v|--version|version) echo "crq $CRQ_VERSION"; return 0 ;; + esac + # Everything else needs gh + jq, and an authenticated gh. + command -v gh >/dev/null 2>&1 || die "gh (GitHub CLI) is required β€” install: https://cli.github.com/" + # crq uses 'gh api --slurp' (added in gh 2.48.0). Gate on the PARSED version, not '--help' grepping + # (that varies by env and false-negatived). Only fail when the version is DEFINITIVELY older; if we + # can't parse a version, proceed silently rather than risk bricking the tool on a heuristic. + ghver="$(gh --version 2>/dev/null | sed -n 's/.*version \([0-9][0-9.]*\).*/\1/p' | head -1)" + if [ -n "$ghver" ]; then + ghmaj="${ghver%%.*}"; ghmin="${ghver#*.}"; ghmin="${ghmin%%.*}" + if [ "${ghmaj:-0}" -lt 2 ] || { [ "${ghmaj:-0}" -eq 2 ] && [ "${ghmin:-0}" -lt 48 ]; }; then + die "gh $ghver is too old β€” crq needs >= 2.48.0 for 'gh api --slurp'; upgrade: https://cli.github.com/" + fi + fi + command -v jq >/dev/null 2>&1 || die "jq is required β€” install: https://jqlang.github.io/jq/" + gh auth token >/dev/null 2>&1 || die "gh is not logged in β€” run: gh auth login" case "$cmd" in init) cmd_init "$@" ;; enqueue) [ $# -eq 2 ] || die "usage: crq enqueue "; cmd_enqueue "$@" ;; pump) cmd_pump ;; wait) [ $# -eq 2 ] || die "usage: crq wait "; cmd_wait "$@" ;; + autoreview) cmd_autoreview "$@" ;; calibrate) cmd_refresh ;; status) cmd_status ;; cancel) [ $# -eq 2 ] || die "usage: crq cancel "; cmd_cancel "$@" ;; gc) cmd_gc ;; refresh) cmd_refresh ;; unlock) cmd_unlock "${1:-}" ;; - ""|-h|--help|help) usage ;; *) die "unknown command: $cmd (try 'crq help')" ;; esac } diff --git a/examples/agent-loop.sh b/examples/agent-loop.sh deleted file mode 100755 index 8c94028..0000000 --- a/examples/agent-loop.sh +++ /dev/null @@ -1,50 +0,0 @@ -#!/usr/bin/env bash -# Example: an autonomous PR-review loop that uses crq to coordinate CodeRabbit -# review requests with every other agent sharing your account's rate limit. -# -# The ONLY crq-specific line is `crq wait "$REPO" "$PR"`. Everything else is your -# normal review-loop work. Because crq serializes firing account-wide, you can run -# this same loop for many PRs across many machines at once without competing. -# -# REPO=owner/name PR=123 ./agent-loop.sh -set -uo pipefail - -REPO="${REPO:?set REPO=owner/name}" -PR="${PR:?set PR=}" - -# crq config (or put these in ~/.config/crq/env, which crq sources automatically) -: "${CRQ_REPO:?set CRQ_REPO=youruser/crq-state (run 'crq init' once first)}" - -review_cycle_should_continue() { - # Stop when the PR is closed/merged. Replace with your own convergence check - # (e.g. both review bots reviewed the latest commit with nothing actionable left). - [ "$(gh pr view "$PR" --repo "$REPO" --json state -q .state 2>/dev/null)" = "OPEN" ] -} - -wait_for_review_to_land() { - # Block until CodeRabbit posts a review newer than 'since' (ISO8601), ~20 min cap. - local since="$1" deadline=$(( $(date +%s) + 1200 )) - while [ "$(date +%s)" -lt "$deadline" ]; do - local n - n="$(gh api "repos/$REPO/issues/$PR/comments" --paginate \ - --jq "[.[]|select(.user.login==\"coderabbitai[bot]\")|select(.created_at > \"$since\")]|length" 2>/dev/null)" - [ "${n:-0}" -gt 0 ] && return 0 - sleep 30 - done - return 0 -} - -apply_fixes_and_push() { - # YOUR work: read the review, fix real findings, run the project's gates, - # commit and push one round. Left as a stub here. - echo "[agent] processing review feedback for $REPO#$PR ..." -} - -while review_cycle_should_continue; do - fired_at="$(date -u +%Y-%m-%dT%H:%M:%SZ)" - crq wait "$REPO" "$PR" # <-- coordinated, FIFO, never fires while rate-limited - wait_for_review_to_land "$fired_at" - apply_fixes_and_push -done - -echo "[agent] $REPO#$PR converged / closed." diff --git a/examples/monitor.sh b/examples/monitor.sh new file mode 100755 index 0000000..87cda59 --- /dev/null +++ b/examples/monitor.sh @@ -0,0 +1,68 @@ +#!/usr/bin/env bash +# monitor.sh β€” background watcher for one PR, for fully-unattended review loops. +# +# It does two things: +# 1. Wakes you (exits NEW_FEEDBACK) when a review bot posts new feedback, so your agent +# can process it. +# 2. Between rounds, keeps the PR in crq's account-wide queue with the NON-blocking +# `crq enqueue` + `crq pump`. It never posts "@coderabbitai review" directly β€” crq +# owns that, so this monitor never competes with your other PRs/agents. +# +# Run it with your agent runner's background mode and re-arm it after each round. It exits: +# NEW_FEEDBACK -> new bot comment/review landed β€” go process it +# CRQ_PUMP (log line only; keeps running) crq pumped the queue +# IDLE_TIMEOUT ~75 min with nothing β€” re-arm if still working +# +# ./monitor.sh [owner/repo] (repo auto-detected from the current git repo) +set -u +PR="${1:?usage: monitor.sh [owner/repo]}" +REPO="${2:-$(gh repo view --json nameWithOwner -q .nameWithOwner 2>/dev/null)}" +: "${REPO:?usage: monitor.sh [owner/repo] β€” REPO not given and not inside a GitHub repo}" +# Honor crq's configured review bot + rate-limit marker (defaults = CodeRabbit, which crq +# coordinates). Override CRQ_BOT / CRQ_RL_MARKER for another bot. +# shellcheck source=/dev/null +[ -f "${CRQ_CONFIG:-$HOME/.config/crq/env}" ] && . "${CRQ_CONFIG:-$HOME/.config/crq/env}" +BOT="${CRQ_BOT:-coderabbitai[bot]}" +RL="${CRQ_RL_MARKER:-rate limited by coderabbit.ai}" +IDLE_CAP=$(( $(date -u +%s) + 4500 )) + +bot_count() { + # --slurp + a standalone jq with `add`: combine all pages before counting. Plain --paginate --jq + # runs the filter per page, so a new (non-bot) page could shift the string and false-wake the loop. + # Exclude rate-limit WARNING posts from EVERY counter (case-insensitive) β€” they're not review + # feedback and would otherwise wake the loop as NEW_FEEDBACK when no actual review arrived. + local f='add|map(select(.user.login==$bot and ((.body//"")|ascii_downcase|contains(($rl|ascii_downcase))|not)))|length' + c=$(gh api "repos/$REPO/pulls/$PR/comments" --paginate --slurp 2>/dev/null | jq --arg bot "$BOT" --arg rl "$RL" "$f" 2>/dev/null) + r=$(gh api "repos/$REPO/pulls/$PR/reviews" --paginate --slurp 2>/dev/null | jq --arg bot "$BOT" --arg rl "$RL" "$f" 2>/dev/null) + i=$(gh api "repos/$REPO/issues/$PR/comments" --paginate --slurp 2>/dev/null | jq --arg bot "$BOT" --arg rl "$RL" "$f" 2>/dev/null) + # A successful-but-empty response yields "0"; an EMPTY string means the gh call failed. Don't + # fabricate 0:0:0 from a transient failure (it would look like feedback vanished and false-wake) β€” + # signal an error so the caller skips this tick. + [ -n "$c" ] && [ -n "$r" ] && [ -n "$i" ] || return 1 + echo "$c:$r:$i" +} +cr_last_review() { # echoes the last CodeRabbit-reviewed commit (short); returns 1 if the lookup FAILED + local out + out="$(gh api "repos/$REPO/pulls/$PR/reviews" --paginate \ + --jq ".[]|select(.user.login==\"$BOT\" and .commit_id!=null)|.commit_id" 2>/dev/null)" || return 1 + printf '%s' "$out" | tail -1 | cut -c1-9 +} + +BASE=""; while [ -z "$BASE" ]; do BASE=$(bot_count) || sleep 5; done # establish a real baseline (retry past transient failures) +echo "monitor PR#$PR repo=$REPO base=$BASE" +while true; do + CUR=$(bot_count) || { sleep 60; continue; } # transient API failure -> skip this tick, don't false-wake + [ "$CUR" != "$BASE" ] && { echo "NEW_FEEDBACK $BASE -> $CUR"; exit 0; } + # compare the REMOTE PR head (not the local checkout, which may be ahead/behind/elsewhere). + # Skip the enqueue decision if EITHER lookup fails β€” don't treat an unreadable head/review as + # "needs a review" and enqueue redundantly (CRREV=$(...) fails -> the && chain short-circuits). + HEAD=$(gh api "repos/$REPO/pulls/$PR" --jq '.head.sha // empty' 2>/dev/null | cut -c1-9) + # Require a real short SHA (a failed lookup leaves a non-empty non-hex error body) AND a successful + # cr_last_review β€” don't enqueue on an unreadable head/review. + if [ -n "$HEAD" ] && [ -z "${HEAD//[0-9a-f]/}" ] && CRREV=$(cr_last_review) && [ "$CRREV" != "$HEAD" ]; then + crq enqueue "$REPO" "$PR" >/dev/null 2>&1 # join the account-wide FIFO queue + crq pump >/dev/null 2>&1 && echo "CRQ_PUMP $HEAD" # fire <=1 review if globally unblocked + fi + [ "$(date -u +%s)" -ge "$IDLE_CAP" ] && { echo "IDLE_TIMEOUT"; exit 0; } + sleep 60 +done diff --git a/examples/review-loop.sh b/examples/review-loop.sh new file mode 100755 index 0000000..f1c2e5d --- /dev/null +++ b/examples/review-loop.sh @@ -0,0 +1,75 @@ +#!/usr/bin/env bash +# review-loop.sh β€” autonomously address CodeRabbit feedback on a PR until it converges. +# +# Run one of these per PR, on as many PRs/machines as you like. Because every review +# request goes through `crq`, they all share ONE account-wide queue and fire one at a +# time, in order, only when CodeRabbit has capacity β€” no stampede, no rate-limit spam. +# +# The only crq-specific line is `crq wait "$REPO" "$PR"`. Everything else is your loop. +# +# REPO=owner/name PR=123 ./review-loop.sh +set -uo pipefail + +REPO="${REPO:?set REPO=owner/name}" +PR="${PR:?set PR=}" +# crq reads its config from here too; source it so CRQ_REPO need not be exported in this shell. +# shellcheck source=/dev/null +[ -f "${CRQ_CONFIG:-$HOME/.config/crq/env}" ] && . "${CRQ_CONFIG:-$HOME/.config/crq/env}" +: "${CRQ_REPO:?run 'crq init' once and configure ~/.config/crq/env (see the README)}" +BOT="${CRQ_BOT:-coderabbitai[bot]}" # honor crq's configured review bot… +RL="${CRQ_RL_MARKER:-rate limited by coderabbit.ai}" # …and rate-limit marker, instead of hardcoding + +# Keep looping unless the PR is *explicitly* CLOSED/MERGED β€” a transient gh/API failure returns +# an empty state, which must NOT be mistaken for a real closure (that would exit the loop early). +still_open() { local s; s="$(gh pr view "$PR" --repo "$REPO" --json state -q .state 2>/dev/null)"; [ "$s" != "CLOSED" ] && [ "$s" != "MERGED" ]; } + +# Replace this with your real logic: read CodeRabbit's findings, fix the genuine ones, +# run the project's gates (tests/lint/typecheck), then commit & push ONE round. Pushing a +# new commit is what makes the next CodeRabbit review meaningful. +process_review_and_push() { + echo "[loop] TODO: read review, fix findings, validate, commit & push for $REPO#$PR" +} + +# Block until CodeRabbit posts something newer than $1 (ISO8601), ~20 min cap. CodeRabbit can +# complete as a conversation comment OR a formal PR review, so check both. +wait_for_review() { + local since="$1" _ n r + for _ in $(seq 1 40); do + # --slurp + a standalone jq with `add`: combine all pages before counting (plain --paginate + # runs the filter per page; gh forbids --slurp with --jq, so pipe to jq). + # A rate-limit WARNING is a fresh coderabbitai comment but NOT real feedback β€” exclude it, + # else we'd "process" a round that never got reviewed (crq requeues it; we shouldn't push). + n=$(gh api "repos/$REPO/issues/$PR/comments" --paginate --slurp 2>/dev/null \ + | jq --arg bot "$BOT" --arg rl "$RL" --arg since "$since" 'add | map(select(.user.login==$bot and ((.updated_at // .created_at) > $since) and ((.body//"")|ascii_downcase|contains($rl|ascii_downcase)|not))) | length' 2>/dev/null) + r=$(gh api "repos/$REPO/pulls/$PR/reviews" --paginate --slurp 2>/dev/null \ + | jq --arg bot "$BOT" --arg since "$since" 'add | map(select(.user.login==$bot and .submitted_at > $since)) | length' 2>/dev/null) + { [ "${n:-0}" -gt 0 ] || [ "${r:-0}" -gt 0 ]; } && return 0 + sleep 30 + done + return 1 # timed out β€” no new review landed +} + +while still_open; do + # crq wait is coordinated/FIFO and never fires while rate-limited. Exit codes: + # 0 = our review was fired 3 = deduped (this commit was already reviewed) other = timeout/error + crq wait "$REPO" "$PR"; rc=$? + case "$rc" in + 0) ;; # fired -> wait for new feedback below + 3) # Deduped: this HEAD was already reviewed, so the feedback ALREADY exists β€” process it (don't + # skip), then back off so an unchanged head (e.g. process pushed nothing) can't hot-loop. + echo "[loop] $REPO#$PR already reviewed at this commit β€” processing existing feedback" + process_review_and_push; sleep "${LOOP_IDLE_SLEEP:-60}"; continue ;; + *) echo "[loop] crq wait did not fire (timeout/error) β€” backing off"; sleep "${LOOP_IDLE_SLEEP:-60}"; continue ;; + esac + # Start the feedback window AFTER crq fires (a delayed response that lands while we were blocked + # in `crq wait` would otherwise falsely satisfy the poll). Back up 1s so a comment created in the + # same second as 'now' isn't missed by the strictly-newer (`>`) comparison. + since="$(date -u -d '1 second ago' +%Y-%m-%dT%H:%M:%SZ 2>/dev/null || date -u -v-1S +%Y-%m-%dT%H:%M:%SZ)" + if ! wait_for_review "$since"; then + echo "[loop] no new review within the cap β€” not pushing a round on stale feedback" + continue + fi + process_review_and_push +done + +echo "βœ… $REPO#$PR converged or closed." diff --git a/skills/coderabbit-queue/SKILL.md b/skills/coderabbit-queue/SKILL.md index 7750d8d..06cbfa7 100644 --- a/skills/coderabbit-queue/SKILL.md +++ b/skills/coderabbit-queue/SKILL.md @@ -23,6 +23,14 @@ This enqueues `(repo, pr)`, blocks until it's *your* turn and the account is unb then posts the review command exactly once. It is safe to run for many PRs across many machines simultaneously β€” `crq` guarantees no two reviews fire at the same time. +## Required prerequisite + +**CodeRabbit auto-review must be OFF.** crq is pull-only β€” it controls *when* reviews happen by +posting `@coderabbitai review`. If auto-review is enabled, CodeRabbit reviews every push on its +own, bypassing the queue and consuming the shared rate limit, which defeats crq. Disable it +org-wide in the CodeRabbit dashboard (Settings β†’ Review β†’ Automatic Review) or per-repo via +`.coderabbit.yaml` (`reviews.auto_review.enabled: false`). + ## Setup (once per account) ```bash @@ -33,9 +41,16 @@ crq init # creates gate repo, dashboard issue, ca If `crq` is not installed: `curl -fsSL https://raw.githubusercontent.com/kristofferR/coderabbit-queue/main/install.sh | bash` +> **Trust note:** `curl … | bash` runs remote code without local review β€” fine if you trust this +> repo, but for autonomous agents or stricter setups, prefer downloading `install.sh` (or the single +> `crq` script) and reading it before running, as the README's manual-install section shows. + ## Commands you'll use - `crq wait ` β€” the loop primitive (enqueue + block until fired). +- `crq autoreview` β€” emulate native auto-review + incremental review (which are off) for ALL open + PRs, rate-coordinated. Run as a background watcher. `--no-incremental` = first review only; + `--once` = single pass. Use this to keep every PR reviewed hands-off without native auto-review. - `crq status` β€” show the queue + rate-limit state (good for a quick check). - `crq pump` β€” fire ≀1 queued review if the window is open (any agent can call it; the lock serializes it). `crq wait` calls this internally; you rarely call it directly.