Stand up a CodeRabbit-behavior lab repo to empirically harden crq's parsers

[kristofferR]

Idea

Spin up a separate, throwaway repo whose only purpose is to make crq better — a
CodeRabbit (and Codex) behavior lab. In it we deliberately do weird, adversarial, and
edge-case things to PRs to provoke the bots into responding in every shape they're capable
of, capture the raw output, and feed what we learn back into crq's parsing, severity, dedup,
and rate-limit logic.

Right now crq's correctness is reverse-engineered from whatever reviews we happen to see in
real work. That's a biased, sparse sample — we only harden the parser against formats we
stumble into. A dedicated lab lets us go looking for the formats instead of waiting to get
burned by one in production.

Why this matters for crq

crq's value is almost entirely in faithfully normalizing CodeRabbit/Codex output to JSON.
Every response shape we don't anticipate is a finding crq silently drops or mis-parses. The
parser already juggles a lot of distinct surfaces (internal/crq/feedback.go):

• Inline review comments (with severity inferred from body text / emoji like 🔵)
• Review-body "Outside diff range" findings
• Collapsed <details> / <summary> sections
• "Prompt for all review comments with AI agents" prompt-blocks (In @file: / - Around line N:)
• Codex findings posted as issue comments
• @coderabbitai rate limit replies (parsed for the real account-wide limit)
• Resolution / outdated / nitpick / non-actionable classification

Each of those was learned the hard way. A lab repo turns "learn the hard way, in prod" into
"learn on purpose, in a sandbox."

What we'd actually do in the lab ("the weird stuff")

Open PRs engineered to elicit specific bot behaviors, then record the raw responses. Ideas:

• Force every severity tier — plant obvious critical bugs, security issues, perf issues,
  pure nitpicks, and style-only changes; catalog exactly how CodeRabbit labels each (emoji,
  wording, "Actionable" vs "Nitpick" vs "Outside diff range").
• Force every container format — huge diffs (Outside-diff-range), tiny diffs, binary
  files, renames, deletions, generated files, lockfiles — see which land inline vs in the
  review body vs collapsed <details>.
• Break the parsers on purpose — code/comments containing markdown, backticks, fake
  <details> blocks, fake "Prompt for AI agents" sections, emoji, triple-backtick fences,
  and other content designed to confuse our regexes. Confirm crq doesn't mis-attribute.
• Multi-finding / multi-file PRs — verify ordering, dedup (prompt-block vs inline at the
  same location), and that nothing is lost.
• Resolution & re-review behavior — resolve, then push; mark outdated; reopen; confirm
  crq's "still-open from earlier commits" logic matches GitHub's actual state.
• Rate-limit behavior — exercise @coderabbitai rate limit across plan tiers/states and
  capture every message variant crq must parse to know the real limit.
• Codex side — same battery for Codex's issue-comment findings and its re-review-on-push
  behavior.
• Config matrix — vary .coderabbit.yaml (auto-review on/off, path filters, profiles,
  tone) and record how the output changes.

Deliverables

1. A corpus of real bot responses — captured raw (comment bodies / review payloads) and
   committed as golden fixtures the lab can re-emit.
2. A parser test suite for crq built from that corpus, so regressions are caught by
   go test, not by a missed finding in production.
3. A living doc ("CodeRabbit response formats observed") cataloging each shape, what
   triggers it, and which crq code path handles it.
4. Concrete crq issues/PRs for every gap the lab uncovers.

Open questions

• Where does it live / which account? Separate public repo under kristofferR, or under
  the anon account? It will accumulate a lot of CodeRabbit traffic.
• Rate-limit cost. Experiments burn the same account-wide CodeRabbit review budget and
  GitHub REST quota that real work shares. Lab runs should probably be gated/scheduled (or use
  a separate CodeRabbit org/seat) so they don't starve actual PRs. crq's own queue should
  arguably drive the lab too, for realism.
• Reproducibility. Bot behavior drifts as CodeRabbit ships changes — the corpus needs a
  date/version stamp and periodic refresh, not a one-time snapshot.
• Automation. Could a crq lab subcommand (or a workflow) generate the provocation PRs,
  trigger reviews, and harvest responses into fixtures automatically?

Scope note

This is meta-tooling for crq, not a crq feature itself. The output is knowledge + test
fixtures + parser hardening, with the lab repo kept entirely separate so its noise never
touches crq's real history.