workflow: log each review to ai-review-log (Phase 2 v2) - #42
Merged
Conversation
After claude-code-action finishes, read Claude's PR comment and POST a summary issue to HarperFast/ai-review-log (private). Labels: repo:oauth, verdict:pending, phase:baseline. Nathan sweeps verdict:pending weekly and relabels useful/noise/partial to calibrate. Design notes: - One issue per review (not per finding). Simpler to ship; per-finding granularity can be added in v2.1 if calibration needs it. - `if: always()` so we log even if the review itself errored — the review_status field in the log issue body captures that. - Authentication: PAT AI_REVIEW_LOG_TOKEN is scoped to HarperFast/ai-review-log only (Issues R/W, Metadata R). Passed via curl's Authorization header for the log POST only; GH_TOKEN for `gh pr view` stays as the default GITHUB_TOKEN (read PR context). - Fail-open: if AI_REVIEW_LOG_TOKEN is missing, Claude didn't post a comment, or the POST fails, we warn but never fail the job. The review's primary value is the PR comment; the log is secondary. - Title derivation from the PR comment body differentiates "no blockers" from findings count. Labels already pre-created in the log repo. Requires AI_REVIEW_LOG_TOKEN secret on HarperFast/oauth. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
After
claude-code-actionposts its review comment on a PR, this adds a follow-up workflow step that reads the comment and POSTs a summary issue toHarperFast/ai-review-log(private). Each logged issue gets labelsrepo:oauth,verdict:pending,phase:baseline. You sweepverdict:pendingweekly and relabeluseful/noise/partial— that triage becomes the calibration data.Design
if: always()so we log even when the review step itself errors — the review's outcome status is captured in the log body.AI_REVIEW_LOG_TOKEN(PAT scoped to justHarperFast/ai-review-logIssues R/W) is passed viacurl'sAuthorizationheader for the log POST only.GH_TOKENforgh pr viewstays as the defaultGITHUB_TOKEN(read-only on the reviewed repo).[oauth] PR #N: no blockersvs[oauth] PR #N: 3 finding(s) — triage pending. The body includes source URL, model, date, review-job status, and the full Claude comment verbatim.Prereqs (done)
HarperFast/ai-review-logrepo created (private/internal, issues enabled)AI_REVIEW_LOG_TOKENadded toHarperFast/oauthsecrets (fine-grained, Issues R/W on log repo only)repo:*,verdict:*,phase:*)Test plan
HarperFast/ai-review-logwith correct labels and body