Skip to content

Reduce redundant PR diff/file-list API reads with head-SHA-aware prefetch caching in top review workflows - #47997

Merged
pelikhan merged 2 commits into
mainfrom
copilot/deep-report-cache-pr-file-lists
Jul 25, 2026
Merged

Reduce redundant PR diff/file-list API reads with head-SHA-aware prefetch caching in top review workflows#47997
pelikhan merged 2 commits into
mainfrom
copilot/deep-report-cache-pr-file-lists

Conversation

Copilot AI commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

REST API usage has grown sharply, with PR review workflows repeatedly re-fetching the same PR diff/file metadata on each run. This change makes prefetch reuse deterministic per PR head commit so repeated reviewers consume cached data instead of reissuing equivalent API reads.

  • Shared prefetch helper: head-SHA cache validation

    • Updated shared/pr-diff-data-fetch.md to gate cache hits on a persisted head-SHA marker (/tmp/gh-aw/agent/pr-data-head-sha.txt) matching the current PR head.
    • Included headRefOid in pre-fetched metadata and fallback head resolution, so stale cache reuse is avoided when the PR advances.
  • Prefetch producer alignment

    • Updated pr-data-prefetch.yml to write the head-SHA marker alongside existing prefetch artifacts and include headRefOid in metadata payloads.
  • Top reviewer workflow key consistency

    • Normalized Matt Pocock reviewer cache keying/fallback to use pull_request.head.sha || issue.number, matching shared prefetch behavior and slash-command compatibility.
    • Recompiled affected workflow lock files to keep runtime artifacts in sync.
  • Test Quality Sentinel cache adoption

    • Added cache restore/save wiring to test-quality-sentinel.md, keyed by PR head SHA (with issue fallback).
    • Added sentinel-specific head-SHA marker validation (test-data-head-sha.txt) to short-circuit redundant gh pr view / gh pr diff fetches on repeated runs of the same head commit.
  • Contract coverage for cache invariants

    • Added pkg/cli/pr_prefetch_cache_contract_test.go to lock in:
      • head-SHA marker persistence in shared prefetch flow,
      • head-aware cache key presence in top workflows,
      • sentinel head-SHA cache behavior wiring.
# shared/pr-diff-data-fetch.md (cache-hit gate)
if [ -n "$CURRENT_HEAD_SHA" ] && [ "$CURRENT_HEAD_SHA" = "$CACHE_HEAD_SHA" ] && \
   [ -f /tmp/gh-aw/agent/pr-diff.patch ] && \
   [ -f /tmp/gh-aw/agent/pr-meta.json ] && \
   [ -f /tmp/gh-aw/agent/pr-review-comments.json ]; then
  echo "Cache hit: using pre-fetched PR data for head ${CURRENT_HEAD_SHA}"
else
  # refresh prefetch artifacts and persist head marker
fi

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Add caching for PR file-lists and diffs in top-4 workflows Reduce redundant PR diff/file-list API reads with head-SHA-aware prefetch caching in top review workflows Jul 25, 2026
Copilot AI requested a review from pelikhan July 25, 2026 13:21
@github-actions

Copy link
Copy Markdown
Contributor

Hey @copilot-swe-agent 🤖 — thanks for working on the PR file-list caching optimization! This PR is currently in draft status with no changes committed yet. Here are a few things to complete before this is ready for review:

  • Add implementation and tests — the checklist indicates several tasks are incomplete, including implementing the cache logic, adding tests for cache hit/miss behavior, and validation. Make sure the branch includes all the actual code changes.
  • Commit the code changes — the current PR shows 0 files changed, 0 additions, 0 deletions. Push the implementation once the work is complete.
  • Ensure test coverage — add focused tests covering cache hit/miss scenarios and head-SHA keying as listed in the checklist.

Once the work is complete and the PR is ready, you can mark it as ready for review (remove the draft status). The core team will pick it up from there!

Generated by ✅ Contribution Check · sonnet46 · 61.6 AIC · ⌖ 10.4 AIC · ⊞ 6.2K ·

@pelikhan
pelikhan marked this pull request as ready for review July 25, 2026 13:22
Copilot AI review requested due to automatic review settings July 25, 2026 13:22
@pelikhan
pelikhan merged commit 3f934e4 into main Jul 25, 2026
23 checks passed
@pelikhan
pelikhan deleted the copilot/deep-report-cache-pr-file-lists branch July 25, 2026 13:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds head-SHA-aware caching to reduce repeated PR metadata and diff API reads.

Changes:

  • Persists and validates PR head-SHA cache markers.
  • Adds caching to Test Quality Sentinel and normalizes Matt reviewer keys.
  • Adds contract tests and recompiles generated workflows.
Show a summary per file
File Description
pkg/cli/pr_prefetch_cache_contract_test.go Adds cache contract checks.
.github/workflows/test-quality-sentinel.md Adds Sentinel prefetch caching.
.github/workflows/test-quality-sentinel.lock.yml Recompiles Sentinel workflow.
.github/workflows/shared/pr-diff-data-fetch.md Validates shared cache head SHA.
.github/workflows/pr-data-prefetch.yml Persists producer head metadata.
.github/workflows/pr-code-quality-reviewer.lock.yml Recompiles shared prefetch logic.
.github/workflows/mattpocock-skills-reviewer.md Adds slash-command cache fallback.
.github/workflows/mattpocock-skills-reviewer.lock.yml Recompiles Matt reviewer.
.github/workflows/impeccable-skills-reviewer.lock.yml Recompiles shared prefetch logic.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (2)

.github/workflows/test-quality-sentinel.md:39

  • The issue-number fallback makes slash commands reach this step, but github.event.pull_request.base.sha is empty for those events and the compiled checkout has no PR ref. Consequently the local git diff calls at lines 81 and 109 fail and || true silently produces incomplete analysis inputs that can be cached. Resolve/check out the PR head and base for slash events, or derive these analyses from the API diff instead.
      PR_NUMBER: ${{ github.event.issue.number || github.event.pull_request.number }}
      PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
      EXPR_GITHUB_EVENT_PULL_REQUEST_BASE_SHA: ${{ github.event.pull_request.base.sha }}

.github/workflows/shared/pr-diff-data-fetch.md:55

  • The metadata payload now includes headRefOid, but the output contract below still lists the old field set at line 105. Update that documentation so consumers know the head OID is available.
          --json number,title,body,headRefName,headRefOid,additions,deletions,changedFiles,files \
  • Files reviewed: 9/9 changed files
  • Comments generated: 4
  • Review effort level: Medium

Comment on lines +21 to +25
cache:
key: pr-test-prefetch-${{ github.event.pull_request.head.sha || github.event.issue.number }}
path: /tmp/gh-aw/agent
restore-keys:
- pr-test-prefetch-${{ github.event.pull_request.number || github.event.issue.number }}-
---
cache:
key: pr-prefetch-${{ github.event.pull_request.head.sha }}
key: pr-prefetch-${{ github.event.pull_request.head.sha || github.event.issue.number }}
CACHE_HEAD_SHA="$(tr -d '\n' < /tmp/gh-aw/agent/pr-data-head-sha.txt)"
fi
# Skip fetch only when cache data matches current PR head commit.
if [ -n "$CURRENT_HEAD_SHA" ] && [ "$CURRENT_HEAD_SHA" = "$CACHE_HEAD_SHA" ] && [ -f /tmp/gh-aw/agent/pr-diff.patch ] && [ -f /tmp/gh-aw/agent/pr-meta.json ] && [ -f /tmp/gh-aw/agent/pr-review-comments.json ]; then
Comment on lines +46 to +47
assert.Contains(t, text, "key: pr-test-prefetch-${{ github.event.pull_request.head.sha || github.event.issue.number }}", "Test Quality Sentinel should define a head-aware cache key")
assert.Contains(t, text, "test-data-head-sha.txt", "Test Quality Sentinel should persist cache head SHA marker")
@github-actions

Copy link
Copy Markdown
Contributor

🎉 This pull request is included in a new release.

Release: v0.83.4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[deep-report] Cache PR file-lists/diffs in the top-4 review workflows to curb +46%/30d REST API growth

3 participants