fix(ci-cd-gaps-assessment): resolve AI credits rate limit with DataOps refactor by Copilot · Pull Request #5165 · github/gh-aw-firewall

Copilot · 2026-06-17T14:57:32Z

The ci-cd-gaps-assessment workflow hit the Copilot AI credits rate limit (HTTP 429) due to daily scheduling, agent-driven GitHub data fetching via the full MCP server stack, and no per-run budget cap.

Changes

Schedule: daily → weekly on Monday — 7× fewer runs for a slow-changing gap assessment
DataOps pre-fetch step: All GitHub data (gh workflow list, gh run list, aggregated stats) is now materialized to /tmp/gh-aw/ci-assessment/ before the agent starts — zero AI tokens for data collection
Remove agentic-workflows tool + imports: Eliminated the shared/mcp-pagination.md / shared/mcp/gh-aw.md MCP server stack and associated tool-schema overhead
gh-proxy mode: Switched github: tools from full MCP server to mode: gh-proxy with toolsets: [default]; added bash: true so the agent reads pre-fetched files
max-ai-credits: 500: Hard per-run budget cap to prevent future rate-limit events
close-older-discussions: true: Prevents stale discussion accumulation on the weekly cadence
Prompt tightened: Agent is directed to cat pre-fetched files rather than explore via API calls; prompt body reduced ~40%

- Change schedule from daily to weekly on Monday (7× fewer runs) - Switch GitHub tools from MCP server to gh-proxy mode - Remove agentic-workflows tool and its MCP imports - Add DataOps steps to pre-fetch all workflow/run data via gh CLI - Add bash tool so agent reads pre-fetched files instead of calling APIs - Add max-ai-credits: 500 guardrail to prevent future rate limit hits - Add close-older-discussions: true to avoid stale discussion accumulation - Tighten prompt body to use pre-fetched data (lower input tokens) - Recompile lock file

github-actions · 2026-06-17T15:36:30Z

✅ Coverage Check Passed

Overall Coverage

Metric	Base	PR	Delta
Lines	97.30%	97.34%	📈 +0.04%
Statements	97.16%	97.20%	📈 +0.04%
Functions	98.84%	98.84%	➡️ +0.00%
Branches	91.93%	91.96%	📈 +0.03%

📁 Per-file Coverage Changes (1 files)

File	Lines (Before → After)	Statements (Before → After)
`src/workdir-setup.ts`	92.7% → 94.5% (+1.82%)	92.7% → 94.5% (+1.82%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

Copilot

Pull request overview

This PR adjusts the ci-cd-gaps-assessment agentic workflow to reduce Copilot AI credits consumption by running less frequently and by pre-materializing GitHub Actions data on the runner before the agent starts, shifting the agent toward local file reads instead of API exploration.

Changes:

Switched schedule from daily to weekly (Monday) and added a per-run max-ai-credits: 500 cap.
Added a runner-side prefetch step that writes workflow/run data and aggregated stats into /tmp/gh-aw/ci-assessment/, and updated the prompt to direct the agent to read those files.
Switched GitHub tooling to mode: gh-proxy, enabled bash: true, and updated the compiled lock workflow accordingly (including CLI proxy startup).

Show a summary per file

File	Description
.github/workflows/ci-cd-gaps-assessment.md	Updates schedule/budget, adds CI data prefetch step, and rewrites agent instructions to consume pre-fetched files.
.github/workflows/ci-cd-gaps-assessment.lock.yml	Regenerates the compiled workflow to reflect gh-proxy/CLI-proxy usage, the new schedule, and the prefetch step.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 2/2 changed files
Comments generated: 9

      - name: Setup Scripts
        id: setup
-        uses: github/gh-aw-actions/setup@5c2fe865bb4dc46e1450f6ee0d0541d759aea73a # v0.79.6
+        uses: github/gh-aw-actions/setup@v0.79.8
        with:
          destination: ${{ runner.temp }}/gh-aw/actions


      - name: Setup Scripts
        id: setup
-        uses: github/gh-aw-actions/setup@5c2fe865bb4dc46e1450f6ee0d0541d759aea73a # v0.79.6
+        uses: github/gh-aw-actions/setup@v0.79.8
        with:
          destination: ${{ runner.temp }}/gh-aw/actions


+        env:
+          GH_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }}
+          GITHUB_SERVER_URL: ${{ github.server_url }}
+          CLI_PROXY_POLICY: '{"allow-only":{"repos":"all","min-integrity":"none"}}'


-            echo "::error::Failed to find gh-aw binary for MCP server"
-            exit 1
-          fi
+        run: bash "${RUNNER_TEMP}/gh-aw/actions/download_docker_images.sh" ghcr.io/github/gh-aw-firewall/agent:0.27.2@sha256:f88e5b17b6b7a600117bc121114d6ce2155c88c983c0c939c5df884f730fa1d6 ghcr.io/github/gh-aw-firewall/api-proxy:0.27.2@sha256:ee39841d980878ebbb87592903b06d31a1af500c71525c9616f7e8e2a27041a4 ghcr.io/github/gh-aw-firewall/cli-proxy:0.27.2@sha256:02f3ec08f32dc26c5427920c6a2e2f3036238fce44802f2f11ef49ed8621b5d0 ghcr.io/github/gh-aw-firewall/squid:0.27.2@sha256:2e3a717e5f19a654cd9a2263beb52012b56bcb68562ec5ae2e42f9d156b49591 ghcr.io/github/gh-aw-mcpg:v0.3.25@sha256:c10331ad17668ef89f38f5e356678788a40b0cd5fef96e8f92e1d9c1de47cbaa ghcr.io/github/github-mcp-server:v1.1.2@sha256:30197479d8036c7811892bc07e06f9a05c9ef3cdd79bc59f256d50647f95788c


+      gh workflow list --repo "$GITHUB_REPOSITORY" \
+        --json name,state,path \
+        > /tmp/gh-aw/ci-assessment/workflows.json \
+        || echo '[]' > /tmp/gh-aw/ci-assessment/workflows.json


+      gh run list --repo "$GITHUB_REPOSITORY" \
+        --limit 50 \
+        --json name,status,conclusion,createdAt,event,workflowName \
+        > /tmp/gh-aw/ci-assessment/recent-runs.json \
+        || echo '[]' > /tmp/gh-aw/ci-assessment/recent-runs.json


+      gh run list --repo "$GITHUB_REPOSITORY" \
+        --event pull_request \
+        --limit 30 \
+        --json name,status,conclusion,createdAt,workflowName \
+        > /tmp/gh-aw/ci-assessment/pr-runs.json \
+        || echo '[]' > /tmp/gh-aw/ci-assessment/pr-runs.json


      - name: Setup Scripts
        id: setup
-        uses: github/gh-aw-actions/setup@5c2fe865bb4dc46e1450f6ee0d0541d759aea73a # v0.79.6
+        uses: github/gh-aw-actions/setup@v0.79.8


      - name: Setup Scripts
        id: setup
-        uses: github/gh-aw-actions/setup@5c2fe865bb4dc46e1450f6ee0d0541d759aea73a # v0.79.6
+        uses: github/gh-aw-actions/setup@v0.79.8


lpcox · 2026-06-17T15:52:24Z

@copilot address review feedback

github-actions · 2026-06-17T15:52:42Z

⏳ Copilot review left inline comments.

@copilot To proceed:

Ask @copilot to address the review feedback (reply to this comment or the review thread)
Once the fix is pushed, add the ready-for-aw label to trigger agentic CI smoke tests

github-actions · 2026-06-17T16:21:08Z

❌ Smoke Copilot BYOK AOAI (Entra) reports failed. AOAI BYOK (Entra) mode investigation needed...

github-actions · 2026-06-17T16:21:19Z

🔑 Smoke Copilot PAT PAT auth validated. All systems operational. ✅

github-actions · 2026-06-17T16:21:39Z

✅ Build Test Suite completed successfully!

github-actions · 2026-06-17T16:21:49Z

❌ Contribution Check failed. Please review the logs for details.

github-actions · 2026-06-17T16:22:01Z

🔌 Smoke Services — All services reachable! ✅

github-actions · 2026-06-17T16:22:09Z

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

github-actions · 2026-06-17T16:22:11Z

✅ Smoke Gemini completed. All facets verified. 💎

github-actions · 2026-06-17T16:22:15Z

Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.

github-actions · 2026-06-17T16:22:16Z

✅ Smoke Copilot BYOK AOAI (api-key) completed. Copilot AOAI BYOK (api-key) mode operational. 🔓

github-actions · 2026-06-17T16:22:16Z

❌ Smoke Claude failed

github-actions · 2026-06-17T16:22:16Z

❌ Smoke Copilot BYOK reports failed. BYOK mode investigation needed...

github-actions · 2026-06-17T16:22:25Z

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

github-actions · 2026-06-17T16:22:32Z

📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅

github-actions · 2026-06-17T16:24:44Z

🚀 Security Guard has started processing this pull request

github-actions · 2026-06-17T16:52:21Z

Merged PRs: Reduce Pelis Advisor AI credit burn with fixed low-cost model + turn cap; fix(ci-cd-gaps-assessment): resolve AI credits rate limit with DataOps refactor

MCP Testing ✅
GitHub.com Connectivity ✅
File I/O ✅
BYOK Inference ✅
Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw)
✅ PASS @lpcox @Copilot

🔑 BYOK (AOAI api-key) report filed by Smoke Copilot BYOK AOAI (api-key)

github-actions · 2026-06-17T16:52:40Z

🔬 Smoke Test Results — PR #5165

"fix(ci-cd-gaps-assessment): resolve AI credits rate limit with DataOps refactor"
Author: @Copilot | Assignees: @lpcox @Copilot

Test	Status
GitHub MCP connectivity	✅
GitHub.com HTTP connectivity	⚠️ pre-step data unavailable (template vars unresolved)
File write/read	⚠️ pre-step data unavailable (template vars unresolved)

Overall: ⚠️ PARTIAL — MCP ✅, pre-computed step outputs not injected into agent context.

📰 BREAKING: Report filed by Smoke Copilot

github-actions · 2026-06-17T16:52:41Z

🔬 Smoke Test: Copilot PAT Auth — PASS

Test	Result
GitHub MCP connectivity	✅
github.com HTTP connectivity	✅
File write/read	✅ `/tmp/gh-aw/agent/smoke-test-copilot-pat-27703426010.txt`

PR: fix(ci-cd-gaps-assessment): resolve AI credits rate limit with DataOps refactor
Author: @Copilot | Assignees: @lpcox @Copilot
Auth mode: PAT (COPILOT_GITHUB_TOKEN)

Overall: PASS ✅

🔑 PAT report filed by Smoke Copilot PAT

github-actions · 2026-06-17T16:52:52Z

PR: fix(ci-cd-gaps-assessment): resolve AI credits rate limit with DataOps refactor
✅ GitHub PR review
✅ safeinputs-gh fallback via gh API
✅ Playwright GitHub title check
✅ local file write/read
✅ discussion interaction
✅ npm ci && npm run build
Overall: PASS

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex

github-actions · 2026-06-17T16:53:02Z

Smoke Test: API Proxy OpenTelemetry Tracing

Scenario	Result	Detail
1. Module Loading	✅	`otel.js` loads; exports `startRequestSpan`, `setTokenAttributes`, `setBudgetAttributes`, `endSpan`, `endSpanError`, `shutdown`, `isEnabled` + internals
2. Test Suite	✅	59 passed, 0 failed across `otel.test.js` + `otel-fanout.test.js` (span creation, token attrs, exporters, serialization, parent context)
3. Env Var Forwarding	✅	`api-proxy-service-config.ts` forwards `GH_AW_OTLP_ENDPOINTS`, `OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_EXPORTER_OTLP_HEADERS`, `GITHUB_AW_OTEL_TRACE_ID`, `GITHUB_AW_OTEL_PARENT_SPAN_ID`, `OTEL_SERVICE_NAME`; `observability-environment.ts` auto-passes all `OTEL_*` vars
4. Token Tracker Integration	✅	`onUsage` callback present in `token-tracker-http.js` (line 324); `onSpanEnd` hook wired in `upstream-response.js` → `setTokenAttributes()` → GenAI semantic conventions
5. OTEL Diagnostics	✅	No active container run; graceful degradation confirmed — when no OTLP endpoint configured, `FileSpanExporter` writes to `/var/log/api-proxy/otel.jsonl` (no errors)

All scenarios pass ✅

📡 OTel tracing validated by Smoke OTel Tracing

github-actions · 2026-06-17T16:53:41Z

Chroot Version Comparison

Runtime	Host Version	Chroot Version	Match?
Python	Python 3.12.13	Python 3.12.3	❌
Node.js	v24.16.0	v22.22.3	❌
Go	go1.22.12	go1.22.12	✅

Result: Not all tests passed — Python and Node.js versions differ between host and chroot.

Tested by Smoke Chroot

github-actions · 2026-06-17T16:54:26Z

Smoke Test Results — FAIL

Check	Result
Redis PING	❌ timeout (no response on `host.docker.internal:6379`)
PostgreSQL pg_isready	❌ `no response` on port 5432
PostgreSQL SELECT 1	❌ skipped (no connection)

Overall: FAIL — service containers are not reachable from this runner. host.docker.internal resolves to 172.17.0.1 but ports 6379 and 5432 are not responding.

🔌 Service connectivity validated by Smoke Services

github-actions · 2026-06-17T16:54:35Z

Smoke Test Results

GitHub MCP Testing: ✅ (Last 2 merged PRs: Fix duplicate-code-detector: Add missing GH_TOKEN for gh CLI authentication #5146, refactor(agent-service): extract resolveAgentImageConfig from buildAgentService #5129)
GitHub.com Connectivity: ❌ (Status: 000, Exit: 35)
File Writing Testing: ✅
Bash Tool Testing: ✅

Overall status: FAIL

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

localhost

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "localhost"

See Network Configuration for more information.

💎 Faceted by Smoke Gemini

github-actions · 2026-06-17T16:54:51Z

🏗️ Build Test Suite Results

Ecosystem	Project	Build/Install	Tests	Status
Bun	elysia	✅	1/1 passed	✅ PASS
Bun	hono	✅	1/1 passed	✅ PASS
C++	fmt	✅	N/A	✅ PASS
C++	json	✅	N/A	✅ PASS
Deno	oak	N/A	1/1 passed	✅ PASS
Deno	std	N/A	1/1 passed	✅ PASS
.NET	hello-world	✅	N/A	✅ PASS
.NET	json-parse	✅	N/A	✅ PASS
Go	color	✅	1/1 passed	✅ PASS
Go	env	✅	1/1 passed	✅ PASS
Go	uuid	✅	1/1 passed	✅ PASS
Java	gson	✅	1/1 passed	✅ PASS
Java	caffeine	✅	1/1 passed	✅ PASS
Node.js	clsx	✅	passed	✅ PASS
Node.js	execa	✅	passed	✅ PASS
Node.js	p-limit	✅	passed	✅ PASS
Rust	fd	✅	1/1 passed	✅ PASS
Rust	zoxide	✅	1/1 passed	✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Generated by Build Test Suite for issue #5165 · ◷

Initial plan

2e4eee5

Copilot AI assigned Copilot and lpcox Jun 17, 2026

Copilot started work on behalf of lpcox June 17, 2026 15:04 View session

Copilot AI changed the title ~~[WIP] Update max-ai-credits limit for CI/CD workflows~~ fix(ci-cd-gaps-assessment): resolve AI credits rate limit with DataOps refactor Jun 17, 2026

Copilot finished work on behalf of lpcox June 17, 2026 15:20

Copilot AI requested a review from lpcox June 17, 2026 15:20

lpcox marked this pull request as ready for review June 17, 2026 15:34

Copilot AI review requested due to automatic review settings June 17, 2026 15:34

Copilot started reviewing on behalf of lpcox June 17, 2026 15:35 View session

Copilot AI reviewed Jun 17, 2026

View reviewed changes

Copilot started work on behalf of lpcox June 17, 2026 15:54 View session

Copilot stopped work on behalf of lpcox due to an error June 17, 2026 15:55
Failed to fetch job details: HTTP 500 Internal Server Error

lpcox added the ready-for-aw label Jun 17, 2026

lpcox had a problem deploying to aoai-model June 17, 2026 16:21 — with GitHub Actions Failure

lpcox temporarily deployed to aoai-model June 17, 2026 16:21 — with GitHub Actions Inactive

lpcox temporarily deployed to aoai-model June 17, 2026 16:22 — with GitHub Actions Inactive

lpcox had a problem deploying to aoai-model June 17, 2026 16:48 — with GitHub Actions Failure

github-actions Bot added the smoke-copilot-byok-aoai-apikey label Jun 17, 2026

github-actions Bot added the smoke-copilot-pat label Jun 17, 2026

github-actions Bot added the smoke-codex label Jun 17, 2026

github-actions Bot added smoke-gemini build-test labels Jun 17, 2026

lpcox temporarily deployed to aoai-model June 17, 2026 17:04 — with GitHub Actions Inactive

lpcox temporarily deployed to aoai-model June 17, 2026 17:05 — with GitHub Actions Inactive

This was referenced Jun 17, 2026

[aw] Smoke Copilot BYOK AOAI (Entra) failed #5190

Closed

[aw] Smoke Copilot BYOK failed #5191

Closed

lpcox merged commit cf07064 into main Jun 17, 2026
122 of 143 checks passed

lpcox deleted the copilot/aw-increase-ai-credits-limit branch June 17, 2026 17:10

github-actions Bot mentioned this pull request Jun 17, 2026

fix(contribution-check): switch to gpt-5.4-mini and add cancel-in-progress concurrency #5163

Merged

Conversation

Copilot AI commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Uh oh!

github-actions Bot commented Jun 17, 2026

✅ Coverage Check Passed

Overall Coverage

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Copilot's findings

Uh oh!

lpcox commented Jun 17, 2026

Uh oh!

github-actions Bot commented Jun 17, 2026

Uh oh!

github-actions Bot commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 17, 2026

Uh oh!

github-actions Bot commented Jun 17, 2026

Uh oh!

github-actions Bot commented Jun 17, 2026

🔬 Smoke Test Results — PR #5165

Uh oh!

github-actions Bot commented Jun 17, 2026

🔬 Smoke Test: Copilot PAT Auth — PASS

Uh oh!

github-actions Bot commented Jun 17, 2026

Uh oh!

github-actions Bot commented Jun 17, 2026

Smoke Test: API Proxy OpenTelemetry Tracing

Uh oh!

github-actions Bot commented Jun 17, 2026

Chroot Version Comparison

Uh oh!

github-actions Bot commented Jun 17, 2026

Smoke Test Results — FAIL

Uh oh!

github-actions Bot commented Jun 17, 2026

Smoke Test Results

Copilot AI commented Jun 17, 2026 •

edited

Loading

github-actions Bot commented Jun 17, 2026 •

edited

Loading

github-actions Bot commented Jun 17, 2026 •

edited

Loading

github-actions Bot commented Jun 17, 2026 •

edited

Loading

github-actions Bot commented Jun 17, 2026 •

edited

Loading

github-actions Bot commented Jun 17, 2026 •

edited

Loading

github-actions Bot commented Jun 17, 2026 •

edited

Loading

github-actions Bot commented Jun 17, 2026 •

edited

Loading

github-actions Bot commented Jun 17, 2026 •

edited

Loading

github-actions Bot commented Jun 17, 2026 •

edited

Loading

github-actions Bot commented Jun 17, 2026 •

edited

Loading

github-actions Bot commented Jun 17, 2026 •

edited

Loading

github-actions Bot commented Jun 17, 2026 •

edited

Loading

github-actions Bot commented Jun 17, 2026 •

edited

Loading