Skip to content

feat: optimize config-consistency-auditor — early exit, trimmed toolset, leaner prompt - #6087

Merged
lpcox merged 3 commits into
mainfrom
copilot/optimize-copilot-token-usage
Jul 10, 2026
Merged

feat: optimize config-consistency-auditor — early exit, trimmed toolset, leaner prompt#6087
lpcox merged 3 commits into
mainfrom
copilot/optimize-copilot-token-usage

Conversation

Copilot AI commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

config-consistency-auditor was running at ~125.7 AIC/run regardless of whether any config-touching PRs were merged, making it the costliest workflow in the fleet. On most weekdays no relevant PRs exist, so the entire agent session was wasted.

Changes

Early exit before agent launch (~96% savings on no-PR days)

Added a fetch_prs pre-check job that filters merged PRs via jq against config-relevant paths. The agent job is gated with if: needs.fetch_prs.outputs.pr_count != '0' — skipping the full agent run entirely when nothing needs auditing.

if: needs.fetch_prs.outputs.pr_count != '0'
jobs:
  fetch_prs:
    outputs:
      pr_count: ${{ steps.filter.outputs.pr_count }}
    steps:
      - name: Filter config-touching merged PRs
        id: filter
        run: |
          gh pr list --state merged --limit 50 --json number,title,mergedAt,files > /tmp/all-prs.json
          PR_COUNT=$(jq --arg cutoff "$CUTOFF" '[.[] | select(.mergedAt >= $cutoff) | select(
            (.files // []) | map(.path) | any(test("src/config-file\\.ts|src/types/|..."))
          )] | length' /tmp/all-prs.json)
          echo "pr_count=$PR_COUNT" >> "$GITHUB_OUTPUT"

DataOps pre-step (reduces agent context on active days)

Added a steps: block in the agent job that pre-fetches the filtered PR list to /tmp/gh-aw/relevant-prs.json. Agent reads compact JSON instead of fetching and filtering via tool calls.

Network and toolset trimming (~13–22K AIC/run on active days)

  • Removed node from network.allowed — not used by this workflow
  • toolsets: [default, pull_requests]toolsets: [pull_requests] — removes ~15 unused tool schemas (~7.5–10.5K tokens/turn)

Prompt reduction (~36% smaller, ~4.3 KB from 6.7 KB)

  • Security classification condensed from 8 lines to 3
  • Step 2 now references pre-fetched file; removed redundant path filter list
  • Removed embedded 17-line example PR body template from Step 6
  • Step 5 cross-references the classification section instead of duplicating it

- Add fetch_prs pre-check job for early exit on days with no config-touching PRs
- Add agent-level if: condition gated on fetch_prs.outputs.pr_count != '0'
- Add steps: block to pre-fetch filtered PR list to /tmp/gh-aw/relevant-prs.json
- Remove node network group (unused by this workflow)
- Switch toolsets from [default, pull_requests] to [pull_requests] only
- Trim prompt: condense security classification to 3-line summary
- Trim prompt: update Step 2 to reference pre-fetched data
- Trim prompt: remove embedded example PR body template (~17 lines)
- Trim prompt: condense Step 5 security check (cross-references classification section)
- Recompile: regenerate config-consistency-auditor.lock.yml

Closes #6086
Copilot AI changed the title [WIP] Optimize Copilot token usage in config consistency auditor feat: optimize config-consistency-auditor — early exit, trimmed toolset, leaner prompt Jul 10, 2026
Copilot finished work on behalf of lpcox July 10, 2026 11:35
Copilot AI requested a review from lpcox July 10, 2026 11:35
@lpcox
lpcox marked this pull request as ready for review July 10, 2026 11:36
Copilot AI review requested due to automatic review settings July 10, 2026 11:36

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the config-consistency-auditor agentic workflow to reduce unnecessary runs and token/context usage by skipping the agent session when no relevant PRs were merged and by trimming the toolset and prompt.

Changes:

  • Add a fetch_prs pre-check job and gate the agent job on pr_count != '0' to early-exit on no-PR days.
  • Pre-fetch and persist a filtered “relevant merged PRs” list for the agent to read, reducing tool calls/context on active days.
  • Trim workflow toolsets and reduce prompt verbosity to lower AIC/tokens per run.
Show a summary per file
File Description
.github/workflows/config-consistency-auditor.md Adds pre-check job + gating, pre-fetch step, and prompt/toolset/network trimming.
.github/workflows/config-consistency-auditor.lock.yml Regenerated lock workflow reflecting the updated spec (but currently includes unintended toolchain/template regressions vs repo norms).
.github/aw/actions-lock.json Updates action pin set used by agentic workflow compilation/locking.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 3/3 changed files
  • Comments generated: 4
  • Review effort level: Low

Comment on lines +1 to +3
# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"d34d9401edb2d47424af7074d4f35629ac2f26b34e7da57db6e3555c1a390613","body_hash":"4396e8f842fb7a13647d33cdb2a1c614e2a244d7e1750e601e3d61ce12ca4800","compiler_version":"v0.81.6","strict":true,"agent_id":"copilot","engine_versions":{"copilot":"1.0.65"}}
# gh-aw-manifest: {"version":1,"secrets":["COPILOT_GITHUB_TOKEN","GH_AW_CI_TRIGGER_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/cache","sha":"27d5ce7f107fe9357f9df03efb73ab90386fccae","version":"v5.0.5"},{"repo":"actions/cache/restore","sha":"27d5ce7f107fe9357f9df03efb73ab90386fccae","version":"v5.0.5"},{"repo":"actions/cache/restore","sha":"55cc8345863c7cc4c66a329aec7e433d2d1c52a9","version":"v6.1.0"},{"repo":"actions/cache/save","sha":"55cc8345863c7cc4c66a329aec7e433d2d1c52a9","version":"v6.1.0"},{"repo":"actions/checkout","sha":"9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0","version":"v7.0.0"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"v0.81.6","version":"v0.81.6"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.11","digest":"sha256:979723c628182da7729333f2208bb249fd25ddee579645cf9a3892d681a929c7","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.11@sha256:979723c628182da7729333f2208bb249fd25ddee579645cf9a3892d681a929c7"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.11","digest":"sha256:807e4831999b44513b0a66e5859d478dc4da7ae74ab1918cec967d513f95bf9d","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.11@sha256:807e4831999b44513b0a66e5859d478dc4da7ae74ab1918cec967d513f95bf9d"},{"image":"ghcr.io/github/gh-aw-firewall/cli-proxy:0.27.11"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.11","digest":"sha256:ff27ea0525ad953a6adee28a5fbe9d2e22be47dbec755c15767af4ea3f91df7d","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.11@sha256:ff27ea0525ad953a6adee28a5fbe9d2e22be47dbec755c15767af4ea3f91df7d"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.30","digest":"sha256:35625d1a2269b1238606078c879f59a91cffc4ac33eb54bf39c6418822c1a8be","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.30@sha256:35625d1a2269b1238606078c879f59a91cffc4ac33eb54bf39c6418822c1a8be"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b","pinned_image":"ghcr.io/github/gh-aw-node@sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b"},{"image":"ghcr.io/github/github-mcp-server:v1.4.0","digest":"sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036","pinned_image":"ghcr.io/github/github-mcp-server:v1.4.0@sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036"}]}
# This file was automatically generated by gh-aw (v0.81.6). DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md
Comment on lines 100 to 104
- name: Setup Scripts
id: setup
uses: github/gh-aw-actions/setup@bf7ba42ce6443bf79fa184c9c6a35de202690bfc # v0.82.7
uses: github/gh-aw-actions/setup@v0.81.6
with:
destination: ${{ runner.temp }}/gh-aw/actions
Comment on lines +807 to +828
@@ -830,9 +823,9 @@
GH_AW_TOOL_CACHE_MOUNT="$GH_AW_TOOL_CACHE:$GH_AW_TOOL_CACHE:ro"
fi
fi
# shellcheck disable=SC1003,SC2016,SC2086
awf --config "${RUNNER_TEMP}/gh-aw/awf-config.json" --container-workdir "${GITHUB_WORKSPACE}" --mount "${RUNNER_TEMP}/gh-aw:${RUNNER_TEMP}/gh-aw:ro" --mount "${RUNNER_TEMP}/gh-aw:/host${RUNNER_TEMP}/gh-aw:ro" ${GH_AW_TOOL_CACHE_MOUNT:+--mount "$GH_AW_TOOL_CACHE_MOUNT"} ${GH_AW_DOCKER_HOST:+--docker-host "$GH_AW_DOCKER_HOST"} --env-all --exclude-env COPILOT_GITHUB_TOKEN --exclude-env GH_TOKEN --exclude-env GITHUB_MCP_SERVER_TOKEN --exclude-env MCP_GATEWAY_API_KEY --log-level info --skip-pull --difc-proxy-host awmg-cli-proxy:18443 --difc-proxy-ca-cert /tmp/gh-aw/difc-proxy-tls/ca.crt \
-- /bin/bash -c 'set +o histexpand; export PATH="${RUNNER_TEMP}/gh-aw/mcp-cli/bin:$PATH" && : "${RUNNER_TOOL_CACHE:?RUNNER_TOOL_CACHE must be set}"; GH_AW_TOOL_CACHE="$RUNNER_TOOL_CACHE"; export PATH="$(find "$GH_AW_TOOL_CACHE" -maxdepth 5 -type d -name bin 2>/dev/null | tr '\''\n'\'' '\'':'\'')$PATH"; [ -n "$GOROOT" ] && export PATH="$GOROOT/bin:$PATH" || true; [ -n "$ERLANG_HOME" ] && export PATH="$ERLANG_HOME/bin:$PATH" || true && GH_AW_NODE_EXEC="${GH_AW_NODE_BIN:-}"; if [ -z "$GH_AW_NODE_EXEC" ] || [ ! -x "$GH_AW_NODE_EXEC" ]; then GH_AW_NODE_EXEC="$(command -v node 2>/dev/null || true)"; fi; if [ -z "$GH_AW_NODE_EXEC" ]; then echo "node runtime missing on this runner — check runtimes.node in workflow YAML" >&2; exit 127; fi; GH_AW_NPM_GLOBAL_ROOT="$(npm root -g 2>/dev/null || true)"; if [ -n "$GH_AW_NPM_GLOBAL_ROOT" ]; then export NODE_PATH="${GH_AW_NPM_GLOBAL_ROOT}${NODE_PATH:+:${NODE_PATH}}"; fi; "$GH_AW_NODE_EXEC" ${RUNNER_TEMP}/gh-aw/actions/copilot_harness.cjs /usr/local/bin/copilot --add-dir /tmp/gh-aw/ --log-level all --log-dir /tmp/gh-aw/sandbox/agent/logs/ --disable-builtin-mcps --no-ask-user --allow-all-tools --add-dir /tmp/gh-aw/cache-memory/ --allow-all-paths --add-dir "${GITHUB_WORKSPACE}" --prompt-file /tmp/gh-aw/aw-prompts/prompt.txt' 2>&1 | tee -a /tmp/gh-aw/agent-stdio.log
# shellcheck disable=SC1003,SC2086
sudo -E awf --config "${RUNNER_TEMP}/gh-aw/awf-config.json" --container-workdir "${GITHUB_WORKSPACE}" --mount "${RUNNER_TEMP}/gh-aw:${RUNNER_TEMP}/gh-aw:ro" --mount "${RUNNER_TEMP}/gh-aw:/host${RUNNER_TEMP}/gh-aw:ro" ${GH_AW_TOOL_CACHE_MOUNT:+--mount "$GH_AW_TOOL_CACHE_MOUNT"} ${GH_AW_DOCKER_HOST:+--docker-host "$GH_AW_DOCKER_HOST"} ${GH_AW_DOCKER_HOST_PATH_PREFIX_ARGS} --env-all --exclude-env COPILOT_GITHUB_TOKEN --exclude-env GH_TOKEN --exclude-env GITHUB_MCP_SERVER_TOKEN --exclude-env MCP_GATEWAY_API_KEY --log-level info --proxy-logs-dir /tmp/gh-aw/sandbox/firewall/logs --audit-dir /tmp/gh-aw/sandbox/firewall/audit --enable-host-access --allow-host-ports 80,443,8080 --skip-pull --difc-proxy-host host.docker.internal:18443 --difc-proxy-ca-cert /tmp/gh-aw/difc-proxy-tls/ca.crt \
-- /bin/bash -c 'set +o histexpand; export PATH="${RUNNER_TEMP}/gh-aw/mcp-cli/bin:$PATH" && : "${RUNNER_TOOL_CACHE:?RUNNER_TOOL_CACHE must be set}"; GH_AW_TOOL_CACHE="$RUNNER_TOOL_CACHE"; export PATH="$(find "$GH_AW_TOOL_CACHE" -maxdepth 5 -type d -name bin 2>/dev/null | tr '\''\n'\'' '\'':'\'')$PATH"; [ -n "$GOROOT" ] && export PATH="$GOROOT/bin:$PATH" || true && GH_AW_NODE_EXEC="${GH_AW_NODE_BIN:-}"; if [ -z "$GH_AW_NODE_EXEC" ] || [ ! -x "$GH_AW_NODE_EXEC" ]; then GH_AW_NODE_EXEC="$(command -v node 2>/dev/null || true)"; fi; if [ -z "$GH_AW_NODE_EXEC" ]; then echo "node runtime missing on this runner — check runtimes.node in workflow YAML" >&2; exit 127; fi; GH_AW_NPM_GLOBAL_ROOT="$(npm root -g 2>/dev/null || true)"; if [ -n "$GH_AW_NPM_GLOBAL_ROOT" ]; then export NODE_PATH="${GH_AW_NPM_GLOBAL_ROOT}${NODE_PATH:+:${NODE_PATH}}"; fi; "$GH_AW_NODE_EXEC" ${RUNNER_TEMP}/gh-aw/actions/copilot_harness.cjs /usr/local/bin/copilot --add-dir /tmp/gh-aw/ --log-level all --log-dir /tmp/gh-aw/sandbox/agent/logs/ --disable-builtin-mcps --no-ask-user --allow-all-tools --add-dir /tmp/gh-aw/cache-memory/ --allow-all-paths --add-dir "${GITHUB_WORKSPACE}" --prompt-file /tmp/gh-aw/aw-prompts/prompt.txt' 2>&1 | tee -a /tmp/gh-aw/agent-stdio.log
Comment on lines 63 to 67
"astral-sh/setup-uv@v8.3.2": {
"repo": "astral-sh/setup-uv",
"version": "v8.3.2",
"sha": "11f9893b081a58869d3b5fccaea48c9e9e46f990"
},
"github/gh-aw-actions/setup-cli@v0.82.7": {
"repo": "github/gh-aw-actions/setup-cli",
"version": "v0.82.7",
"sha": "bf7ba42ce6443bf79fa184c9c6a35de202690bfc"
},
"github/gh-aw-actions/setup@v0.82.7": {
"repo": "github/gh-aw-actions/setup",
"version": "v0.82.7",
"sha": "bf7ba42ce6443bf79fa184c9c6a35de202690bfc"
}
@github-actions

github-actions Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

✅ Copilot review passed with no inline comments.

@copilot Add the ready-for-aw label to this PR to trigger agentic CI smoke tests.

@lpcox

lpcox commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

@copilot discard all updates except those to the .md file

@github-actions

Copy link
Copy Markdown
Contributor

✅ Coverage Check Passed

Overall Coverage

Metric Base PR Delta
Lines 99.17% 99.21% 📈 +0.04%
Statements 99.13% 99.17% 📈 +0.04%
Functions 99.45% 99.45% ➡️ +0.00%
Branches 95.75% 95.75% ➡️ +0.00%
📁 Per-file Coverage Changes (1 files)
File Lines (Before → After) Statements (Before → After)
src/log-directory-setup.ts 96.2% → 100.0% (+3.78%) 96.3% → 100.0% (+3.71%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

Copilot AI commented Jul 10, 2026

Copy link
Copy Markdown
Contributor Author

@copilot discard all updates except those to the .md file

Done. Reverted .github/workflows/config-consistency-auditor.lock.yml and .github/aw/actions-lock.json to their pre-PR state. Only the .md changes remain (commit revert: discard lock.yml and actions-lock.json changes, keep .md edits only).

Copilot finished work on behalf of lpcox July 10, 2026 12:13
@lpcox
lpcox merged commit 2265b4a into main Jul 10, 2026
15 checks passed
@lpcox
lpcox deleted the copilot/optimize-copilot-token-usage branch July 10, 2026 13:02
lpcox added a commit that referenced this pull request Jul 10, 2026
* chore: recompile workflows after #6087

Recompile config-consistency-auditor.lock.yml after merging
feat: optimize config-consistency-auditor.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* chore: revert smoke-gvisor.lock.yml changes

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

⚡ Copilot Token Optimization2026-07-10 — Config Consistency Auditor

3 participants