Skip to content

Reduce test-coverage-reporter prompt surface and coverage payload size#4871

Merged
lpcox merged 2 commits into
mainfrom
copilot/optimize-token-usage
Jun 13, 2026
Merged

Reduce test-coverage-reporter prompt surface and coverage payload size#4871
lpcox merged 2 commits into
mainfrom
copilot/optimize-token-usage

Conversation

Copilot AI commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

test-coverage-reporter was spending tokens on unused tool injection and oversized precomputed coverage inputs. This tightens the workflow around its actual job: consume step outputs and post a discussion via safeoutputs.

  • Remove unused agent tools

    • Disable GitHub MCP injection with github: false
    • Disable bash tool injection with bash: false
    • Result: the compiled lock no longer includes the GitHub MCP server/toolset prompt surface
  • Trim precomputed coverage context

    • Strip the repeated workspace prefix from COVERAGE_JSON entries
    • Filter COVERAGE_TABLE to actionable files only:
      • files below 80% statement coverage
      • security-critical files (docker-manager, host-iptables, squid-config, domain-patterns, cli)
  • Lockfile + regression coverage

    • Recompile test-coverage-reporter.lock.yml
    • Add a focused workflow test that asserts:
      • source config disables GitHub/bash tools
      • compiled lock excludes GitHub MCP artifacts
      • coverage payload filtering remains present
  • Illustrative change

tools:
  github: false
  bash: false
const filtered = Object.fromEntries(
  Object.entries(d)
    .filter(([k, v]) => k === 'total' || (v.statements && v.statements.pct < 80))
    .map(([k, v]) => [k === 'total' ? k : k.replace(process.cwd() + '/', ''), v])
);

const SECURITY_CRITICAL = ['docker-manager', 'host-iptables', 'squid-config', 'domain-patterns', 'cli'];

const rows = Object.entries(d)
  .filter(([k]) => k !== 'total')
  .map(([k, v]) => ({ file: k.replace(process.cwd() + '/', ''), stmts: v.statements.pct, branch: v.branches.pct, funcs: v.functions.pct, lines: v.lines.pct }))
  .filter(r => r.stmts < 80 || SECURITY_CRITICAL.some(s => r.file.includes(s)));

Copilot AI changed the title [WIP] Optimize Copilot token usage for test coverage reporting Reduce test-coverage-reporter prompt surface and coverage payload size Jun 13, 2026
Copilot finished work on behalf of lpcox June 13, 2026 14:45
Copilot AI requested a review from lpcox June 13, 2026 14:45
@lpcox lpcox marked this pull request as ready for review June 13, 2026 15:40
Copilot AI review requested due to automatic review settings June 13, 2026 15:40
@github-actions

Copy link
Copy Markdown
Contributor

✅ Coverage Check Passed

Overall Coverage

Metric Base PR Delta
Lines 96.60% 96.64% 📈 +0.04%
Statements 96.47% 96.51% 📈 +0.04%
Functions 98.78% 98.78% ➡️ +0.00%
Branches 91.22% 91.25% 📈 +0.03%
📁 Per-file Coverage Changes (1 files)
File Lines (Before → After) Statements (Before → After)
src/config-writer.ts 89.9% → 91.1% (+1.19%) 89.9% → 91.1% (+1.19%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR tightens the test-coverage-reporter agent workflow to reduce prompt/tool injection surface and shrink the precomputed coverage context it feeds to the agent, while adding a regression test to ensure these optimizations remain in place.

Changes:

  • Disables GitHub MCP and bash tool injection in the source workflow (tools.github: false, tools.bash: false) and ensures the compiled lock no longer contains GitHub MCP artifacts.
  • Reduces coverage payload size by (1) stripping the workspace prefix from COVERAGE_JSON keys and (2) filtering COVERAGE_TABLE rows to <80% statement coverage plus a small set of security-critical path substrings.
  • Adds a Jest workflow regression test to assert the source + lock continue to reflect the intended tool disabling and coverage filtering.
Show a summary per file
File Description
scripts/ci/test-coverage-reporter-workflow.test.ts Adds a regression test asserting tool injection removal and coverage payload trimming are present in source + lock workflows.
.github/workflows/test-coverage-reporter.md Disables GitHub/bash agent tools and trims precomputed coverage JSON/table context.
.github/workflows/test-coverage-reporter.lock.yml Recompiled lock reflecting removal of GitHub MCP injection and inclusion of the new coverage filtering logic.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 3/3 changed files
  • Comments generated: 0

@github-actions

Copy link
Copy Markdown
Contributor

🔍 Smoke Test: Copilot PAT Auth — Results

PR: "Reduce test-coverage-reporter prompt surface and coverage payload size"
Author: @Copilot | Assignees: @lpcox, @Copilot
Auth mode: PAT (COPILOT_GITHUB_TOKEN)

Test Result
GitHub MCP connectivity
GitHub.com HTTP (200)
File write/read ❌ (pre-step template vars not substituted)

Overall: FAIL — file test unverifiable due to unresolved ${{ steps.smoke-data.outputs.* }} expressions.

🔑 PAT report filed by Smoke Copilot PAT

@github-actions

Copy link
Copy Markdown
Contributor

🔬 Smoke Test Results

Test Status
GitHub MCP (list PRs)
GitHub.com HTTP connectivity ❌ (pre-step data not expanded)
File write/read ❌ (pre-step data not expanded)

PR: Reduce test-coverage-reporter prompt surface and coverage payload size
Author: @Copilot | Assignees: @lpcox @Copilot

Overall: ❌ FAIL

Tests 2 & 3 could not be verified — steps.smoke-data.outputs variables were not substituted by the workflow runner.

📰 BREAKING: Report filed by Smoke Copilot

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Copilot BYOK (Direct) Mode — ✅ PASS

Test Results:

  • ✅ GitHub MCP connectivity (2 PRs fetched)
  • ✅ GitHub.com connectivity (HTTP 200)
  • ✅ File write/read test (/tmp/gh-aw/agent/smoke-test-copilot-byok.txt)
  • ✅ BYOK inference via api-proxy sidecar

Recent PRs (for context):

Configuration: Direct BYOK mode (COPILOT_PROVIDER_API_KEY) → api-proxy sidecar → api.githubcopilot.com

Authors/Assignees: @lpcox, @Copilot


All smoke tests passed. Running in direct BYOK inference path.

🔑 BYOK report filed by Smoke Copilot BYOK

@github-actions

Copy link
Copy Markdown
Contributor

Titles

  • Reduce test-coverage-reporter prompt surface and coverage payload size by Copilot
  • feat(api-proxy): implement OTLP fan-out to multiple endpoints
  • fix(docker-host): pass through loopback TCP DOCKER_HOST for ARC/DinD orchestration

Results

  • GitHub read check: ✅
  • Playwright GitHub title check: ✅
  • Temp file write/read: ✅
  • Build (npm ci && npm run build): ✅
  • Discussion lookup/comment: ✅

Overall

PASS

🔮 The oracle has spoken through Smoke Codex

@github-actions

Copy link
Copy Markdown
Contributor

🏗️ Build Test Suite Results

Ecosystem Project Build/Install Tests Status
Bun elysia 1/1 passed ✅ PASS
Bun hono 1/1 passed ✅ PASS
C++ fmt N/A ✅ PASS
C++ json N/A ✅ PASS
Deno oak N/A 1/1 passed ✅ PASS
Deno std N/A 1/1 passed ✅ PASS
.NET hello-world N/A ✅ PASS
.NET json-parse N/A ✅ PASS
Go color passed ✅ PASS
Go env passed ✅ PASS
Go uuid passed ✅ PASS
Java gson 1/1 passed ✅ PASS
Java caffeine 1/1 passed ✅ PASS
Node.js clsx passed ✅ PASS
Node.js execa passed ✅ PASS
Node.js p-limit passed ✅ PASS
Rust fd 1/1 passed ✅ PASS
Rust zoxide 1/1 passed ✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Generated by Build Test Suite for issue #4871 ·

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: GitHub Actions Services Connectivity

Check Result
Redis PING ❌ Timeout/no response
PostgreSQL pg_isready ❌ No response
PostgreSQL SELECT 1 ❌ Timeout/error

Overall: FAILhost.docker.internal service containers unreachable from this runner.

🔌 Service connectivity validated by Smoke Services

@github-actions

Copy link
Copy Markdown
Contributor

@Copilot @lpcox
GitHub MCP Testing: ✅
GitHub.com Connectivity: ✅
File Write/Read Test: ✅
BYOK Inference Test: ✅
Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw)
Overall: PASS

🔑 BYOK (AOAI api-key) report filed by Smoke Copilot BYOK AOAI (api-key)

@lpcox lpcox merged commit e724a39 into main Jun 13, 2026
74 of 79 checks passed
@lpcox lpcox deleted the copilot/optimize-token-usage branch June 13, 2026 16:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants