refactor: deduplicate BYOK COPILOT_MODEL test scaffolding by Copilot · Pull Request #5661 · github/gh-aw-firewall

Copilot · 2026-06-28T23:31:54Z

Two BYOK COPILOT_MODEL tests in config-assembly-model-detection.test.ts duplicated the full arrange/act/assert scaffold — differing only in whether copilotProviderBaseUrl came from direct config or an env file.

Changes

Extracted expectByokModelAllowed(buildConfigOverrides) helper inside the describe block, following the existing runCopilotModelDetectionCase pattern already present in the file
Each test now handles only its unique setup (direct URL vs. env file write) and delegates the rest to the helper

function expectByokModelAllowed(buildConfigOverrides: Record<string, unknown>): void {
  mockBuildConfigOnce({
    copilotProviderApiKey: 'byok-api-key-for-azure-foundry',
    additionalEnv: { COPILOT_MODEL: 'o4-mini-aw' },
    ...buildConfigOverrides,
  });
  // ... assembleAndValidateConfig + assertions
}

it('should allow custom COPILOT_MODEL values in BYOK mode with a provider base URL', () => {
  expectByokModelAllowed({ copilotProviderBaseUrl: '...' });
});

it('should allow custom COPILOT_MODEL values when provider base URL is set via env file', () => {
  const envFilePath = path.join(getTestDir(), 'byok.env');
  fs.writeFileSync(envFilePath, 'COPILOT_PROVIDER_BASE_URL=...\n');
  expectByokModelAllowed({ envFile: envFilePath });
});

Copilot

Pull request overview

This PR refactors config-assembly-model-detection.test.ts to reduce duplicated BYOK COPILOT_MODEL test scaffolding by extracting a shared helper for the common arrange/act/assert sequence, while keeping each test’s unique setup (direct config vs env file) local to the test case.

Changes:

Extracted a local expectByokModelAllowed(buildConfigOverrides) helper to consolidate repeated BYOK configuration + assertions.
Updated the two BYOK COPILOT_MODEL tests to delegate common logic to the helper, differing only in how copilotProviderBaseUrl is provided (direct override vs env file).

Show a summary per file

File	Description
src/commands/validators/config-assembly-model-detection.test.ts	Deduplicates BYOK `COPILOT_MODEL` test setup via a shared helper while preserving test-specific setup paths.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 1/1 changed files
Comments generated: 0
Review effort level: Low

github-actions · 2026-06-28T23:50:43Z

✅ Copilot review passed with no inline comments.

@copilot Add the ready-for-aw label to this PR to trigger agentic CI smoke tests.

github-actions · 2026-06-28T23:51:15Z

Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.

github-actions · 2026-06-28T23:51:16Z

🔑 Smoke Copilot PAT PAT auth validated. All systems operational. ✅

github-actions · 2026-06-28T23:51:16Z

🚀 Security Guard has started processing this pull request

github-actions · 2026-06-28T23:51:16Z

📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅

github-actions · 2026-06-28T23:51:17Z

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

github-actions · 2026-06-28T23:51:19Z

✅ Smoke Copilot BYOK AOAI (api-key) completed. Copilot AOAI BYOK (api-key) mode operational. 🔓

github-actions · 2026-06-28T23:51:19Z

✅ Smoke Gemini completed. All facets verified. 💎

github-actions · 2026-06-28T23:51:19Z

✅ Smoke Claude passed

github-actions · 2026-06-28T23:51:19Z

✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓

github-actions · 2026-06-28T23:51:21Z

✅ Contribution Check completed successfully!

Contribution check complete: PR #5661 follows the applicable CONTRIBUTING.md guidelines; no comment needed.

github-actions · 2026-06-28T23:51:21Z

✅ Build Test Suite completed successfully!

github-actions · 2026-06-28T23:51:23Z

✅ Smoke Copilot BYOK AOAI (Entra) completed. Copilot AOAI BYOK (Entra) mode operational. 🔓

Smoke test summary comment and label applied; no further action needed

github-actions · 2026-06-28T23:51:26Z

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

github-actions · 2026-06-28T23:51:44Z

🔌 Smoke Services — All services reachable! ✅

github-actions · 2026-06-28T23:52:25Z

✅ Coverage Check Passed

Overall Coverage

Metric	Base	PR	Delta
Lines	98.16%	98.20%	📈 +0.04%
Statements	98.10%	98.13%	📈 +0.03%
Functions	99.54%	99.54%	➡️ +0.00%
Branches	94.14%	94.14%	➡️ +0.00%

📁 Per-file Coverage Changes (1 files)

File	Lines (Before → After)	Statements (Before → After)
`src/workdir-setup.ts`	92.7% → 94.5% (+1.82%)	92.7% → 94.5% (+1.82%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

github-actions · 2026-06-28T23:54:34Z

✅ Smoke Test: Copilot BYOK (Direct) Mode - PASS

All tests verified:

✅ GitHub MCP connectivity
✅ GitHub.com HTTP 200
✅ File I/O operations
✅ Direct BYOK inference (agent → api-proxy → api.githubcopilot.com)

Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY) via api-proxy sidecar.

🔑 BYOK report filed by Smoke Copilot BYOK

github-actions · 2026-06-28T23:54:41Z

🔬 Smoke Test Results

Test	Result
GitHub MCP connectivity	✅
GitHub.com HTTP	✅ 200
File write/read	❌ pre-step data unavailable (template vars not expanded)

Overall: FAIL

PR: refactor: deduplicate BYOK COPILOT_MODEL test scaffolding
Author: @Copilot | Assignees: @lpcox @Copilot

📰 BREAKING: Report filed by Smoke Copilot

github-actions · 2026-06-28T23:54:48Z

Smoke Test: Claude Engine Validation

API check: ✅ PASS
gh check: ✅ PASS
File check: ✅ PASS

Overall result: PASS

Generated by Smoke Claude for #5661 · 36.1 AIC · ⊞ 3.3K · ◷

github-actions · 2026-06-28T23:55:05Z

🧪 Smoke Test Results

Test	Result
GitHub MCP connectivity	✅
GitHub.com HTTP	✅ 200
File write/read	⚠️ template vars unresolved

Overall: PASS (verifiable tests passed)
PR: refactor: deduplicate BYOK COPILOT_MODEL test scaffolding
@lpcox @Copilot — Auth mode: PAT (COPILOT_GITHUB_TOKEN)

🔑 PAT report filed by Smoke Copilot PAT

github-actions · 2026-06-28T23:55:27Z

Smoke Test: API Proxy OpenTelemetry Tracing

Scenario	Status	Details
Module Loading	✅	`otel.js` loads; exports `startRequestSpan`, `setTokenAttributes`, `setBudgetAttributes`, `endSpan`, `endSpanError`, `shutdown`, `isEnabled`
Test Suite	✅	59/59 tests passed across `otel.test.js` + `otel-fanout.test.js`
Env Var Forwarding	✅	`api-proxy-env-config.ts` forwards `GH_AW_OTLP_ENDPOINTS`, `OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_EXPORTER_OTLP_HEADERS`, `GITHUB_AW_OTEL_TRACE_ID`, `GITHUB_AW_OTEL_PARENT_SPAN_ID`, `OTEL_SERVICE_NAME`
Token Tracker Integration	✅	`onUsage` callback present in `token-tracker-http.js` (line 283, invoked at line 324)
OTEL Diagnostics	i️	Container not running during smoke test; provider initialized with file fallback (`/var/log/api-proxy/otel.jsonl`); no OTLP spans exported (expected)

All scenarios pass. OTEL tracing integration is functional.

📡 OTel tracing validated by Smoke OTel Tracing

github-actions · 2026-06-28T23:56:01Z

Chroot Version Comparison Results

Runtime	Host Version	Chroot Version	Match?
Python	Python 3.12.13	Python 3.12.3	❌ NO
Node.js	v24.17.0	v22.23.0	❌ NO
Go	go1.22.12	go1.22.12	✅ YES

Overall: ❌ Not all versions match — Python and Node.js versions differ between host and chroot environments.

Tested by Smoke Chroot

github-actions · 2026-06-28T23:56:07Z

refactor: deduplicate BYOK COPILOT_MODEL test scaffolding by Copilot
fix: handle EACCES during chroot-home cleanup in rootless Docker
docs(runner-doctor): add C7 failure mode for DIFC probe on *.ghe.com
✅ GitHub reads
✅ Playwright title
✅ File write
✅ Discussion comment
✅ Build
Overall: PASS

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex

github-actions · 2026-06-28T23:57:07Z

Smoke Test Results: Gemini Engine

GitHub MCP Testing: ✅
GitHub.com Connectivity: ✅ (301)
File Writing Testing: ✅
Bash Tool Testing: ✅

PRs Reviewed:

fix: handle EACCES during chroot-home cleanup in rootless Docker
docs(runner-doctor): add C7 failure mode for DIFC probe on *.ghe.com

Overall Status: PASS

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

localhost

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "localhost"

See Network Configuration for more information.

💎 Faceted by Smoke Gemini

github-actions · 2026-06-28T23:57:19Z

@lpcox

MCP connectivity via GitHub MCP tool: ✅
GitHub.com connectivity: ✅
File write/read test: ❌
BYOK inference test: ✅

Running in direct BYOK mode (AWF_AUTH_TYPE=github-oidc + AWF_AUTH_AZURE_* + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw) authenticated via Microsoft Entra

Overall: FAIL

🪪 BYOK (AOAI Entra) report filed by Smoke Copilot BYOK AOAI (Entra)

github-actions · 2026-06-28T23:57:49Z

@Copilot @lpcox

GitHub MCP connectivity: ✅
GitHub.com connectivity: ✅
File write/read test: ✅
BYOK inference test: ✅
Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw)
Overall: PASS

🔑 BYOK (AOAI api-key) report filed by Smoke Copilot BYOK AOAI (api-key)

github-actions · 2026-06-28T23:58:19Z

Smoke Test: GitHub Actions Services Connectivity

Check	Result
Redis PING	❌ Connection timeout
PostgreSQL pg_isready	❌ No response
PostgreSQL SELECT 1	❌ No response

Overall: FAIL

host.docker.internal resolves to 172.17.0.1 but all connections timed out — service containers appear unreachable from this runner environment.

🔌 Service connectivity validated by Smoke Services

github-actions · 2026-06-28T23:59:49Z

🏗️ Build Test Suite Results

Ecosystem	Project	Build/Install	Tests	Status
Bun	elysia	✅	1/1 passed	✅ PASS
Bun	hono	✅	1/1 passed	✅ PASS
C++	fmt	✅	N/A	✅ PASS
C++	json	✅	N/A	✅ PASS
Deno	oak	N/A	1/1 passed	✅ PASS
Deno	std	N/A	1/1 passed	✅ PASS
.NET	hello-world	✅	N/A	✅ PASS
.NET	json-parse	✅	N/A	✅ PASS
Go	color	✅	passed	✅ PASS
Go	env	✅	passed	✅ PASS
Go	uuid	✅	passed	✅ PASS
Java	gson	✅	1/1 passed	✅ PASS
Java	caffeine	✅	1/1 passed	✅ PASS
Node.js	clsx	✅	passed	✅ PASS
Node.js	execa	✅	passed	✅ PASS
Node.js	p-limit	✅	passed	✅ PASS
Rust	fd	✅	1/1 passed	✅ PASS
Rust	zoxide	✅	1/1 passed	✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Generated by Build Test Suite for #5661 · 77.3 AIC · ⊞ 7.8K · ◷

Initial plan

b7b35bd

Copilot AI assigned Copilot and lpcox Jun 28, 2026

Copilot started work on behalf of lpcox June 28, 2026 23:32 View session

refactor: extract shared BYOK model helper in detection tests

b5de0ed

Copilot AI changed the title ~~[WIP] Refactor BYOK COPILOT_MODEL tests to eliminate duplicate code~~ refactor: deduplicate BYOK COPILOT_MODEL test scaffolding Jun 28, 2026

Copilot AI requested a review from lpcox June 28, 2026 23:34

Copilot finished work on behalf of lpcox June 28, 2026 23:34

lpcox marked this pull request as ready for review June 28, 2026 23:39

Copilot AI review requested due to automatic review settings June 28, 2026 23:39

Copilot started reviewing on behalf of lpcox June 28, 2026 23:39 View session

Copilot AI reviewed Jun 28, 2026

View reviewed changes

lpcox added the ready-for-aw label Jun 28, 2026

lpcox temporarily deployed to aoai-model June 28, 2026 23:51 — with GitHub Actions Inactive

github-actions Bot added the smoke-copilot-byok label Jun 28, 2026

github-actions Bot mentioned this pull request Jun 28, 2026

[aw] No-Op Runs #5368

Open

github-actions Bot added the smoke-claude label Jun 28, 2026

github-actions Bot added the smoke-copilot-pat label Jun 28, 2026

github-actions Bot added the smoke-codex label Jun 28, 2026

lpcox temporarily deployed to aoai-model June 28, 2026 23:56 — with GitHub Actions Inactive

github-actions Bot added the smoke-gemini label Jun 28, 2026

github-actions Bot added the smoke-copilot-byok-aoai-entra label Jun 28, 2026

lpcox temporarily deployed to aoai-model June 28, 2026 23:57 — with GitHub Actions Inactive

github-actions Bot added the smoke-copilot-byok-aoai-apikey label Jun 28, 2026

github-actions Bot added the build-test label Jun 28, 2026

lpcox merged commit d9b9f69 into main Jun 29, 2026
91 checks passed

lpcox deleted the copilot/fix-duplicate-code-in-tests branch June 29, 2026 02:55

This was referenced Jun 29, 2026

chore: upgrade gh-aw to v0.81.6 pre-release #5668

Merged

[Test Coverage] add branch coverage for ssl-key-storage and config-writer #5667

Merged

Uh oh!

Conversation

Copilot AI commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Review details

Uh oh!

github-actions Bot commented Jun 28, 2026

Uh oh!

github-actions Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 28, 2026

Uh oh!

github-actions Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 28, 2026

✅ Coverage Check Passed

Overall Coverage

Uh oh!

github-actions Bot commented Jun 28, 2026

Uh oh!

github-actions Bot commented Jun 28, 2026

🔬 Smoke Test Results

Uh oh!

github-actions Bot commented Jun 28, 2026

Smoke Test: Claude Engine Validation

Uh oh!

github-actions Bot commented Jun 28, 2026

🧪 Smoke Test Results

Uh oh!

github-actions Bot commented Jun 28, 2026

Smoke Test: API Proxy OpenTelemetry Tracing

Uh oh!

github-actions Bot commented Jun 28, 2026

Chroot Version Comparison Results

Uh oh!

github-actions Bot commented Jun 28, 2026

Uh oh!

github-actions Bot commented Jun 28, 2026

Smoke Test Results: Gemini Engine

Uh oh!

Copilot AI commented Jun 28, 2026 •

edited

Loading

github-actions Bot commented Jun 28, 2026 •

edited

Loading

github-actions Bot commented Jun 28, 2026 •

edited

Loading

github-actions Bot commented Jun 28, 2026 •

edited

Loading

github-actions Bot commented Jun 28, 2026 •

edited

Loading

github-actions Bot commented Jun 28, 2026 •

edited

Loading

github-actions Bot commented Jun 28, 2026 •

edited

Loading

github-actions Bot commented Jun 28, 2026 •

edited

Loading

github-actions Bot commented Jun 28, 2026 •

edited

Loading

github-actions Bot commented Jun 28, 2026 •

edited

Loading

github-actions Bot commented Jun 28, 2026 •

edited

Loading

github-actions Bot commented Jun 28, 2026 •

edited

Loading

github-actions Bot commented Jun 28, 2026 •

edited

Loading

github-actions Bot commented Jun 28, 2026 •

edited

Loading