Skip to content

refactor(tests): extract shared harness for network-isolation workflow tests - #6496

Merged
lpcox merged 3 commits into
mainfrom
copilot/fix-repeated-workflow-setup
Jul 22, 2026
Merged

refactor(tests): extract shared harness for network-isolation workflow tests#6496
lpcox merged 3 commits into
mainfrom
copilot/fix-repeated-workflow-setup

Conversation

Copilot AI commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Five test cases in the onNetworkReady static DNS pre-registration describe block in src/cli-workflow.test.ts repeated the same ~16-line setup — mocking runtimeNeedsStaticDns, getTopologyContainerIps, and patchComposeWithTopologyHosts, constructing a networkIsolation: true config, and wiring up a startContainers stub that fires onNetworkReady — differing only in peer IPs, config overrides, and assertions.

Changes

  • src/cli-workflow.test.ts — adds runNetworkIsolationWorkflow helper inside the describe block, accepting { needsStaticDns, peerIps, configOverrides, connectTopologyContainers } with sensible defaults; each of the five tests now only supplies what's unique to it:
// Before — repeated in every test:
mockedRuntimeNeedsStaticDns.mockReturnValue(true);
const peerIps = new Map([['mcp-gateway', '172.30.0.100']]);
mockedGetTopologyContainerIps.mockResolvedValue(peerIps);
mockedPatchComposeWithTopologyHosts.mockImplementation(() => {});
const config: WrapperConfig = { ...baseConfig, networkIsolation: true, topologyAttach: ['mcp-gateway'], containerRuntime: 'gvisor' };
const startContainers = jest.fn().mockImplementation(
  async (_w, _d, _l, _s, onNetworkReady?) => { if (onNetworkReady) await onNetworkReady(); }
);
await runMainWorkflow(config, createWorkflowDependencies({ startContainers, connectTopologyContainers }), createWorkflowOptions());

// After:
await runNetworkIsolationWorkflow({
  needsStaticDns: true,
  peerIps: new Map([['mcp-gateway', '172.30.0.100']]),
  configOverrides: { topologyAttach: ['mcp-gateway'], containerRuntime: 'gvisor' },
  connectTopologyContainers,
});

Reduces ~80 lines of boilerplate; future changes to the topology test harness now require a single edit point.

Removes ~80 lines of repeated test setup across 5 nearly-identical blocks in the
`onNetworkReady static DNS pre-registration` describe block. The new
`runNetworkIsolationWorkflow` helper centralises:
- mockedRuntimeNeedsStaticDns.mockReturnValue(...)
- mockedGetTopologyContainerIps.mockResolvedValue(peerIps)
- mockedPatchComposeWithTopologyHosts.mockImplementation(() => {})
- building the WrapperConfig with networkIsolation: true
- creating the startContainers stub that invokes onNetworkReady
- calling runMainWorkflow

Each test now only declares its unique inputs (needsStaticDns, peerIps,
configOverrides, connectTopologyContainers) and its specific assertions.

Closes #6490
Copilot AI changed the title [WIP] Refactor duplicated network-isolation workflow test setup refactor(tests): extract shared harness for network-isolation workflow tests Jul 22, 2026
Copilot finished work on behalf of lpcox July 22, 2026 13:39
Copilot AI requested a review from lpcox July 22, 2026 13:39
@lpcox
lpcox marked this pull request as ready for review July 22, 2026 15:18
Copilot AI review requested due to automatic review settings July 22, 2026 15:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors network-isolation workflow tests by extracting repeated setup into a shared helper.

Changes:

  • Adds a configurable network-isolation test harness.
  • Simplifies five topology and static-DNS test cases.
Show a summary per file
File Description
src/cli-workflow.test.ts Centralizes repeated workflow mock setup and execution.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 1/1 changed files
  • Comments generated: 1
  • Review effort level: Medium

Comment thread src/cli-workflow.test.ts Outdated
* Each test only needs to declare the inputs and assertions unique to it.
*/
const runNetworkIsolationWorkflow = async ({
needsStaticDns = false,
@lpcox

lpcox commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator

@copilot address review feedback

Copilot AI commented Jul 22, 2026

Copy link
Copy Markdown
Contributor Author

@copilot address review feedback

Addressed in fe0b320.

I removed the ineffective needsStaticDns harness input/mock wiring and updated the affected test names/calls to reflect the unconditional network-isolation pre-registration behavior.

@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Smoke Gemini completed. All facets verified. 💎

@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Contribution Check completed successfully!

Contribution check complete: the PR follows the applicable guidelines in CONTRIBUTING.md. The change is a test refactor with no new functionality, includes tests in the touched test file, stays in the correct source area, and the PR description is clear with a related issue reference.

@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅

@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓

@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

🔑 Smoke Copilot PAT PAT auth validated. All systems operational. ✅

@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

🛡️ Smoke Copilot Network Isolation confirmed the egress allowlist is enforced. ✅

@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Build Test Suite completed successfully!

@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

🔌 Smoke Services — All services reachable! ✅

@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

📰 VERDICT: Smoke Docker Sbx has concluded. All systems operational. This is a developing story. 🎤

@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Smoke Copilot BYOK AOAI (api-key) completed. Copilot AOAI BYOK (api-key) mode operational. 🔓

@github-actions

Copy link
Copy Markdown
Contributor

🚀 Security Guard has started processing this pull request

@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

@github-actions

Copy link
Copy Markdown
Contributor

✅ Coverage Check Passed

Overall Coverage

Metric Base PR Delta
Lines 98.99% 99.02% 📈 +0.03%
Statements 98.92% 98.95% 📈 +0.03%
Functions 99.40% 99.40% ➡️ +0.00%
Branches 95.34% 95.34% ➡️ +0.00%
📁 Per-file Coverage Changes (1 files)
File Lines (Before → After) Statements (Before → After)
src/log-directory-setup.ts 96.2% → 100.0% (+3.78%) 96.3% → 100.0% (+3.71%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

@github-actions

Copy link
Copy Markdown
Contributor

🔥 Smoke Test: Copilot Network Isolation — PASS

@lpcox

EGRESS_RESULT allow=pass deny=pass

  • ✅ Allowed domain (api.github.com): HTTP 200
  • ✅ Blocked domain (example.com): blocked (403 via proxy)

Overall: PASS

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • example.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "example.com"

See Network Configuration for more information.

🛡️ Egress verdict from Smoke Copilot Network Isolation
Add label ready-for-aw to run again

@github-actions github-actions Bot added the smoke-copilot-network-isolation Copilot network-isolation egress smoke test label Jul 22, 2026
@lpcox
lpcox enabled auto-merge (squash) July 22, 2026 16:21
@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Claude Engine Validation

Check Result
API status ✅ PASS
gh check ✅ PASS
File status ✅ PASS

Overall result: PASS

Generated by Smoke Claude for #6496 · 31.2 AIC · ⊞ 3.2K ·
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

🔬 Smoke Test Results — PAT Auth

Test Status
GitHub MCP connectivity ✅ PASS
GitHub.com HTTP ⚠️ N/A (template vars unresolved)
File write/read ⚠️ N/A (template vars unresolved)

Auth mode: PAT (COPILOT_GITHUB_TOKEN)
Author: @lpcox

⚠️ Pre-step outputs were not interpolated (${{ steps.smoke-data.outputs.* }} literals). MCP test passed.

Overall: PARTIAL (MCP ✅, pre-step data unavailable)

🔑 PAT report filed by Smoke Copilot PAT
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

🔬 Smoke Test Results

Test Result
GitHub MCP connectivity ⚠️ PR data filtered by secrecy policy
GitHub.com HTTP ⚠️ Pre-step data not resolved (template vars unsubstituted)
File write/read ⚠️ Pre-step data not resolved (template vars unsubstituted)

Overall: INCONCLUSIVE — workflow template variables (${{ steps.smoke-data.outputs.* }}) were not substituted before reaching this agent. Pre-computed test results unavailable.

Author: @lpcox

📰 BREAKING: Report filed by Smoke Copilot
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test

  • Last merged PRs: Support COPILOT_MODEL=auto in AWF validation and api-proxy resolution; Auto-allow topology-attached container hostnames in Squid ACL
  • GitHub PR query: ✅
  • PR list query: ✅
  • Playwright: ✅
  • File write: ✅
  • Discussion lookup/comment: ✅
  • Build: ✅
  • Overall: PASS

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test Results — Services Connectivity

Check Result
Redis PING ❌ Name resolution failure (host.docker.internal unreachable)
PostgreSQL pg_isready ❌ No response
PostgreSQL SELECT 1 ❌ Name resolution failure

Overall: FAILhost.docker.internal DNS is not resolvable in this environment.

🔌 Service connectivity validated by Smoke Services
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Copilot BYOK (Direct) Mode ✅ PASS

  • ✅ GitHub MCP Testing
  • ✅ GitHub.com Connectivity
  • ✅ File Write/Read Test
  • ✅ BYOK Inference Test

Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY) via api-proxy → api.githubcopilot.com

🔑 BYOK report filed by Smoke Copilot BYOK
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test Results\n- GitHub MCP: ✅ (Filtered by DIFC)\n- GitHub.com: ✅ (via proxy)\n- File Writing: ✅\n- Bash Tool: ✅\n\nOverall status: PASS

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • localhost

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "localhost"

See Network Configuration for more information.

💎 Faceted by Smoke Gemini
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

🔍 Smoke Test: API Proxy OpenTelemetry Tracing

Scenario Result Notes
1. Module Loading ✅ Pass otel.js loads successfully; exports: startRequestSpan, setTokenAttributes, setBudgetAttributes, endSpan, endSpanError, shutdown, isEnabled
2. Test Suite ✅ Pass 59 tests passed, 0 failed across otel.test.js + otel-fanout.test.js
3. Env Var Forwarding ✅ Pass src/services/api-proxy-env-config.ts forwards GH_AW_OTLP_ENDPOINTS, GITHUB_AW_OTEL_TRACE_ID, GITHUB_AW_OTEL_PARENT_SPAN_ID, and standard OTEL_* vars to the api-proxy container
4. Token Tracker Integration ✅ Pass onUsage callback exists in token-tracker-http.js (line 285); invoked after normalized usage extraction as the OTEL hook point
5. OTEL Diagnostics ✅ Pass Graceful degradation confirmed: when no OTLP endpoint is configured, otel.js falls back to writing span NDJSON to /var/log/api-proxy/otel.jsonl; isEnabled() reflects state correctly

All 5 scenarios passed. OTEL tracing integration is functional.

📡 OTel tracing validated by Smoke OTel Tracing
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

@lpcox
Smoke test: MCP ✅, HTTP ✅, file I/O ✅, BYOK ✅
Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw)
Overall: PASS

🔑 BYOK (AOAI api-key) report filed by Smoke Copilot BYOK AOAI (api-key)
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

🏗️ Build Test Suite Results

Ecosystem Project Build/Install Tests Status
Bun elysia 1/1 passed ✅ PASS
Bun hono 1/1 passed ✅ PASS
C++ fmt N/A ✅ PASS
C++ json N/A ✅ PASS
Deno oak N/A 1/1 passed ✅ PASS
Deno std N/A 1/1 passed ✅ PASS
.NET hello-world N/A ✅ PASS
.NET json-parse N/A ✅ PASS
Go color 1/1 passed ✅ PASS
Go env 1/1 passed ✅ PASS
Go uuid 1/1 passed ✅ PASS
Java gson 1/1 passed ✅ PASS
Java caffeine 1/1 passed ✅ PASS
Node.js clsx All passed ✅ PASS
Node.js execa All passed ✅ PASS
Node.js p-limit All passed ✅ PASS
Rust fd 1/1 passed ✅ PASS
Rust zoxide 1/1 passed ✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Generated by Build Test Suite for #6496 · 49.5 AIC · ⊞ 8.2K ·
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

🧪 Smoke Test: Docker Sbx — PASS

Test Result
GitHub MCP connectivity
GitHub.com HTTP connectivity
File write/read

Overall: PASS

📰 BREAKING: Report filed by Smoke Docker Sbx
Add label ready-for-aw to run again

@lpcox
lpcox merged commit af40848 into main Jul 22, 2026
137 of 142 checks passed
@lpcox
lpcox deleted the copilot/fix-repeated-workflow-setup branch July 22, 2026 16:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Duplicate Code] Repeated network-isolation workflow test setup in cli-workflow.test.ts

3 participants