Skip to content

Stabilize test-coverage-reporter by isolating main-action unit tests from DinD probing#3595

Merged
lpcox merged 2 commits into
mainfrom
copilot/debug-test-coverage-reporter-failure
May 22, 2026
Merged

Stabilize test-coverage-reporter by isolating main-action unit tests from DinD probing#3595
lpcox merged 2 commits into
mainfrom
copilot/debug-test-coverage-reporter-failure

Conversation

Copilot AI commented May 22, 2026

Copy link
Copy Markdown
Contributor

Test Coverage Reporter failed on main because npm run test:coverage intermittently timed out in src/commands/main-action.test.ts. The suite was executing real split-filesystem probe behavior, introducing Docker-dependent latency into a unit test path.

  • Root cause

    • createMainAction now awaits probeSplitFilesystem(...).
    • main-action.test.ts did not mock ../dind-probe, so tests could invoke real probe logic and become timing-sensitive in CI.
  • Changes made

    • Unit test isolation for createMainAction
      • Added jest.mock('../dind-probe').
      • Added a default beforeEach mock for probeSplitFilesystem to return a deterministic non-split result.
    • Scope
      • Test-only change in src/commands/main-action.test.ts; no runtime/workflow behavior changes.
  • Example

    jest.mock('../dind-probe');
    
    mockedDindProbe.probeSplitFilesystem.mockResolvedValue({
      prefix: undefined,
      splitDetected: false,
      inconclusive: false,
    });

Copilot AI linked an issue May 22, 2026 that may be closed by this pull request
Copilot AI changed the title [WIP] Debug test coverage reporter workflow failure Stabilize test-coverage-reporter by isolating main-action unit tests from DinD probing May 22, 2026
Copilot finished work on behalf of lpcox May 22, 2026 16:50
Copilot AI requested a review from lpcox May 22, 2026 16:50
@lpcox lpcox marked this pull request as ready for review May 22, 2026 16:59
Copilot AI review requested due to automatic review settings May 22, 2026 16:59
@github-actions

Copy link
Copy Markdown
Contributor

✅ Coverage Check Passed

Overall Coverage

Metric Base PR Delta
Lines 95.98% 96.05% 📈 +0.07%
Statements 95.81% 95.87% 📈 +0.06%
Functions 98.02% 98.02% ➡️ +0.00%
Branches 89.44% 89.48% 📈 +0.04%
📁 Per-file Coverage Changes (1 files)
File Lines (Before → After) Statements (Before → After)
src/config-writer.ts 83.0% → 85.6% (+2.54%) 83.0% → 85.6% (+2.54%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Stabilizes the createMainAction unit test suite by preventing main-action.test.ts from executing real DinD split-filesystem probe logic (which can introduce Docker-dependent latency/timeouts in CI).

Changes:

  • Mocked ../dind-probe in main-action.test.ts to avoid invoking real probe behavior.
  • Added a default beforeEach stub for probeSplitFilesystem returning a deterministic “no split detected” result.
Show a summary per file
File Description
src/commands/main-action.test.ts Mocks DinD probe module and provides a default resolved probe result to keep createMainAction tests deterministic and fast.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 1/1 changed files
  • Comments generated: 0

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test Results

GitHub API: Recent PRs verified (2 entries)
GitHub check: Playwright check PASS
File verify: Smoke test file exists

Overall: PASS

💥 [THE END] — Illustrated by Smoke Claude

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test Results

✅ GitHub MCP: chore(deps): bump dompurify from 3.3.2 to 3.4.5 in /docs-site
❌ GitHub.com HTTP: test file missing
❌ File Write/Read: /tmp/smoke-test.txt not found

Status: FAIL

CC: @Copilot @lpcox

📰 BREAKING: Report filed by Smoke Copilot

@github-actions

Copy link
Copy Markdown
Contributor

BYOK Smoke Test Results

Running in offline BYOK mode (COPILOT_OFFLINE=true) via api-proxy → api.githubcopilot.com

  • ✅ GitHub MCP connectivity (PR: "chore(deps): bump dompurify from 3.3.2 to 3.4.5 in /docs-site")
  • ❌ File write/read test (file not found at expected path)
  • ✅ BYOK inference path working
  • ❌ GitHub.com connectivity data missing

Status: PARTIAL FAIL (2/4 tests passed)

@Copilot @lpcox — File test failed, but BYOK inference and MCP working correctly.

🔑 BYOK report filed by Smoke Copilot BYOK

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test Results

  • GitHub MCP Testing: ❌ (mcpscripts tool missing)
  • GitHub.com Connectivity: ❌ (Status 400)
  • File Writing Testing: ✅
  • Bash Tool Testing: ✅

Overall status: FAIL

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • localhost

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "localhost"

See Network Configuration for more information.

💎 Faceted by Smoke Gemini

@github-actions

Copy link
Copy Markdown
Contributor

Chroot Runtime Version Comparison

Results from testing runtime versions between host and chroot environments:

Runtime Host Version Chroot Version Match?
Python 3.12.13 3.12.3 ❌ NO
Node.js v24.15.0 v22.22.3 ❌ NO
Go go1.22.12 go1.22.12 ✅ YES

Overall Status: ❌ Not all versions match

Details

  • Python: Minor version mismatch (3.12.13 vs 3.12.3)
  • Node.js: Major version mismatch (v24 vs v22)
  • Go: Versions match exactly

The chroot environment successfully exposes host binaries, but version mismatches indicate the container may be using different system paths or the bind mounts may not be covering all necessary directories.

Tested by Smoke Chroot

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Codex: FAIL
Merged PRs: Remove unused export from ProbeResult interface; fix: upgrade test-coverage-improver sandbox to AWF v0.25.51
GitHub PR review: ✅ | Safe Inputs GH CLI: ❌ | Playwright: ✅ | Tavily: ❌
File/Bash: ✅ | Discussion: ✅ #3589 | Build: ✅
Overall status: FAIL

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Services Connectivity — ❌ FAIL

  • Redis: Connection timeout to host.docker.internal:6379
  • PostgreSQL (pg_isready): No response from host.docker.internal:5432
  • PostgreSQL (query): Not tested (connection failed)

Overall: FAIL — Services not reachable from AWF sandbox

🔌 Service connectivity validated by Smoke Services

@github-actions

Copy link
Copy Markdown
Contributor

🏗️ Build Test Suite Results

All build tests completed successfully!

Ecosystem Project Build/Install Tests Status
Bun elysia 1/1 passed ✅ PASS
Bun hono 1/1 passed ✅ PASS
C++ fmt N/A ✅ PASS
C++ json N/A ✅ PASS
Deno oak N/A 1/1 passed ✅ PASS
Deno std N/A 1/1 passed ✅ PASS
.NET hello-world N/A ✅ PASS
.NET json-parse N/A ✅ PASS
Go color 1/1 passed ✅ PASS
Go env 1/1 passed ✅ PASS
Go uuid 1/1 passed ✅ PASS
Java gson 1/1 passed ✅ PASS
Java caffeine 1/1 passed ✅ PASS
Node.js clsx 1/1 passed ✅ PASS
Node.js execa 1/1 passed ✅ PASS
Node.js p-limit 1/1 passed ✅ PASS
Rust fd 1/1 passed ✅ PASS
Rust zoxide 1/1 passed ✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Summary

  • Total projects tested: 18
  • Total ecosystems: 8 (Bun, C++, Deno, .NET, Go, Java, Node.js, Rust)
  • Build/install success rate: 100% (16/16 applicable)
  • Test success rate: 100% (12/12 applicable)
  • Overall status:ALL TESTS PASSED

All language ecosystems successfully built and tested within the AWF firewall environment.

Generated by Build Test Suite for issue #3595 · ● 14.1M ·

@lpcox lpcox merged commit 7ac7c19 into main May 22, 2026
66 of 71 checks passed
@lpcox lpcox deleted the copilot/debug-test-coverage-reporter-failure branch May 22, 2026 17:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[aw] Test Coverage Reporter failed

3 participants