fix: diagnose ARC/DinD DNS isolation and quiet rootless chmod noise#6328
Conversation
Addresses two of the ARC/DinD implementation gaps from #6326. Gap 3 — DinD-spawned containers cannot resolve Kubernetes DNS names: When the external DIFC proxy is addressed by a Kubernetes Service name (e.g. awmg-cli-proxy), the cli-proxy fails with getaddrinfo EAI_AGAIN because containers on the DinD Docker network do not reach the Kubernetes cluster DNS resolver. The previous 'could not connect to the external DIFC proxy' error hid this root cause. Add detectDnsResolutionFailure() which scans cli-proxy logs for EAI_AGAIN/ENOTFOUND and, when found, augments the startup error with an actionable DNS-isolation explanation and fix guidance (address by IP, or configure dockerd --dns). Gap 4 — post-job chmod noise on rootless/non-privileged runners: The best-effort artifact permission repair emitted an alarming [WARN] when the repair container was denied CHOWN/chmod (Operation not permitted) on restricted runners. The agent has already finished and artifacts remain readable by the owning user, so downgrade benign permission errors to debug while keeping WARN for genuine, unexpected failures. Adds unit tests for both paths. Remaining gaps (#1 topology auto-defaults, #2 network-isolation smoke-test coverage) are larger and tracked in #6326. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: b593fd45-255a-49a2-9d21-fd78108650a2
|
| Metric | Base | PR | Delta |
|---|---|---|---|
| Lines | 99.02% | 99.04% | 📈 +0.02% |
| Statements | 98.96% | 98.98% | 📈 +0.02% |
| Functions | 99.35% | 99.35% | ➡️ +0.00% |
| Branches | 95.15% | 95.14% | 📉 -0.01% |
📁 Per-file Coverage Changes (4 files)
| File | Lines (Before → After) | Statements (Before → After) |
|---|---|---|
src/container-lifecycle.ts |
100.0% → 99.1% (-0.87%) | 100.0% → 99.2% (-0.83%) |
src/container-startup-diagnostics.ts |
98.6% → 98.8% (+0.16%) | 98.7% → 98.8% (+0.15%) |
src/artifact-permissions.ts |
97.0% → 97.3% (+0.33%) | 97.0% → 97.3% (+0.33%) |
src/log-directory-setup.ts |
96.2% → 100.0% (+3.78%) | 96.3% → 100.0% (+3.71%) |
Coverage comparison generated by scripts/ci/compare-coverage.ts
There was a problem hiding this comment.
Pull request overview
Improves ARC/DinD diagnostics and reduces benign artifact-permission warning noise.
Changes:
- Detects DNS-resolution failures in CLI proxy logs and provides actionable guidance.
- Downgrades expected restricted-runner permission errors to debug logging.
- Adds unit coverage for both behaviors.
Show a summary per file
| File | Description |
|---|---|
src/container-startup-diagnostics.ts |
Detects unresolved hostnames in container logs. |
src/container-startup-diagnostics-coverage.test.ts |
Tests DNS failure detection. |
src/container-lifecycle.ts |
Adds DNS guidance to CLI proxy startup errors. |
src/artifact-permissions.ts |
Reclassifies expected permission failures. |
src/artifact-permissions.test.ts |
Tests benign permission-error logging. |
Review details
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 5/5 changed files
- Comments generated: 1
- Review effort level: Medium
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
|
✅ Copilot review passed with no inline comments. @lpcox Add the |
|
| Metric | Base | PR | Delta |
|---|---|---|---|
| Lines | 99.02% | 99.04% | 📈 +0.02% |
| Statements | 98.96% | 98.98% | 📈 +0.02% |
| Functions | 99.35% | 99.35% | ➡️ +0.00% |
| Branches | 95.15% | 95.14% | 📉 -0.01% |
📁 Per-file Coverage Changes (4 files)
| File | Lines (Before → After) | Statements (Before → After) |
|---|---|---|
src/container-lifecycle.ts |
100.0% → 99.1% (-0.87%) | 100.0% → 99.2% (-0.83%) |
src/container-startup-diagnostics.ts |
98.6% → 98.8% (+0.16%) | 98.7% → 98.8% (+0.15%) |
src/artifact-permissions.ts |
97.0% → 97.3% (+0.33%) | 97.0% → 97.3% (+0.33%) |
src/log-directory-setup.ts |
96.2% → 100.0% (+3.78%) | 96.3% → 100.0% (+3.71%) |
Coverage comparison generated by scripts/ci/compare-coverage.ts
|
✅ Smoke Gemini completed. All facets verified. 💎 Smoke test completed with partial success (File/Bash ✅, Connectivity/MCP ❌). Results posted to PR #6328. |
|
📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅ |
|
✅ Smoke Copilot BYOK AOAI (api-key) completed. Copilot AOAI BYOK (api-key) mode operational. 🔓 |
|
📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤 |
|
🔌 Smoke Services — All services reachable! ✅ |
|
🚀 Security Guard has started processing this pull request |
|
✅ Build Test Suite completed successfully! |
|
✅ Contribution Check completed successfully! PR #6328 follows the contribution guidelines in CONTRIBUTING.md: it includes tests, keeps files in src/ and colocated tests, and has a clear description referencing related issues. No comment needed. |
|
❌ Smoke Copilot BYOK AOAI (Entra) reports failed. AOAI BYOK (Entra) mode investigation needed... Smoke test comment not posted: add_comment tool errors encountered; tests executed. |
|
✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓 |
|
✅ Smoke Claude passed |
|
Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded. |
|
✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟 |
|
📰 VERDICT: Smoke Docker Sbx has concluded. All systems operational. This is a developing story. 🎤 |
|
🔑 Smoke Copilot PAT PAT auth validated. All systems operational. ✅ |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
✅ Smoke Claude passed |
|
🚀 Security Guard has started processing this pull request |
|
✅ Smoke Gemini completed. All facets verified. 💎 |
|
✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓 |
|
✅ Contribution Check completed successfully! PR #6328 appears to satisfy CONTRIBUTING.md: tests are added, the description references related issues, and the changed files are in the right locations; no review comment needed. |
|
✅ Smoke Copilot BYOK AOAI (api-key) completed. Copilot AOAI BYOK (api-key) mode operational. 🔓 |
|
✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟 |
Smoke Test: Copilot PAT Auth
Overall: PASS (core connectivity verified) Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
Smoke Test: Claude Engine ValidationOverall result: ✅ PASS
Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
Smoke Test: Copilot BYOK (Direct) Mode ✅ PASS
Status: Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY) via api-proxy → api.githubcopilot.com Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
Smoke Test: GitHub Actions Services Connectivity
Overall: FAIL
Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
Smoke Test: API Proxy OpenTelemetry Tracing
All 5 scenarios passed. OTEL tracing integration is working correctly. Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
Running in direct BYOK mode (AWF_AUTH_TYPE=github-oidc + AWF_AUTH_AZURE_* + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw) authenticated via Microsoft Entra cc @lpcox Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
|
fix: diagnose ARC/DinD DNS isolation and quiet rootless chmod noise ✅ Reviewed: chore: upgrade gh-aw extension to latest pre-release Overall: FAIL Warning Firewall blocked 2 domainsThe following domains were blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"
- "registry.npmjs.org"See Network Configuration for more information.
|
Chroot Version Comparison
Not all runtimes match between host and chroot environment. Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
🔬 Smoke Test Results
Overall: FAIL Pre-step outputs ( Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
Smoke Test Results: Gemini
Overall Status: PASS Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "localhost"See Network Configuration for more information.
|
|
@lpcox Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
🏗️ Build Test Suite Results
Overall: 8/8 ecosystems passed — ✅ PASS Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
🔥 Smoke Test: Docker Sbx — PASS
Overall: PASS — @lpcox
|
Addresses two of the four ARC/DinD implementation gaps identified in #6326. Both are self-contained, testable code changes that directly reduce the user confusion reported in gh-aw#40107.
Gap 3 — DinD-spawned containers cannot resolve Kubernetes DNS names
Problem: When AWF components (e.g. the external DIFC proxy) are addressed by a Kubernetes Service name such as
awmg-cli-proxy, the cli-proxy fails withgetaddrinfo EAI_AGAIN. Containers created by the Docker-in-Docker daemon run on the DinD Docker network, which does not forward DNS to the Kubernetes cluster resolver. The previous error —awf-cli-proxy could not connect to the external DIFC proxy— hid this root cause, leaving operators stuck (see the reproduction in gh-aw#40107).Fix: Added
detectDnsResolutionFailure()incontainer-startup-diagnostics.ts, which scans the cli-proxy container logs forEAI_AGAIN/ENOTFOUNDand extracts the unresolved hostname. When present,createCliProxyStartupError()now appends an actionable explanation of the DNS-isolation root cause and concrete fixes (address the proxy by IP, or configuredockerd --dns <kube-dns-ip>), plus a link to the ARC/DinD guide.Gap 4 — post-job chmod noise on rootless/non-privileged runners
Problem: The best-effort artifact permission repair emitted an alarming
[WARN] Rootless artifact permission repair failed ...when the repair container was denied CHOWN/chmod (Operation not permitted) on restricted runners. Because the agent has already finished and artifacts remain readable by the owning user, this warning is non-actionable and confuses users into thinking something broke.Fix: In
artifact-permissions.ts, benign permission errors (Operation not permitted,Permission denied,EPERM,EACCES) are now logged atdebugwith a clear "expected on restricted runners" note. Genuine, unexpected failures (e.g. missing image) still surface asWARN.Testing
npm run type-check— passesnpm run lint— 0 errors (only pre-existing warnings in untouched lines)npm run test:unit— all tests pass except one pre-existing, environment-dependent failure incompose-generator.test.ts(macOS/tmp→/private/tmpsymlink; fails onmaintoo, unrelated to this change)detectDnsResolutionFailure: EAI_AGAIN extraction, ENOTFOUND extraction, no-failure, non-zero exit, exception handlingfixArtifactPermissionsForRootless: benign permission error no longer emits WARNNot included (tracked in #6326)
runner.topology: arc-dindauto-defaults fornetwork.isolation/dind.preStageDirs/dockerHostPathPrefix: larger behavioral change, needs its own design + tests.Closes part of #6326.