docs(runner-doctor): add B17 (Tailscale DNS black-hole) and B18 (Azure CLI/.azure) failure modes by Copilot · Pull Request #6735 · github/gh-aw-firewall

Copilot · 2026-07-29T23:11:50Z

Adds two new failure-mode entries to the self-hosted runner doctor catalog, mirrored across all three canonical locations kept in sync by the updater workflow.

New entries

B17 — Tailscale + network-isolation DNS black-hole

Tailscale's 0.0.0.0/0 policy route captures Docker bridge traffic, making host-specific DNS servers (168.63.129.16, 100.100.100.100, link-local) unreachable from the isolation network
Every Squid CONNECT fails with 503 TCP_TUNNEL:HIER_NONE (server field -:-) on allowlisted hosts
Fixed in AWF fix(dns): filter non-portable DNS in network-isolation + Tailscale #6705: filterForNetworkIsolation() in src/dns-resolver.ts strips non-portable DNS before generating squid.conf; falls back to 8.8.8.8/8.8.4.4

B18 — Azure CLI / ADO MCP auth invisible in sandbox

.azure was in home.forbiddenSubdirs (not toolSubdirs), so OIDC-provisioned Azure config never reached the sandbox
AZURE_CONFIG_DIR and ADO_MCP_AUTH_TOKEN were not in the always-forwarded env var list
Fixed in AWF Expose Azure CLI state and ADO auth token inside AWF sandbox #6690: .azure moved to toolSubdirs; both env vars added to env-passthrough.ts

Files changed

shared/self-hosted-failure-modes.md — B17 and B18 table rows + error-string quick-lookup entries
self-hosted-runner-doctor.md — matching error-string bullets and fix notes
.github/agents/self-hosted-runner-doctor.md — same additions to the embedded catalog copy

Fixes 🩺 Runner Doctor Update2026-07-29 scan — B17 Tailscale+isolation DNS black-hole, B18 Azure CLI/.azure whitelist #6727

…e CLI/.azure) failure modes

Copilot

Pull request overview

Adds B17 and B18 diagnostics to the runner-doctor catalog.

Changes:

Documents Tailscale-related DNS failures.
Documents Azure CLI/ADO sandbox authentication failures.
Mirrors lookup and remediation guidance across all doctor copies.

Show a summary per file

File	Description
`.github/workflows/shared/self-hosted-failure-modes.md`	Adds canonical B17/B18 entries.
`.github/workflows/self-hosted-runner-doctor.md`	Adds matching diagnostic guidance.
`.github/agents/self-hosted-runner-doctor.md`	Updates the portable embedded catalog.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (2)

.github/agents/self-hosted-runner-doctor.md:237

The probe expands ADO_MCP_AUTH_TOKEN into stdout. The portable doctor can execute and retain this output in logs or diagnostics, exposing the bearer token. Check only whether it is set (for example, with test -n "${ADO_MCP_AUTH_TOKEN:-}") and print a boolean/status; report AZURE_CONFIG_DIR separately because it is not secret.

| B18 | Azure CLI, Azure DevOps CLI, and Azure/ADO MCP servers fail inside the AWF agent sandbox even after OIDC login completes in runner setup steps and relevant Azure/Microsoft domains are allowlisted; `~/.azure` config is invisible inside the sandbox, and `ADO_MCP_AUTH_TOKEN`/`AZURE_CONFIG_DIR` are not forwarded when `--enable-api-proxy` is active | `.azure` was not in the whitelisted `$HOME` tool subdirs (`home.toolSubdirs`) in the canonical sandbox mount policy — it was actually in `home.forbiddenSubdirs` — so Azure CLI config written by runner setup steps never reaches the sandbox; separately, `AZURE_CONFIG_DIR` and `ADO_MCP_AUTH_TOKEN` were not in the always-forwarded host env var list, so the MCP bridge could not authenticate | **Fixed in AWF (PR github/gh-aw-firewall#6690, merged 2026-07-28):** `.azure` added to `home.toolSubdirs` and removed from `home.forbiddenSubdirs` in the canonical mount policy (applies to both compose and sbx runtimes); `AZURE_CONFIG_DIR` and `ADO_MCP_AUTH_TOKEN` added to the always-forwarded env vars in `src/services/agent-environment/env-passthrough.ts`. Upgrade to AWF version including github/gh-aw-firewall#6690. | Inside the agent sandbox: `ls -la ~/.azure` — presence confirms fix; `echo $AZURE_CONFIG_DIR $ADO_MCP_AUTH_TOKEN` — non-empty on patched AWF when set on the runner | github/gh-aw-firewall#6686, github/gh-aw-firewall#6690 |

.github/workflows/shared/self-hosted-failure-modes.md:60

The probe expands ADO_MCP_AUTH_TOKEN into stdout. Runner-doctor command output can be retained in logs or posted in diagnostics, so this exposes the bearer token the new entry is intended to troubleshoot. Check only whether it is set (for example, with test -n "${ADO_MCP_AUTH_TOKEN:-}") and print a boolean/status; AZURE_CONFIG_DIR can be reported separately because it is not secret.

| B18 | Azure CLI, Azure DevOps CLI, and Azure/ADO MCP servers fail inside the AWF agent sandbox even after OIDC login completes in runner setup steps and relevant Azure/Microsoft domains are allowlisted; `~/.azure` config is invisible inside the sandbox, and `ADO_MCP_AUTH_TOKEN`/`AZURE_CONFIG_DIR` are not forwarded when `--enable-api-proxy` is active | `.azure` was not in the whitelisted `$HOME` tool subdirs (`home.toolSubdirs`) in the canonical sandbox mount policy — it was actually in `home.forbiddenSubdirs` — so Azure CLI config written by runner setup steps never reaches the sandbox; separately, `AZURE_CONFIG_DIR` and `ADO_MCP_AUTH_TOKEN` were not in the always-forwarded host env var list, so the MCP bridge could not authenticate | **Fixed in AWF (PR github/gh-aw-firewall#6690, merged 2026-07-28):** `.azure` added to `home.toolSubdirs` and removed from `home.forbiddenSubdirs` in the canonical mount policy (applies to both compose and sbx runtimes); `AZURE_CONFIG_DIR` and `ADO_MCP_AUTH_TOKEN` added to the always-forwarded env vars in `src/services/agent-environment/env-passthrough.ts`. Upgrade to AWF version including github/gh-aw-firewall#6690. | Inside the agent sandbox: `ls -la ~/.azure` — presence confirms fix; `echo $AZURE_CONFIG_DIR $ADO_MCP_AUTH_TOKEN` — non-empty on patched AWF when set on the runner | github/gh-aw-firewall#6686, github/gh-aw-firewall#6690 |

Files reviewed: 3/3 changed files
Comments generated: 2
Review effort level: Medium

 | B15 | AWF startup fails with `--network-isolation is not yet supported with --enable-host-access`; happens when `localhost` is in the domain allowlist because the gh-aw compiler auto-enables `--enable-host-access` and also emits `network.isolation: true` + `topologyAttach` | In `src/cli.ts`, `--network-isolation` and `--enable-host-access` are mutually exclusive with no overlap support. The gh-aw compiler (v0.82.x+) does not detect the conflict: it emits `isolation:true` + `topologyAttach` unconditionally while also auto-enabling `--enable-host-access` when `localhost` is in the allowlist. The compiled workflow always fails at runtime. | **Fixed in AWF (PR github/gh-aw-firewall#6657, merged 2026-07-28):** The mutual-exclusion guard is removed; `--enable-host-access` now coexists with `--network-isolation` in topology mode. In topology mode the agent is on an `internal` Docker network with no host route, so `--enable-host-access` only drives Squid port ACLs and the `host.docker.internal` hosts-file entry — no incompatible iptables changes. `allowHostServicePorts` (iptables-based GitHub Actions services) is still suppressed unconditionally. Upgrade AWF to version including github/gh-aw-firewall#6657. | `awf --network-isolation --enable-host-access ...` — on AWF versions before github/gh-aw-firewall#6657, reproduces immediately with `not yet supported` error; on patched AWF the combination is accepted without error; check compiler output for both flags being emitted together when `localhost` is in allowlist | github/gh-aw-firewall#6651, github/gh-aw-firewall#6657 |
 | B16 | AWF rejects a `--mount` entry with a non-absolute host path error; the mount spec was defined as `${TERRAFORM_CLI_PATH}/terraform:...` (or similar env-var pattern) but the literal string `${TERRAFORM_CLI_PATH}` reaches AWF's volume validator | The gh-aw compiler wraps `sandbox.agent.mounts` specs containing `${}` references in single quotes in the generated shell invocation. Bash single quotes prevent variable substitution, so the literal variable reference string (e.g. `${TERRAFORM_CLI_PATH}`) reaches AWF's volume validation instead of the resolved absolute path. | **Fixed in AWF (PR github/gh-aw-firewall#6655, merged 2026-07-27):** `expandEnvVarsInMount()` in `src/parsers/volume-parsers.ts` now expands `${VAR_NAME}` and `$VAR_NAME` patterns from `process.env` before path validation. If the variable is undefined, AWF emits a precise error (`Environment variable is not set: ${VAR_NAME}`) rather than a misleading path-absoluteness failure. Upgrade AWF to the version including github/gh-aw-firewall#6655. | Inspect the generated `awf` invocation in the compiled lock file — if mount specs appear in single quotes containing `${...}`, the bug is present. Running `awf ... --mount '${VAR}/path:/dest'` on AWF versions before github/gh-aw-firewall#6655 reproduces the rejection; on patched AWF the mount spec is accepted after variable expansion. | github/gh-aw-firewall#6649, github/gh-aw-firewall#6655 |
+| B17 | In `--network-isolation`/topology mode, when a workflow also brings up Tailscale as a later step, every Squid `CONNECT` to an allowlisted API host (e.g. `api.githubcopilot.com`) fails with `503 TCP_TUNNEL:HIER_NONE` (server field `-:-`, DNS never resolved); agent reports `503 Service Unavailable` retries and fails. Same workflow works on host-access mode or in isolation mode without Tailscale. | When Tailscale starts, it can install a policy-routing rule covering `0.0.0.0/0` (exit node / accepted subnet route). Docker bridge traffic on the isolation network follows that route through the Tailscale tunnel. Host-specific DNS servers reachable only via the original network path (Azure DHCP DNS `168.63.129.16`, Tailscale Magic DNS `100.100.100.100`, link-local `169.254.x.x`) become unreachable once Tailscale captures the route, so Squid's DNS queries black-hole and every CONNECT fails at resolution. | **Fixed in AWF (PR github/gh-aw-firewall#6705, merged 2026-07-29):** new `isNonPortableDns(ip)` / `filterForNetworkIsolation(servers, logger)` in `src/dns-resolver.ts` strip non-portable DNS servers (Azure DHCP DNS, Tailscale Magic DNS, link-local) before generating `squid.conf` when `config.networkIsolation` is true; falls back to `8.8.8.8`/`8.8.4.4` if all detected servers are non-portable. Users needing specific DNS in isolation mode can still override via `--dns-servers`. Upgrade to AWF version including github/gh-aw-firewall#6705. | Squid access log shows `CONNECT ... 503 TCP_TUNNEL:HIER_NONE` with server field `-:-` for an allowlisted domain, specifically after a Tailscale-up step in isolation/topology mode; inspect `audit/awf-resolved-config.json` for `networkIsolation: true` + Tailscale in the same run; check whether the DNS servers in use are Azure DHCP (`168.63.129.16`) or Tailscale Magic DNS (`100.100.100.100`) | github/gh-aw-firewall#6704, github/gh-aw-firewall#6705 |
+| B18 | Azure CLI, Azure DevOps CLI, and Azure/ADO MCP servers fail inside the AWF agent sandbox even after OIDC login completes in runner setup steps and relevant Azure/Microsoft domains are allowlisted; `~/.azure` config is invisible inside the sandbox, and `ADO_MCP_AUTH_TOKEN`/`AZURE_CONFIG_DIR` are not forwarded when `--enable-api-proxy` is active | `.azure` was not in the whitelisted `$HOME` tool subdirs (`home.toolSubdirs`) in the canonical sandbox mount policy — it was actually in `home.forbiddenSubdirs` — so Azure CLI config written by runner setup steps never reaches the sandbox; separately, `AZURE_CONFIG_DIR` and `ADO_MCP_AUTH_TOKEN` were not in the always-forwarded host env var list, so the MCP bridge could not authenticate | **Fixed in AWF (PR github/gh-aw-firewall#6690, merged 2026-07-28):** `.azure` added to `home.toolSubdirs` and removed from `home.forbiddenSubdirs` in the canonical mount policy (applies to both compose and sbx runtimes); `AZURE_CONFIG_DIR` and `ADO_MCP_AUTH_TOKEN` added to the always-forwarded env vars in `src/services/agent-environment/env-passthrough.ts`. Upgrade to AWF version including github/gh-aw-firewall#6690. | Inside the agent sandbox: `ls -la ~/.azure` — presence confirms fix; `echo $AZURE_CONFIG_DIR $ADO_MCP_AUTH_TOKEN` — non-empty on patched AWF when set on the runner | github/gh-aw-firewall#6686, github/gh-aw-firewall#6690 |


 | B15 | AWF startup fails with `--network-isolation is not yet supported with --enable-host-access`; happens when `localhost` is in the domain allowlist because the gh-aw compiler auto-enables `--enable-host-access` and also emits `network.isolation: true` + `topologyAttach` | In `src/cli.ts`, `--network-isolation` and `--enable-host-access` are mutually exclusive with no overlap support. The gh-aw compiler (v0.82.x+) does not detect the conflict: it emits `isolation:true` + `topologyAttach` unconditionally while also auto-enabling `--enable-host-access` when `localhost` is in the allowlist. The compiled workflow always fails at runtime. | **Fixed in AWF (PR github/gh-aw-firewall#6657, merged 2026-07-28):** The mutual-exclusion guard is removed; `--enable-host-access` now coexists with `--network-isolation` in topology mode. In topology mode the agent is on an `internal` Docker network with no host route, so `--enable-host-access` only drives Squid port ACLs and the `host.docker.internal` hosts-file entry — no incompatible iptables changes. `allowHostServicePorts` (iptables-based GitHub Actions services) is still suppressed unconditionally. Upgrade AWF to version including github/gh-aw-firewall#6657. | `awf --network-isolation --enable-host-access ...` — on AWF versions before github/gh-aw-firewall#6657, reproduces immediately with `not yet supported` error; on patched AWF the combination is accepted without error; check compiler output for both flags being emitted together when `localhost` is in allowlist | github/gh-aw-firewall#6651, github/gh-aw-firewall#6657 |
 | B16 | AWF rejects a `--mount` entry with a non-absolute host path error; the mount spec was defined as `${TERRAFORM_CLI_PATH}/terraform:...` (or similar env-var pattern) but the literal string `${TERRAFORM_CLI_PATH}` reaches AWF's volume validator | The gh-aw compiler wraps `sandbox.agent.mounts` specs containing `${}` references in single quotes in the generated shell invocation. Bash single quotes prevent variable substitution, so the literal variable reference string (e.g. `${TERRAFORM_CLI_PATH}`) reaches AWF's volume validation instead of the resolved absolute path. | **Fixed in AWF (PR github/gh-aw-firewall#6655, merged 2026-07-27):** `expandEnvVarsInMount()` in `src/parsers/volume-parsers.ts` now expands `${VAR_NAME}` and `$VAR_NAME` patterns from `process.env` before path validation. If the variable is undefined, AWF emits a precise error (`Environment variable is not set: ${VAR_NAME}`) rather than a misleading path-absoluteness failure. Upgrade AWF to the version including github/gh-aw-firewall#6655. | Inspect the generated `awf` invocation in the compiled lock file — if mount specs appear in single quotes containing `${...}`, the bug is present. Running `awf ... --mount '${VAR}/path:/dest'` on AWF versions before github/gh-aw-firewall#6655 reproduces the rejection; on patched AWF the mount spec is accepted after variable expansion. | github/gh-aw-firewall#6649, github/gh-aw-firewall#6655 |
+| B17 | In `--network-isolation`/topology mode, when a workflow also brings up Tailscale as a later step, every Squid `CONNECT` to an allowlisted API host (e.g. `api.githubcopilot.com`) fails with `503 TCP_TUNNEL:HIER_NONE` (server field `-:-`, DNS never resolved); agent reports `503 Service Unavailable` retries and fails. Same workflow works on host-access mode or in isolation mode without Tailscale. | When Tailscale starts, it can install a policy-routing rule covering `0.0.0.0/0` (exit node / accepted subnet route). Docker bridge traffic on the isolation network follows that route through the Tailscale tunnel. Host-specific DNS servers reachable only via the original network path (Azure DHCP DNS `168.63.129.16`, Tailscale Magic DNS `100.100.100.100`, link-local `169.254.x.x`) become unreachable once Tailscale captures the route, so Squid's DNS queries black-hole and every CONNECT fails at resolution. | **Fixed in AWF (PR github/gh-aw-firewall#6705, merged 2026-07-29):** new `isNonPortableDns(ip)` / `filterForNetworkIsolation(servers, logger)` in `src/dns-resolver.ts` strip non-portable DNS servers (Azure DHCP DNS, Tailscale Magic DNS, link-local) before generating `squid.conf` when `config.networkIsolation` is true; falls back to `8.8.8.8`/`8.8.4.4` if all detected servers are non-portable. Users needing specific DNS in isolation mode can still override via `--dns-servers`. Upgrade to AWF version including github/gh-aw-firewall#6705. | Squid access log shows `CONNECT ... 503 TCP_TUNNEL:HIER_NONE` with server field `-:-` for an allowlisted domain, specifically after a Tailscale-up step in isolation/topology mode; inspect `audit/awf-resolved-config.json` for `networkIsolation: true` + Tailscale in the same run; check whether the DNS servers in use are Azure DHCP (`168.63.129.16`) or Tailscale Magic DNS (`100.100.100.100`) | github/gh-aw-firewall#6704, github/gh-aw-firewall#6705 |
+| B18 | Azure CLI, Azure DevOps CLI, and Azure/ADO MCP servers fail inside the AWF agent sandbox even after OIDC login completes in runner setup steps and relevant Azure/Microsoft domains are allowlisted; `~/.azure` config is invisible inside the sandbox, and `ADO_MCP_AUTH_TOKEN`/`AZURE_CONFIG_DIR` are not forwarded when `--enable-api-proxy` is active | `.azure` was not in the whitelisted `$HOME` tool subdirs (`home.toolSubdirs`) in the canonical sandbox mount policy — it was actually in `home.forbiddenSubdirs` — so Azure CLI config written by runner setup steps never reaches the sandbox; separately, `AZURE_CONFIG_DIR` and `ADO_MCP_AUTH_TOKEN` were not in the always-forwarded host env var list, so the MCP bridge could not authenticate | **Fixed in AWF (PR github/gh-aw-firewall#6690, merged 2026-07-28):** `.azure` added to `home.toolSubdirs` and removed from `home.forbiddenSubdirs` in the canonical mount policy (applies to both compose and sbx runtimes); `AZURE_CONFIG_DIR` and `ADO_MCP_AUTH_TOKEN` added to the always-forwarded env vars in `src/services/agent-environment/env-passthrough.ts`. Upgrade to AWF version including github/gh-aw-firewall#6690. | Inside the agent sandbox: `ls -la ~/.azure` — presence confirms fix; `echo $AZURE_CONFIG_DIR $ADO_MCP_AUTH_TOKEN` — non-empty on patched AWF when set on the runner | github/gh-aw-firewall#6686, github/gh-aw-firewall#6690 |


lpcox · 2026-07-30T00:09:27Z

@copilot address review feedback

… in-sandbox OIDC re-login required

Copilot · 2026-07-30T00:13:02Z

@copilot address review feedback

Done in fa9c9f4. The B18 entry across all three catalog locations now:

Adds an "Authentication caveat" block that explicitly states Expose Azure CLI state and ADO auth token inside AWF sandbox #6690 deliberately scrubs msal_token_cache*, accessTokens.json, and service_principal_entries.json, so a pre-AWF az login is not inherited by the sandbox — only account metadata (tenant IDs, subscription list) is mounted
Documents that the agent must perform an in-sandbox OIDC re-login with a fresh writable AZURE_CONFIG_DIR, with a concrete az login --federated-token ... example, and a reference to [awf] agent: Azure CLI and ADO authentication not available inside sandbox #6686 for the working pattern
Updates the Probe column to note that az account show will still fail until that in-sandbox re-login is performed
Updates the quick-lookup entries in all three files with a matching note

github-actions · 2026-07-30T00:22:47Z

✅ Copilot review passed with no inline comments.

@copilot Add the ready-for-aw label to this PR to trigger agentic CI smoke tests.

Initial plan

d68a374

Copilot AI assigned Copilot and lpcox Jul 29, 2026

Copilot AI linked an issue Jul 29, 2026 that may be closed by this pull request

🩺 Runner Doctor Update2026-07-29 scan — B17 Tailscale+isolation DNS black-hole, B18 Azure CLI/.azure whitelist #6727

Closed

Copilot started work on behalf of lpcox July 29, 2026 23:12 View session

docs(runner-doctor): add B17 (Tailscale DNS black-hole) and B18 (Azur…

fa787bd

…e CLI/.azure) failure modes

Copilot AI changed the title ~~[WIP] Update runner doctor knowledge base with new lessons~~ docs(runner-doctor): add B17 (Tailscale DNS black-hole) and B18 (Azure CLI/.azure) failure modes Jul 29, 2026

Copilot finished work on behalf of lpcox July 29, 2026 23:17

Copilot AI requested a review from lpcox July 29, 2026 23:17

lpcox marked this pull request as ready for review July 29, 2026 23:22

Copilot AI review requested due to automatic review settings July 29, 2026 23:22

Copilot started reviewing on behalf of lpcox July 29, 2026 23:22 View session

Copilot AI reviewed Jul 29, 2026

View reviewed changes

Copilot started work on behalf of lpcox July 30, 2026 00:10 View session

docs(runner-doctor): clarify B18 auth caveat — token caches scrubbed,…

0912a9b

… in-sandbox OIDC re-login required

Copilot finished work on behalf of lpcox July 30, 2026 00:13

lpcox merged commit 30eb107 into main Jul 30, 2026
15 of 16 checks passed

lpcox deleted the copilot/update-runner-doctor-knowledge-base branch July 30, 2026 15:03

github-actions Bot mentioned this pull request Jul 30, 2026

[aw] No-Op Runs #6709

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

docs(runner-doctor): add B17 (Tailscale DNS black-hole) and B18 (Azure CLI/.azure) failure modes - #6735

docs(runner-doctor): add B17 (Tailscale DNS black-hole) and B18 (Azure CLI/.azure) failure modes#6735
lpcox merged 3 commits into
mainfrom
copilot/update-runner-doctor-knowledge-base

Copilot AI commented Jul 29, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

lpcox commented Jul 30, 2026

Uh oh!

Copilot AI commented Jul 30, 2026

Uh oh!

github-actions Bot commented Jul 30, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

Copilot AI commented Jul 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

New entries

Files changed

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Review details

Uh oh!

lpcox commented Jul 30, 2026

Uh oh!

Copilot AI commented Jul 30, 2026

Uh oh!

github-actions Bot commented Jul 30, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Jul 29, 2026 •

edited

Loading