github · lpcox · Jul 30, 2026 · Jul 29, 2026 · Jul 29, 2026 · Jul 30, 2026
diff --git a/.github/agents/self-hosted-runner-doctor.md b/.github/agents/self-hosted-runner-doctor.md
@@ -85,6 +85,8 @@ Prefer the narrowest match. Examples:
 - `⚠️ Firewall blocked N domain(s)` warning lists `awmgmcpg` or `172.30.0.x` as a blocked domain on every run, even with no actual external blocks → B13 (internal MCP gateway traffic counted by log aggregator as denied; fixed in github/gh-aw-firewall#6689 with `isInternalAwfDomain()` filter)
 - `--network-isolation is not yet supported with --enable-host-access` → B15 (compiler auto-emits both flags when `localhost` in allowlist + topology; fixed in github/gh-aw-firewall#6657)
 - AWF rejects `--mount` with "host path must be absolute" and the path visibly contains `${VAR_NAME}` unexpanded → B16 (single-quote wrapping by compiler prevents shell expansion of `${}` in mount specs; fixed in github/gh-aw-firewall#6655)
+- `503 TCP_TUNNEL:HIER_NONE` (server field `-:-`) on an allowlisted API host in network-isolation/topology mode, specifically after a Tailscale-up step → B17 (Tailscale policy-routing captures the default route, making host-specific DNS servers unreachable; fixed in github/gh-aw-firewall#6705 with `filterForNetworkIsolation()` stripping non-portable DNS before Squid config is generated)
+- Azure CLI / ADO MCP auth failures with `~/.azure` missing inside AWF sandbox, or `AZURE_CONFIG_DIR`/`ADO_MCP_AUTH_TOKEN` empty inside agent despite being set on the runner → B18 (`.azure` was in `home.forbiddenSubdirs` and auth env vars were not forwarded; fixed in github/gh-aw-firewall#6690)
 - `EACCES` + `unlink` on `/tmp/awf-...-chroot-home/<path>` during AWF cleanup (not upload-artifact) → B7 (rootless UID-remapped chroot-home files)
 - `EACCES: permission denied, mkdir '/tmp/gh-aw/...'` before containers start on a persistent runner → B8 (stale root-owned pre-flight dirs)
 - `FATAL: http_port: IPv6 is not available` → B3
@@ -129,6 +131,10 @@ B15 / github/gh-aw-firewall#6651, github/gh-aw-firewall#6657 — AWF startup fai
 
 B16 / github/gh-aw-firewall#6649, github/gh-aw-firewall#6655 — AWF rejects `--mount` entries whose host path contains an unexpanded `${VAR}` reference (for example `${TERRAFORM_CLI_PATH}/terraform:/usr/local/bin/terraform`) with a "host path must be absolute" error. The gh-aw compiler wraps `sandbox.agent.mounts` specs containing `${}` in single quotes in the generated shell invocation; Bash single quotes prevent variable substitution, so the literal string reaches AWF's volume validator instead of the resolved path. **Fixed in AWF (PR github/gh-aw-firewall#6655, merged 2026-07-27):** `expandEnvVarsInMount()` in `src/parsers/volume-parsers.ts` now expands `${VAR_NAME}` and `$VAR_NAME` patterns from `process.env` before path validation. If the variable is undefined, AWF emits a precise error (`Environment variable is not set: ${VAR_NAME}`) rather than a misleading path-absoluteness failure. Upgrade AWF to the version including github/gh-aw-firewall#6655.
 
+B17 / github/gh-aw-firewall#6704, github/gh-aw-firewall#6705 — In `--network-isolation`/topology mode, Tailscale can install a policy-routing rule covering `0.0.0.0/0`, capturing the default route and making host-specific DNS servers (Azure DHCP DNS `168.63.129.16`, Tailscale Magic DNS `100.100.100.100`, link-local `169.254.x.x`) unreachable from the Docker bridge network. Squid's DNS queries black-hole and every CONNECT fails with `503 TCP_TUNNEL:HIER_NONE` (server field `-:-`). **Fixed in AWF (PR github/gh-aw-firewall#6705, merged 2026-07-29):** `filterForNetworkIsolation()` in `src/dns-resolver.ts` strips non-portable DNS servers before generating `squid.conf` when `config.networkIsolation` is true; falls back to `8.8.8.8`/`8.8.4.4` if all detected servers are non-portable. Override via `--dns-servers` if specific DNS is required in isolation mode. Upgrade AWF to version including github/gh-aw-firewall#6705.
+
+B18 / github/gh-aw-firewall#6686, github/gh-aw-firewall#6690 — Azure CLI, Azure DevOps CLI, and Azure/ADO MCP servers fail inside the AWF agent sandbox because `.azure` was in `home.forbiddenSubdirs` (not `home.toolSubdirs`) in the canonical sandbox mount policy, and `AZURE_CONFIG_DIR`/`ADO_MCP_AUTH_TOKEN` were absent from the always-forwarded env var list. **Fixed in AWF (PR github/gh-aw-firewall#6690, merged 2026-07-28):** `.azure` added to `home.toolSubdirs` and removed from `home.forbiddenSubdirs` (applies to both compose and sbx runtimes); `AZURE_CONFIG_DIR` and `ADO_MCP_AUTH_TOKEN` added to the always-forwarded env vars in `src/services/agent-environment/env-passthrough.ts`. Upgrade AWF to version including github/gh-aw-firewall#6690.
+
 D8 / github/gh-aw-firewall#6401, github/gh-aw-firewall#6326 — Under `--container-runtime gvisor` or raw `runsc`, MCP calls to the gateway at `172.30.0.1:8080` could be misrouted through Squid and fail with `403 ERR_ACCESS_DENIED` because gVisor's userspace netstack does not use the host iptables DNAT bypass. **Fixed in AWF (PR github/gh-aw-firewall#6401)**: `runtimeUsesIptables()` now skips `awf-iptables-init` for `gvisor`, its `runsc` alias, and `sbx`, and the MCP gateway plus `host.docker.internal` are added to `NO_PROXY` for proxy-aware clients. Caveat: proxy-unaware raw sockets (for example `/dev/tcp`) still fail with `No route to host` under gVisor.
 
 D9 / github/gh-aw-firewall#6336 — sbx microVMs previously mounted the entire host `$HOME`, exposing credentials such as `~/.aws/credentials`, `~/.ssh/id_rsa`, and `~/.docker/config.json`. **Fixed in AWF (PR github/gh-aw-firewall#6336)**: sbx now mounts only whitelisted home subdirectories, and `scrubHomeCredentials()` / `restoreHomeCredentials()` temporarily move nested credential files out of the mounted tree during sandbox lifetime.
@@ -227,6 +233,8 @@ Establish these facts before matching a failure mode:
 | B14 | After upgrading to AWF including the centralized mount-policy refactor (PR github/gh-aw-firewall#6339), the Copilot CLI agent exits immediately (exit code 1, ~0.5 s, zero stdout/stderr); no network calls are made; the issue appears on Docker and gVisor runtimes but not sbx (which moves files aside rather than overlay-masking them) | PR github/gh-aw-firewall#6339 added `~/.copilot/config.json` to the credential **deny list** with reason *"Copilot CLI persisted auth token"*. This is incorrect: `config.json` contains only experiment flags, first-launch timestamps, and plugin metadata — no token. AWF masks denied files with a **read-only** `/dev/null` bind-mount overlay. The Copilot CLI reads **and atomically rewrites** `config.json` via temp-file + rename at startup; a rename cannot target a bind-mounted path and the overlay is read-only, so the CLI crashes silently. | **Fixed in AWF (PR github/gh-aw-firewall#6374, merged 2026-07-18):** `~/.copilot/config.json` is removed from the credential deny list; the file is bind-mounted read-write as part of the `.copilot` tool subdir (pre-#6339 behaviour). Upgrade to the AWF version including github/gh-aw-firewall#6374. Workaround (older AWF): pass `--keep-containers` and inspect the agent container for `EROFS` or rename errors; downgrade AWF past github/gh-aw-firewall#6339 if unpatched. | `awf ... copilot agent --version` — if it exits 1 with no output in ~0.5 s, confirm AWF version includes github/gh-aw-firewall#6374; inspect agent container logs for `EROFS`/`rename` errors on `~/.copilot/config.json` | github/gh-aw-firewall#6339, github/gh-aw-firewall#6374 |
 | B15 | AWF startup fails with `--network-isolation is not yet supported with --enable-host-access`; happens when `localhost` is in the domain allowlist because the gh-aw compiler auto-enables `--enable-host-access` and also emits `network.isolation: true` + `topologyAttach` | In `src/cli.ts`, `--network-isolation` and `--enable-host-access` are mutually exclusive with no overlap support. The gh-aw compiler (v0.82.x+) does not detect the conflict: it emits `isolation:true` + `topologyAttach` unconditionally while also auto-enabling `--enable-host-access` when `localhost` is in the allowlist. The compiled workflow always fails at runtime. | **Fixed in AWF (PR github/gh-aw-firewall#6657, merged 2026-07-28):** The mutual-exclusion guard is removed; `--enable-host-access` now coexists with `--network-isolation` in topology mode. In topology mode the agent is on an `internal` Docker network with no host route, so `--enable-host-access` only drives Squid port ACLs and the `host.docker.internal` hosts-file entry — no incompatible iptables changes. `allowHostServicePorts` (iptables-based GitHub Actions services) is still suppressed unconditionally. Upgrade AWF to version including github/gh-aw-firewall#6657. | `awf --network-isolation --enable-host-access ...` — on AWF versions before github/gh-aw-firewall#6657, reproduces immediately with `not yet supported` error; on patched AWF the combination is accepted without error; check compiler output for both flags being emitted together when `localhost` is in allowlist | github/gh-aw-firewall#6651, github/gh-aw-firewall#6657 |
 | B16 | AWF rejects a `--mount` entry with a non-absolute host path error; the mount spec was defined as `${TERRAFORM_CLI_PATH}/terraform:...` (or similar env-var pattern) but the literal string `${TERRAFORM_CLI_PATH}` reaches AWF's volume validator | The gh-aw compiler wraps `sandbox.agent.mounts` specs containing `${}` references in single quotes in the generated shell invocation. Bash single quotes prevent variable substitution, so the literal variable reference string (e.g. `${TERRAFORM_CLI_PATH}`) reaches AWF's volume validation instead of the resolved absolute path. | **Fixed in AWF (PR github/gh-aw-firewall#6655, merged 2026-07-27):** `expandEnvVarsInMount()` in `src/parsers/volume-parsers.ts` now expands `${VAR_NAME}` and `$VAR_NAME` patterns from `process.env` before path validation. If the variable is undefined, AWF emits a precise error (`Environment variable is not set: ${VAR_NAME}`) rather than a misleading path-absoluteness failure. Upgrade AWF to the version including github/gh-aw-firewall#6655. | Inspect the generated `awf` invocation in the compiled lock file — if mount specs appear in single quotes containing `${...}`, the bug is present. Running `awf ... --mount '${VAR}/path:/dest'` on AWF versions before github/gh-aw-firewall#6655 reproduces the rejection; on patched AWF the mount spec is accepted after variable expansion. | github/gh-aw-firewall#6649, github/gh-aw-firewall#6655 |
+| B17 | In `--network-isolation`/topology mode, when a workflow also brings up Tailscale as a later step, every Squid `CONNECT` to an allowlisted API host (e.g. `api.githubcopilot.com`) fails with `503 TCP_TUNNEL:HIER_NONE` (server field `-:-`, DNS never resolved); agent reports `503 Service Unavailable` retries and fails. Same workflow works on host-access mode or in isolation mode without Tailscale. | When Tailscale starts, it can install a policy-routing rule covering `0.0.0.0/0` (exit node / accepted subnet route). Docker bridge traffic on the isolation network follows that route through the Tailscale tunnel. Host-specific DNS servers reachable only via the original network path (Azure DHCP DNS `168.63.129.16`, Tailscale Magic DNS `100.100.100.100`, link-local `169.254.x.x`) become unreachable once Tailscale captures the route, so Squid's DNS queries black-hole and every CONNECT fails at resolution. | **Fixed in AWF (PR github/gh-aw-firewall#6705, merged 2026-07-29):** new `isNonPortableDns(ip)` / `filterForNetworkIsolation(servers, logger)` in `src/dns-resolver.ts` strip non-portable DNS servers (Azure DHCP DNS, Tailscale Magic DNS, link-local) before generating `squid.conf` when `config.networkIsolation` is true; falls back to `8.8.8.8`/`8.8.4.4` if all detected servers are non-portable. Users needing specific DNS in isolation mode can still override via `--dns-servers`. Upgrade to AWF version including github/gh-aw-firewall#6705. | Squid access log shows `CONNECT ... 503 TCP_TUNNEL:HIER_NONE` with server field `-:-` for an allowlisted domain, specifically after a Tailscale-up step in isolation/topology mode; inspect `audit/awf-resolved-config.json` for `networkIsolation: true` + Tailscale in the same run; check whether the DNS servers in use are Azure DHCP (`168.63.129.16`) or Tailscale Magic DNS (`100.100.100.100`) | github/gh-aw-firewall#6704, github/gh-aw-firewall#6705 |
+| B18 | Azure CLI, Azure DevOps CLI, and Azure/ADO MCP servers fail inside the AWF agent sandbox even after OIDC login completes in runner setup steps and relevant Azure/Microsoft domains are allowlisted; `~/.azure` config is invisible inside the sandbox, and `ADO_MCP_AUTH_TOKEN`/`AZURE_CONFIG_DIR` are not forwarded when `--enable-api-proxy` is active | `.azure` was not in the whitelisted `$HOME` tool subdirs (`home.toolSubdirs`) in the canonical sandbox mount policy — it was actually in `home.forbiddenSubdirs` — so Azure CLI config written by runner setup steps never reaches the sandbox; separately, `AZURE_CONFIG_DIR` and `ADO_MCP_AUTH_TOKEN` were not in the always-forwarded host env var list, so the MCP bridge could not authenticate | **Fixed in AWF (PR github/gh-aw-firewall#6690, merged 2026-07-28):** `.azure` added to `home.toolSubdirs` and removed from `home.forbiddenSubdirs` in the canonical mount policy (applies to both compose and sbx runtimes); `AZURE_CONFIG_DIR` and `ADO_MCP_AUTH_TOKEN` added to the always-forwarded env vars in `src/services/agent-environment/env-passthrough.ts`. **Authentication caveat:** #6690 deliberately scrubs Azure token cache files (`msal_token_cache*`, `accessTokens.json`, `service_principal_entries.json`) — a pre-AWF `az login` session is **not** inherited by the sandbox; only account metadata (tenant IDs, subscription list) is mounted. The agent must perform OIDC re-login inside the sandbox using a fresh writable `AZURE_CONFIG_DIR` (e.g. `az login --federated-token $ARM_OIDC_TOKEN --service-principal --username $ARM_CLIENT_ID --tenant $ARM_TENANT_ID`); see github/gh-aw-firewall#6686 for the working pattern. Upgrade AWF to version including github/gh-aw-firewall#6690 to enable this in-sandbox re-login. | Inside the agent sandbox: `ls -la ~/.azure` — presence confirms fix; `echo $AZURE_CONFIG_DIR $ADO_MCP_AUTH_TOKEN` — non-empty on patched AWF when set on the runner; `az account show` will still fail until in-sandbox OIDC re-login is performed | github/gh-aw-firewall#6686, github/gh-aw-firewall#6690 |
 
 ## Category C — GHES / GHEC / `ghe.com`
 
@@ -300,6 +308,8 @@ Establish these facts before matching a failure mode:
 | `⚠️ Firewall blocked N domain(s)` warning lists `awmgmcpg` or `172.30.0.x` as a blocked domain on every run, even with no actual external blocks | B13 (internal MCP gateway traffic counted by log aggregator as denied; fixed in github/gh-aw-firewall#6689 with `isInternalAwfDomain()` filter) |
 | `--network-isolation is not yet supported with --enable-host-access` | B15 (compiler auto-emits both flags when `localhost` in allowlist + topology; fixed in github/gh-aw-firewall#6657) |
 | AWF rejects `--mount` with "host path must be absolute" (or similar) and the path visibly contains `${VAR_NAME}` unexpanded | B16 (single-quote wrapping by compiler prevents shell expansion of `${}` in mount specs; fixed in github/gh-aw-firewall#6655) |
+| `503 TCP_TUNNEL:HIER_NONE` (server field `-:-`) on an allowlisted API host in network-isolation/topology mode, specifically after a Tailscale-up step | B17 (Tailscale policy-routing captures the default route, making host-specific DNS servers unreachable; fixed in github/gh-aw-firewall#6705 with `filterForNetworkIsolation()` stripping non-portable DNS before Squid config is generated) |
+| Azure CLI / ADO MCP auth failures with `~/.azure` missing inside AWF sandbox, or `AZURE_CONFIG_DIR`/`ADO_MCP_AUTH_TOKEN` empty inside agent despite being set on the runner | B18 (`.azure` was in `home.forbiddenSubdirs` and auth env vars were not forwarded; fixed in github/gh-aw-firewall#6690 — note: pre-AWF `az login` is not inherited; agent must perform OIDC re-login inside the sandbox, see github/gh-aw-firewall#6686) |
 
 ## Known unresolved items