docs(runner-doctor): add C7 failure mode for DIFC probe on *.ghe.com by lpcox · Pull Request #5651 · github/gh-aw-firewall

lpcox · 2026-06-28T17:29:39Z

Summary

Adds failure mode C7 to the runner doctor knowledge base, documenting the DIFC proxy liveness probe issue on data-residency *.ghe.com tenants.

Changes

.github/workflows/shared/self-hosted-failure-modes.md

Added C7 row to Category C table
Added error-string quick-lookup entry for diagnosis=unknown / reachable-but-api-error
Added C7 to "Known unresolved items"

.github/workflows/self-hosted-runner-doctor.md

Added C7 hint to §3 "Match symptom → failure mode"
Added C7 to §4 "Check for known unresolved problems"

.github/agents/self-hosted-runner-doctor.md

All five changes mirrored from the workflow and shared file

Context

On GHEC data-residency (*.ghe.com) tenants, the awf-cli-proxy DIFC probe fails because the proxy is not enterprise-host-aware. AWF ≥ v0.27.12 provides better diagnostics (reachable-but-api-error (HTTP NNN) + targeted hint) but the root cause is unresolved in companion projects.

Closes #5646

Add failure mode C7 documenting the DIFC proxy liveness probe issue on data-residency *.ghe.com tenants where diagnosis=unknown or reachable-but-api-error is reported. Updates: - shared/self-hosted-failure-modes.md: C7 row, error-string entry, known unresolved item - self-hosted-runner-doctor.md (workflow): C7 hint and unresolved note - self-hosted-runner-doctor.md (agent): all five proposed changes Closes #5646 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Updates the self-hosted “runner doctor” knowledge base to document a new failure mode (C7) affecting awf-cli-proxy DIFC-proxy liveness probing on GHEC data-residency (*.ghe.com) tenants, including symptom matching, quick lookup, and explicit unresolved-status guidance.

Changes:

Added failure mode C7 to the Category C failure-mode catalog with symptoms, root cause, mitigation guidance, and citations.
Added a new error-string quick lookup entry mapping DIFC probe diagnosis=unknown / reachable-but-api-error (HTTP NNN) on *.ghe.com to C7.
Added C7 to the “Known unresolved items” lists and to the runner-doctor playbook hints/checklist sections, mirrored across the portable agent copy.

Show a summary per file

File	Description
`.github/workflows/shared/self-hosted-failure-modes.md`	Adds C7 to the shared failure-mode catalog, quick-lookup, and unresolved list.
`.github/workflows/self-hosted-runner-doctor.md`	Adds C7 to the workflow runner-doctor playbook: symptom→mode hint and unresolved-problems section.
`.github/agents/self-hosted-runner-doctor.md`	Mirrors the shared catalog + playbook updates into the portable agent-facing runner-doctor doc.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 3/3 changed files
Comments generated: 0
Review effort level: Low

github-actions · 2026-06-28T18:47:21Z

✅ Smoke Claude passed

github-actions · 2026-06-28T18:47:21Z

✅ Smoke Copilot BYOK AOAI (api-key) completed. Copilot AOAI BYOK (api-key) mode operational. 🔓

github-actions · 2026-06-28T18:47:22Z

📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅

github-actions · 2026-06-28T18:47:23Z

✅ Contribution Check completed successfully!

Contribution guidelines review complete for PR #5651: documentation-only changes are clearly described, reference the related issue, and do not require tests under CONTRIBUTING.md.

github-actions · 2026-06-28T18:47:23Z

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

github-actions · 2026-06-28T18:47:23Z

✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓

github-actions · 2026-06-28T18:47:25Z

🔑 Smoke Copilot PAT PAT auth validated. All systems operational. ✅

github-actions · 2026-06-28T18:47:25Z

Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.

github-actions · 2026-06-28T18:47:27Z

✅ Smoke Gemini completed. All facets verified. 💎

Testing safeoutputs

github-actions · 2026-06-28T18:47:28Z

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

github-actions · 2026-06-28T18:47:32Z

✅ Smoke Copilot BYOK AOAI (Entra) completed. Copilot AOAI BYOK (Entra) mode operational. 🔓

github-actions · 2026-06-28T18:47:40Z

🚀 Security Guard has started processing this pull request

github-actions · 2026-06-28T18:47:45Z

🔌 Smoke Services — All services reachable! ✅

github-actions · 2026-06-28T18:48:04Z

✅ Build Test Suite completed successfully!

github-actions · 2026-06-28T18:50:30Z

Smoke Test: Claude Engine Validation

API check: ✅ PASS
gh check: ✅ PASS
File check: ✅ PASS

Overall result: PASS

Generated by Smoke Claude for #5651 · 58.2 AIC · ⊞ 3.3K · ◷

github-actions · 2026-06-28T18:50:42Z

🤖 Smoke Test Results

Test	Status
GitHub MCP connectivity	✅
GitHub.com HTTP reachability	❌ pre-step data unavailable
File write/read	❌ pre-step data unavailable

Overall: FAIL — pre-step outputs (SMOKE_HTTP_CODE, SMOKE_FILE_PATH, SMOKE_FILE_CONTENT) were not expanded; template variables passed unexpanded to agent.

PR: docs(runner-doctor): add C7 failure mode for DIFC probe on *.ghe.com by @lpcox

📰 BREAKING: Report filed by Smoke Copilot

github-actions · 2026-06-28T18:51:06Z

Smoke test results for Copilot BYOK (Direct) Mode:

docs(runner-doctor): add C7 failure mode for DIFC probe on *.ghe.com: ✅
fix: return 429 instead of 403 when max turns exceeded: ✅
GitHub.com connectivity: ✅
File I/O: ✅
BYOK inference path: ✅

Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw)
Overall: PASS

cc @lpcox

🔑 BYOK (AOAI api-key) report filed by Smoke Copilot BYOK AOAI (api-key)

github-actions · 2026-06-28T18:51:07Z

🔥 Smoke Test: PAT Auth — PR #5651

Test	Result
GitHub MCP connectivity	✅ Connected (fetched PR #5641)
GitHub.com HTTP connectivity	⚠️ Pre-step data unavailable (template not substituted)
File write/read	⚠️ Pre-step data unavailable (template not substituted)

Overall: PARTIAL — MCP auth confirmed; pre-step outputs were not injected into the workflow prompt.

@lpcox — Auth mode: PAT (COPILOT_GITHUB_TOKEN)

🔑 PAT report filed by Smoke Copilot PAT

github-actions · 2026-06-28T18:51:11Z

@lpcox

Smoke Test Results:
• GitHub MCP: ✅
• GitHub.com: ✅
• File I/O: ✅
• BYOK Inference: ✅

Running in direct BYOK mode (AWF_AUTH_TYPE=github-oidc + AWF_AUTH_AZURE_* + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw).

Overall: PASS

🪪 BYOK (AOAI Entra) report filed by Smoke Copilot BYOK AOAI (Entra)

github-actions · 2026-06-28T18:51:16Z

Smoke Test

refactor: extract shared auth header resolution helper for provider adapters ✅
refactor: deduplicate OIDC auth env var mappings via shared constant ✅
GitHub homepage title ✅
Temp file creation ✅
Build ✅
Overall: PASS

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex

github-actions · 2026-06-28T18:51:21Z

Smoke Test: Copilot BYOK (Direct) Mode

Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY via api-proxy → api.githubcopilot.com)

Test Results

✅ GitHub MCP connectivity (fetched 2 merged PRs)
✅ GitHub.com HTTP code (200)
✅ File write/read test (temp I/O working)
✅ BYOK inference test (current response via api-proxy)

Overall Status: PASS

/cc @lpcox

🔑 BYOK report filed by Smoke Copilot BYOK

github-actions · 2026-06-28T18:51:31Z

🔭 Smoke Test: API Proxy OpenTelemetry Tracing

Scenario	Result	Notes
S1: Module Loading	✅	`otel.js` loads successfully; `isEnabled()` → `true`; exports: `startRequestSpan`, `setTokenAttributes`, `setBudgetAttributes`, `endSpan`, `endSpanError`, `shutdown`, `isEnabled`
S2: Test Suite	✅	2 suites (`otel.test.js`, `otel-fanout.test.js`) — 59/59 tests passed
S3: Env Var Forwarding	✅	`src/services/api-proxy-env-config.ts` forwards `GH_AW_OTLP_ENDPOINTS`, `OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_EXPORTER_OTLP_HEADERS`, `GITHUB_AW_OTEL_TRACE_ID`, `GITHUB_AW_OTEL_PARENT_SPAN_ID`, `OTEL_SERVICE_NAME` to the api-proxy container
S4: Token Tracker Integration	✅	`onUsage` callback present in `token-tracker-http.js` (line 324); invoked post-normalization as the OTEL hook point
S5: OTEL Diagnostics	✅	No OTLP endpoint configured → graceful fallback to `/var/log/api-proxy/otel.jsonl`; no errors

All 5 scenarios pass. OTEL tracing integration is functioning correctly.

📡 OTel tracing validated by Smoke OTel Tracing

github-actions · 2026-06-28T18:53:18Z

Chroot Version Comparison Results

Runtime	Host Version	Chroot Version	Match?
Python	`Python 3.12.13`	`Python 3.12.3`	❌ NO
Node.js	`v24.17.0`	`v22.23.0`	❌ NO
Go	`go1.22.12`	`go1.22.12`	✅ YES

Overall: ❌ FAILED — Python and Node.js versions differ between host and chroot environments. The smoke-chroot label was not applied.

Tested by Smoke Chroot

github-actions · 2026-06-28T18:53:21Z

Smoke test PASSED

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

localhost

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "localhost"

See Network Configuration for more information.

💎 Faceted by Smoke Gemini

github-actions · 2026-06-28T18:53:39Z

Smoke Test Results — Services Connectivity

Check	Result
Redis PING	❌ No response (timeout)
PostgreSQL pg_isready	❌ `no response`
PostgreSQL SELECT 1	❌ No response (timeout)

Overall: FAIL — host.docker.internal is unreachable from this sandbox environment.

🔌 Service connectivity validated by Smoke Services

github-actions · 2026-06-28T18:54:07Z

🏗️ Build Test Suite Results

Ecosystem	Project	Build/Install	Tests	Status
Bun	elysia	✅	1/1 passed	✅ PASS
Bun	hono	✅	1/1 passed	✅ PASS
C++	fmt	✅	N/A	✅ PASS
C++	json	✅	N/A	✅ PASS
Deno	oak	N/A	1/1 passed	✅ PASS
Deno	std	N/A	1/1 passed	✅ PASS
.NET	hello-world	✅	N/A	✅ PASS
.NET	json-parse	✅	N/A	✅ PASS
Go	color	✅	1/1 passed	✅ PASS
Go	env	✅	1/1 passed	✅ PASS
Go	uuid	✅	1/1 passed	✅ PASS
Java	gson	✅	1/1 passed	✅ PASS
Java	caffeine	✅	1/1 passed	✅ PASS
Node.js	clsx	✅	All passed	✅ PASS
Node.js	execa	✅	All passed	✅ PASS
Node.js	p-limit	✅	All passed	✅ PASS
Rust	fd	✅	1/1 passed	✅ PASS
Rust	zoxide	✅	1/1 passed	✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Generated by Build Test Suite for #5651 · 50 AIC · ⊞ 7.8K · ◷

Copilot AI review requested due to automatic review settings June 28, 2026 17:29

Copilot started reviewing on behalf of lpcox June 28, 2026 17:30 View session

Copilot AI reviewed Jun 28, 2026

View reviewed changes

lpcox added the ready-for-aw label Jun 28, 2026

lpcox temporarily deployed to aoai-model June 28, 2026 18:47 — with GitHub Actions Inactive

github-actions Bot added the smoke-claude label Jun 28, 2026

lpcox temporarily deployed to aoai-model June 28, 2026 18:50 — with GitHub Actions Inactive

github-actions Bot added the smoke-copilot-byok-aoai-apikey label Jun 28, 2026

github-actions Bot added the smoke-copilot-byok-aoai-entra label Jun 28, 2026

lpcox temporarily deployed to aoai-model June 28, 2026 18:51 — with GitHub Actions Inactive

github-actions Bot added the smoke-codex label Jun 28, 2026

github-actions Bot mentioned this pull request Jun 28, 2026

[aw] No-Op Runs #5368

Open

github-actions Bot added the smoke-copilot-byok label Jun 28, 2026

github-actions Bot added the smoke-gemini label Jun 28, 2026

github-actions Bot added the build-test label Jun 28, 2026

lpcox merged commit 6055a2e into main Jun 28, 2026
80 of 81 checks passed

lpcox deleted the docs-runner-doctor-c7-ghe-difc branch June 28, 2026 19:04

Uh oh!

Conversation

lpcox commented Jun 28, 2026

Summary

Changes

Context

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Review details

Uh oh!

github-actions Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 28, 2026

Uh oh!

github-actions Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 28, 2026

Smoke Test: Claude Engine Validation

Uh oh!

github-actions Bot commented Jun 28, 2026

🤖 Smoke Test Results

Uh oh!

github-actions Bot commented Jun 28, 2026

Uh oh!

github-actions Bot commented Jun 28, 2026

🔥 Smoke Test: PAT Auth — PR #5651

Uh oh!

github-actions Bot commented Jun 28, 2026

Uh oh!

github-actions Bot commented Jun 28, 2026

Smoke Test

Uh oh!

github-actions Bot commented Jun 28, 2026

Smoke Test: Copilot BYOK (Direct) Mode

Test Results

Uh oh!

github-actions Bot commented Jun 28, 2026

🔭 Smoke Test: API Proxy OpenTelemetry Tracing

Uh oh!

github-actions Bot commented Jun 28, 2026

Chroot Version Comparison Results

Uh oh!

github-actions Bot commented Jun 28, 2026

github-actions Bot commented Jun 28, 2026 •

edited

Loading

github-actions Bot commented Jun 28, 2026 •

edited

Loading

github-actions Bot commented Jun 28, 2026 •

edited

Loading

github-actions Bot commented Jun 28, 2026 •

edited

Loading

github-actions Bot commented Jun 28, 2026 •

edited

Loading

github-actions Bot commented Jun 28, 2026 •

edited

Loading

github-actions Bot commented Jun 28, 2026 •

edited

Loading

github-actions Bot commented Jun 28, 2026 •

edited

Loading

github-actions Bot commented Jun 28, 2026 •

edited

Loading

github-actions Bot commented Jun 28, 2026 •

edited

Loading

github-actions Bot commented Jun 28, 2026 •

edited

Loading

github-actions Bot commented Jun 28, 2026 •

edited

Loading

github-actions Bot commented Jun 28, 2026 •

edited

Loading