[CI Failure Doctor] Strict mode firewall validation disallows custom network domains and breaks strict mode tests

[github-actions]

🏥 CI Failure Investigation - Run #35342

Summary

make recompile plus the integration suites started failing after fix: pass shared temporary ID map (#15371) because the new strict-mode firewall validation refuses any custom network.allowed entries when the engine is copilot (no LLM gateway support).

Failure Details

• Run: 35342
• Commit: ec99734 (fix: pass shared temporary ID map (#15371))
• Trigger: push

Root Cause Analysis

The commit rewrote pkg/workflow/strict_mode_validation.go with a validateStrictFirewall helper that now enforces the firewall sandbox for strict mode copilot/codex: it rejects network.allowed entries unless they map to built-in ecosystems (lines ~337‑378). The strict-mode tests in pkg/workflow/strict_mode_test.go still configure network.allowed: ["api.example.com"] while compiler strict mode is enabled, so compiler.CompileWorkflow now returns the new error: strict mode: engine 'copilot' does not support LLM gateway and requires network domains to be from known ecosystems (e.g., 'defaults', 'python', 'node'). Custom domains are not allowed for security.

Failed Jobs and Errors

• build → ./gh-aw compile --validate --verbose --purge --stats exits with compilation failed because the workflow validation now enforces the new strict-mode firewall rules.
• Integration: Workflow GitHub & Git → TestStrictModeAllowsGitHubWorkflowExpression variants all fail with the same copilot firewall rejection (see strict_mode_test.go:865‑935).
• Integration: Workflow Misc Part 2 → TestStrictModeNetwork and the TestStrictModeBashTools variants fail for the same reason (logs around strict_mode_test.go:364 and strict_mode_test.go:593).
• Integration: Workflow Tools & MCP → identical copilot strict-mode network/domain failures.

Investigation Findings

Every failure traces back to the new firewall validation rejecting network.allowed lists that include api.example.com when compiler.SetStrictMode(true) runs. The integration suites exercise those test helpers, so the new security guard rails are currently incompatible with the existing strict-mode regression tests.

Recommended Actions

• Update the strict-mode regression tests (pkg/workflow/strict_mode_test.go) to use known ecosystems/defaults instead of a custom api.example.com domain so that the tests stay green under the new firewall rules.
• Document the copilot/codex restriction (custom domains disallowed when strict mode is on and LLM gateway is unsupported) in the changelog or reference docs so future test changes account for it.
• Add a regression case (or adjust the existing helpers) to assert that the new validateStrictFirewall path rejects custom domains so the behavior stays visible when the policy changes again.

Prevention Strategies

Any future modification to strict-mode firewall policy needs to include an audit of network.allowed test fixtures and sample workflows to ensure they still comply with the updated list of allowed ecosystems. Adding a targeted test for the new error message will also keep the regression visible on the next run.

AI Team Self-Improvement

When editing strict-mode validation logic, remind future AI prompts to cross-check pkg/workflow/strict_mode_test.go and any example workflows for custom network.allowed entries and update them to the known ecosystems (defaults, python, node, etc.) before landing the change.

Historical Context

This is the first time the validation has failed in CI because it landed in fix: pass shared temporary ID map (#15371); there were no prior failures of this pattern on main.

AI generated by CI Failure Doctor

To add this workflow in your repository, run gh aw add githubnext/agentics/workflows/ci-doctor.md@ea350161ad5dcc9624cf510f134c6a9e39a6f94d. See usage guide.

• expires on Feb 14, 2026, 10:21 AM UTC

[github-actions]

Run #35345 (commit f3c6d05) also exhibits the same strict-mode firewall rejection as described above. The Integration: Workflow Permissions job fails because TestPermissionsWarningInNonStrictMode/sufficient_permissions_in_strict_mode_produces_no_error now reports strict mode: engine 'copilot' does not support LLM gateway and requires network domains to be from known ecosystems (e.g., 'defaults', 'python', 'node'). Custom domains are not allowed for security. make recompile aborts with compilation failed as a result. The new failure merely confirms that the regression is still blocking the integration suite.

AI generated by CI Failure Doctor

[github-actions]

Additional Context from Workflow Health Assessment

The strict mode firewall validation introduced in commit ec99734 is currently blocking 7 workflows from compiling, causing a critical degradation in workflow health (54/100, down from 95/100).

Confirmed Affected Workflows

Based on Agentic Maintenance run §21984242074, the following workflows are failing compilation:

1. blog-auditor.md

   • Engine: claude
   • Strict: true
   • Custom domains: githubnext.com, www.githubnext.com

2. cli-consistency-checker.md

   • Engine: copilot
   • Custom domains: api.github.com

3. cli-version-checker.md

   • Engine: claude
   • Strict: true
   • Custom domains: api.github.com, ghcr.io

4. +4 more workflows (need to identify from logs)

Impact Assessment

• Compilation coverage: 95.3% (142/149 workflows) - first time below 100% in 5 days
• Health score: 54/100 (critical degradation of -41 points)
• Agentic Maintenance workflow: Failing due to compilation errors
• CI workflow: Failing due to strict mode test incompatibilities
• Release status: NOT production-ready (BLOCKING)

Recommended Resolution Path

Option 1: Quick fix (disable strict mode)

# For workflows that need custom domains
strict: false
network:
  allowed: [defaults, "api.github.com", "ghcr.io"]

Option 2: Use ecosystem shortcuts (preferred for security)

# Map custom domains to ecosystem shortcuts
strict: true
network:
  allowed: [defaults, node]  # api.github.com covered by 'node'

Next Steps

1. Identify remaining 4 workflows from agentic-maintenance logs
2. Update all 7 workflows with appropriate fix (Option 1 or 2)
3. Run gh aw compile --validate to verify all workflows compile
4. Document migration guide for future workflow authors
5. Update strict mode tests to use ecosystem shortcuts instead of custom domains

This issue is now P0 BLOCKING for production release. The ecosystem cannot achieve 100% compilation coverage until these workflows are fixed.

---

Context provided by Workflow Health Manager - Meta-Orchestrator

AI generated by Workflow Health Manager - Meta-Orchestrator

[github-actions]

This issue is being closed as outdated. A newer issue has been created: #15391

View newer issue

---

This action was performed automatically by the CI Failure Doctor workflow.

[github-actions]

Run #35348 (commit c1ea137f44b290aa59ad9f8b9bb00df7f6975907) hits the same Integration: Workflow Misc Part 2 failures. The strict-mode firewall still rejects custom network.allowed entries for Copilot/Claude, so the same suites (TestAllowedDomainsFromNetworkConfig, TestStrictModeTimeout/Permissions/Network/BashTools, etc.) abort before they can assert anything, printing strict mode: engine 'copilot' does not support LLM gateway and requires network domains to be from known ecosystems.... This is a replay of run #35342, so we continued the investigation here rather than splitting it. Please keep tracking this issue while the test fixtures are aligned with the new firewall policy.

AI generated by CI Failure Doctor