Optimize integration test matrix: isolate slow Docker build test

[Copilot]

The TestDockerBuild_WithMake test takes 38.4s and was blocking the critical path in the "CLI Completion & Other" catch-all group (67.7s total). Isolating it into a dedicated parallel group reduces the critical path to ~40s (41% improvement).

Changes

.github/workflows/ci.yml

• Added new matrix entry "CLI Docker Build" matching TestDockerBuild pattern
• Updated "CLI Completion & Other" skip pattern to exclude TestDockerBuild

- name: "CLI Docker Build"  # Isolate slow Docker build test (~38s)
  packages: "./pkg/cli"
  pattern: "TestDockerBuild"

Impact

• Matrix groups: 31 → 32 (+1)
• Critical path: 67.7s → ~40s (-41%)
• Time saved: ~28s per CI run

Original prompt

---

This section details on the original issue you should resolve

<issue_title>[ci-coach] Optimize integration test matrix by isolating slow Docker build test</issue_title>
<issue_description>## CI Optimization: Reduce Integration Test Critical Path by 41%

Summary

This PR optimizes the integration test matrix by isolating the slow TestDockerBuild tests into a dedicated matrix group, reducing the critical path from 67.7s to ~40s (41% improvement).

Problem Analysis

The "CLI Completion & Other" catch-all group was the slowest integration test group at 67.7s, with a single test dominating execution time:

• TestDockerBuild_WithMake: 38.43s (57% of group time)
• Other Docker tests: ~0.4s combined
• Total Docker tests: 3 tests, 38.4s

This bottleneck blocked the critical path, preventing faster test groups from completing earlier.

Solution

Created a new dedicated matrix group for Docker build tests:

1. New group: "CLI Docker Build"

   • Pattern: TestDockerBuild
   • Expected duration: ~38-40s
   • Runs in parallel with other groups

2. Updated "CLI Completion & Other" skip pattern

   • Added TestDockerBuild to exclusions
   • Reduced from 67.7s to ~29s (estimated)

Changes

File: .github/workflows/ci.yml

Added new matrix entry (after line 107):

- name: "CLI Docker Build"  # Isolate slow Docker build test (~38s)
  packages: "./pkg/cli"
  pattern: "TestDockerBuild"

Updated "CLI Completion & Other" skip pattern (line 111):

skip_pattern: "...|^TestAudit|^TestInspect|TestDockerBuild"

Expected Impact

Before:

• Total matrix groups: 31
• Critical path: 67.7s (CLI Completion & Other)
• Docker tests block other test completion

After:

• Total matrix groups: 32 (+1)
• Critical path: ~40s (CLI Docker Build, runs in parallel)
• Improvement: 41% faster critical path
• Time saved: ~28 seconds per CI run

Cost-Benefit Analysis:

• Impact: High (41% speedup on integration tests)
• Effort: Low (4 line change, no logic changes)
• Risk: Low (existing tests, just rebalancing execution)
• Reversibility: High (easy to revert if issues arise)

Validation Results

✅ YAML syntax valid: Skip pattern correctly includes TestDockerBuild
✅ Matrix count: 32 groups (31 → 32, expected)
✅ CLI groups: 14 CLI test groups (13 → 14, expected)
✅ Pattern isolation: Docker tests will run in dedicated group

Note: Full validation with make lint && make build && make test-unit && make recompile cannot be performed in this workflow environment (no Go/Make available), but YAML structure has been verified.

Testing Plan

• Verify workflow syntax passes GitHub Actions validation
• Monitor first CI run after merge:
  • CLI Docker Build group should take ~38-40s
  • CLI Completion & Other should drop to ~25-30s
  • Critical path should be ~40s (down from 67.7s)
• Compare with historical baselines from previous runs
• Easy rollback: Revert this single commit if issues arise

Metrics Baseline (for comparison)

Current state (from CI run analysis):

• Success rate: 71% (71 success, 26 cancelled, 3 action_required out of 100 runs)
• Matrix groups: 31
• Longest integration group: 67.7s (CLI Completion & Other)

Expected post-merge:

• Matrix groups: 32
• Longest integration group: ~40s (CLI Docker Build)
• CLI Completion & Other: ~29s (reduced from 67.7s)
• Time saved per run: ~28s

Analysis Methodology

This optimization was identified through systematic analysis of CI run data:

1. Analyzed: 100 recent workflow runs
2. Examined: Integration test artifacts with per-test timing from successful runs
3. Identified: TestDockerBuild_WithMake taking 38.43s (57% of group time)
4. Root cause: Single slow test in large catch-all group creating bottleneck
5. Solution: Isolate slow tests into dedicated parallel group

View Detailed Test Timing Analysis

CLI Completion & Other Group (67.7s total)

Top 10 slowest tests:

1. TestDockerBuild_WithMake: 38.43s ⚠️ (isolated by this PR)
2. TestCollectWorkflowFiles_NoLockFile: 3.14s
3. TestCompileWorkflowsWithNoEmit: 3.13s
4. TestCompileWorkflows_TrialMode: 3.03s
5. TestInteractiveWorkflowBuilder_compileWorkflow_SpinnerIntegration: 1.85s
6. TestCompileWorkflows/nonexistent_specific_file: 1.62s
7. TestCompileWorkflows: 1.62s
8. TestDownloadRunArtifactsConcurrent_ConcurrencyLimit: 0.70s
9. TestCompileWithFuzzyDailyScheduleDeterministic: 0.45s
10. TestInstallPackage: 0.41s

Test grouping:

• Docker: 3 tests, 38.4s (isolated by this PR)
• Compile: 54 tests, 12.9s
• Collect: 17 tests, 3.2s
• Completion: 45 tests, 2.1s
• Interactive: 17 tests, 1.9s
• Other: ~2,300 tests, ~11s

After this PR:

• CLI Docker Build: 3 tests, ~38-40s (new dedicated group)
• CLI Completion & Other: 2,440 tests, ~29s (reduced from 67.7s)

---

References:

• Analysis workfl...

• Fixes [ci-coach] Optimize integration test matrix by isolating slow Docker build test #11509

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.