[windows-integration] Improve windows-cli-integration.yml: matrix coverage, chaos scenarios, and missing test dimensions

[github-actions]

Summary

.github/workflows/windows-cli-integration.yml exists with a solid foundation (build/integration/conclusion jobs, SHA-pinned actions, timeouts in pwsh, stdin-hang detection) but is missing several required coverage dimensions. This issue tracks the improvements needed.

---

Missing requirements

1. Systematic matrix instead of ad-hoc spot checks

The current integration job uses ad-hoc steps (pwsh, powershell, cmd). It must be refactored to treat scenario coverage as a structured matrix of:

• shell × launch-mode × environment shape × path style

Concretely: drive scenarios with a strategy.matrix or a consolidated script that iterates all combinations, so new dimensions (e.g., a new shell or a new env toggle) can be added in one place without duplicating step blocks.

---

2. No-profile shells not covered

Add tests for:

- name: "[pwsh -NoProfile] --help"
  shell: pwsh
  run: |
    $proc = Start-Process -FilePath $env:BINARY -ArgumentList "--help" `
      -PassThru -NoNewWindow
    if (-Not $proc.WaitForExit(30000)) { $proc.Kill(); throw "TIMEOUT" }
    if ($proc.ExitCode -ne 0) { throw "Exit $($proc.ExitCode)" }

Required variants:

• pwsh -NoProfile -Command "& <binary> --help"
• powershell -NoProfile -Command "& <binary> --help"
• Default-profile variants for comparison (already present but should be labeled explicitly)

---

3. PATH order and PATH shadowing quirks not covered

Add steps that verify:

• workspace-first PATH: prepend ${{ github.workspace }} to PATH and confirm the correct binary is resolved
• toolcache-first PATH: prepend a toolcache-like directory to PATH and confirm no shadow collision
• duplicate PATH entries: add the workspace directory twice; confirm no error or unexpected behavior

Example:

- name: PATH shadowing – workspace-first
  shell: pwsh
  run: |
    $env:PATH = "$env:GITHUB_WORKSPACE;$env:PATH"
    $proc = Start-Process -FilePath "gh-aw.exe" -ArgumentList "--help" -PassThru -NoNewWindow
    if (-Not $proc.WaitForExit(30000)) { $proc.Kill(); throw "TIMEOUT" }

---

4. PATHEXT and extension resolution behavior not covered

Verify .exe resolution behavior across shells:

- name: PATHEXT – .exe from pwsh
  shell: pwsh
  run: |
    # Confirm PATHEXT contains .exe
    if ($env:PATHEXT -notmatch '\.EXE') { throw "PATHEXT missing .exe" }
    # Invoke without directory prefix to confirm PATH lookup
    ...

Also cover: invoking gh-aw (no .exe) from cmd and pwsh to confirm extension resolution.

---

5. Paths with spaces and mixed slash styles not covered

Add steps that:

• Copy the binary to C:\path with spaces\gh-aw.exe (or a temp dir with spaces)
• Invoke with both \\ and / path separators
• Invoke with paths containing parentheses (mirrors C:\Program Files (x86)\...)

---

6. Unicode and non-ASCII working directory not covered

- name: Unicode working directory
  shell: pwsh
  run: |
    $dir = Join-Path $env:TEMP "tëst-dïr-αβγ"
    New-Item -ItemType Directory -Force -Path $dir | Out-Null
    Push-Location $dir
    $proc = Start-Process -FilePath $env:BINARY -ArgumentList "--help" -PassThru -NoNewWindow
    if (-Not $proc.WaitForExit(30000)) { $proc.Kill(); throw "TIMEOUT" }
    Pop-Location

---

7. Environment toggles not covered

Required coverage:

Toggle	Value	Expectation
NO_COLOR	1	Help output should still complete successfully
TERM	dumb	No ANSI sequences, no hang
TERM	(unset)	Default behavior
CI	true	Matches GitHub Actions environment
CI	(unset)	Verify no CI-only code paths hang without CI=true
Minimal env	only PATH+TEMP+TMP+SystemRoot	Confirm binary doesn't require undocumented env vars

---

8. Command invocation variants not fully covered

Currently tested: --help, version, compile --help.

Missing:

• gh-aw.exe help (positional help subcommand)
• gh-aw.exe run --help
• Unknown/invalid subcommand (negative case — see §9)

---

9. Negative/chaos case missing

At least one step must intentionally invoke an unknown command and assert:

• Exit code is non-zero
• Output contains an error message
• Completion is within the timeout (no hang on bad input)

- name: "[chaos] unknown subcommand exits fast and explicit"
  shell: pwsh
  run: |
    $proc = Start-Process -FilePath $env:BINARY -ArgumentList "totally-unknown-subcommand-xyz" `
      -PassThru -NoNewWindow -RedirectStandardOutput $env:TEMP\stdout.txt `
      -RedirectStandardError $env:TEMP\stderr.txt
    if (-Not $proc.WaitForExit(10000)) {
      $proc.Kill()
      throw "HANG: unknown subcommand did not exit within 10s"
    }
    if ($proc.ExitCode -eq 0) { throw "WRONG: unknown subcommand exited 0" }
    $out = (Get-Content $env:TEMP\stderr.txt -Raw) + (Get-Content $env:TEMP\stdout.txt -Raw)
    if (-Not ($out -match "unknown|not found|invalid|unrecognized")) {
      throw "MISSING: error output does not describe the unknown command"
    }
    Add-Content $env:GITHUB_STEP_SUMMARY "✅ Unknown subcommand fails fast, explicitly, and with a useful message"

---

10. cmd steps lack per-command timeouts

The cmd steps rely only on the job-level timeout-minutes: 20. They should use cmd /c "timeout-wrapper" or be replaced with pwsh-driven ProcessStartInfo calls with explicit WaitForExit to surface hangs clearly.

---

Acceptance criteria

• All 10 gaps above are addressed in .github/workflows/windows-cli-integration.yml
• Scenario coverage is structured as a matrix (or equivalent loop) rather than duplicated step blocks
• At least one negative/chaos case verifies fast, explicit, debuggable failure
• All cmd steps have per-command hang protection
• make recompile is run after any .md workflow source changes (if applicable)
• Workflow passes lint (make agent-report-progress or equivalent)

Reference

Workflow file: .github/workflows/windows-cli-integration.yml
Run that triggered this audit: https://github.com/github/gh-aw/actions/runs/27327772094

Generated by 🪟 Daily Windows Terminal Integration Builder · 41.4 AIC · ⌖ 13.7 AIC · ⊞ 21.9K · ◷