[Visual Test] Label pagination aborts workflow before screenshot capture on run #30984849473

[MichaelFisher1997]

Workflow run

https://github.com/OpenStaticFish/ZigCraft/actions/runs/30984849473

Head SHA: 8d205ff0b27a951ac7136cb72796ecd8e2e91ed4 (the only commit on dev: the nix flake → devenv migration, PR #951)

Exact failure

The failure is not in Vulkan, the headless swapchain, or screenshot readback. The game command never ran, so build-output.log was never created. The diagnose agent found the workspace contained only weston.log (weston startup, then caught signal 15 from the Stop headless Wayland compositor step), no screenshot.png, and no captured game log.

The failing step is Ensure visual-test label exists at .github/workflows/visual-test.yml:55-66. It runs:

if ! gh label list --json name --jq '.[].name' | grep -q '^visual-test$'; then
  gh label create "visual-test" \
    --description "Issues from automated visual regression tests" \
    --color "E06C75"
fi
if ! gh label list --json name --jq '.[].name' | grep -q '^run-visual-test$'; then
  gh label create "run-visual-test" \
    --description "Run deterministic visual regression workflow on a PR" \
    --color "E06C75"
fi

gh label list returns the GitHub default 30 labels per page. This repository currently has 41 labels (verified locally with gh label list --repo OpenStaticFish/ZigCraft --limit 100 --json name). visual-test is at index 22 of 41 (visible on page 1), but run-visual-test is at index 31 of 41 (only on page 2). The precheck at visual-test.yml:62 therefore reports run-visual-test as missing even though it exists. The subsequent gh label create then fails with:

label with name "run-visual-test" already exists; use `--force` to update its color and description

Because the step uses set -euo pipefail (the gh CLI exits non-zero on a duplicate create), the step exits 1. Subsequent steps (Setup Lavapipe Vulkan, Run menu screenshot capture, the screenshot existence check, the golden-image diff, and the artifact uploads) are skipped. Only the Stop headless Wayland compositor step runs (it has if: always()), and it kills the weston process with SIGTERM — which is the signal recorded in weston.log:62.

Why there is no Vulkan, swapchain, or screenshot error

VulkanSwapchain.createSwapchain at modules/engine-graphics/src/vulkan_swapchain.zig:127, the headless color-attachment image setup at the same file lines 128-174, screenshot-mode init at src/game/app.zig:268-276, HomeScreen.init at modules/game-ui/src/screens/home.zig:33-40, and screenshot.requestCapture at modules/engine-graphics/src/vulkan/screenshot.zig:24-44 were never executed. The Zig binary was never launched; only the YAML pipeline ran.

Step state from the run

Pulled from the GitHub Actions jobs API for run 30984849473:

#	Step	Status	Conclusion
1-7	Set up job → Load visual diagnosis prompt	completed	success
8	Ensure visual-test label exists	completed	failure
9	Setup Lavapipe Vulkan	completed	skipped
10	Run menu screenshot capture	completed	skipped
11	Check screenshot exists	completed	success (screenshot_exists=false)
12	Compare against golden image	completed	skipped
13	Upload screenshot artifact	completed	skipped
14	Check build log exists	completed	success (file absent)
15	Upload build log artifact	completed	skipped
16	Stop headless Wayland compositor	completed	success
17	Run opencode visual verification	completed	skipped
18	Run opencode failure diagnosis	in_progress	—

Root cause file/function

• File: .github/workflows/visual-test.yml
• Function/step: Ensure visual-test label exists at lines 55-66
• The precheck at line 62 (gh label list --json name --jq '.[].name' | grep -q '^run-visual-test$') silently truncates to 30 labels and treats the existing run-visual-test label as absent.
• The unconditional-on-miss create at lines 63-65 then fails with a non-zero exit.

Additional observations

• This is a recurring regression: issue [Visual Test] Label pagination aborts workflow before screenshot capture #968 (run #30795834741) and issue [Visual Test] Label preflight pagination aborts workflow before screenshot capture on run #30887567592 #970 (run #30887567592) describe the same root cause. Both are still open, and the Ensure visual-test label exists step in visual-test.yml has not been modified since the devenv migration commit 8d205ff (the only commit on dev), so the regression has re-fired on every nightly schedule run since.
• The diagnosis prompt is also stale (out of scope for this fix, but worth a follow-up): .github/prompts/visual-test-diagnose.md:7 still says screenshot.ppm, while the actual command at .github/workflows/visual-test.yml:81 uses the supported screenshot.png. The current encoder would still reject .ppm at modules/engine-graphics/src/vulkan/screenshot.zig:39 (detectScreenshotFormat only accepts .png, .jpg, .jpeg, .gif, .webp) if a future run ever reverted to .ppm. The prompt also references src/game/screens/home.zig, but the HomeScreen now lives at modules/game-ui/src/screens/home.zig.

Suggested fix

Make the bootstrap idempotent without relying on paginated list output. Apply in .github/workflows/visual-test.yml at lines 55-68:

- name: Ensure visual-test labels exist
  run: |
    gh label create "visual-test" \
      --description "Issues from automated visual regression tests" \
      --color "E06C75" \
      --force
    gh label create "run-visual-test" \
      --description "Run deterministic visual regression workflow on a PR" \
      --color "E06C75" \
      --force
  env:
    GH_TOKEN: ${{ secrets.OPENCODE_PAT }}

--force makes gh label create succeed on existing labels (updating color/description in place) and eliminates the paginated precheck entirely.

Alternatives if a precheck is preferred: query each exact label through gh api repos/$GITHUB_REPOSITORY/labels/<name> (single-resource GET, no pagination), or pass --limit 100 to the existing gh label list call. --limit masks the immediate bug but reintroduces a hidden threshold as the label catalog grows, so --force is the more robust fix.

[github-actions]

Bug — CI workflow regression. Report validated.

I read through the relevant files and the analysis in the issue is accurate:

• .github/workflows/visual-test.yml:55-66 — gh label list paginates at 30 by default. Verified the repo currently has 41 labels, so run-visual-test (index 31) is invisible to the precheck at line 62, and the subsequent gh label create at line 63 exits non-zero under set -euo pipefail, which aborts the workflow before any Vulkan/swapchain/screenshot code runs.
• .github/prompts/visual-test-diagnose.md:7 still says screenshot.ppm, line 29 still points to src/game/screens/home.zig, and the workflow at visual-test.yml:81 now uses screenshot.png from modules/game-ui/src/screens/home.zig. Confirmed the encoder would reject .ppm at modules/engine-graphics/src/vulkan/screenshot.zig:39-41 (detectScreenshotFormat accepts only .png/.jpg/.jpeg/.gif/.webp).

Suggested fix is sound. Replacing the two precheck + gh label create blocks with unconditional --force creates is the right call: it removes the pagination dependency entirely and stays idempotent if either label is renamed or recolored later. I'd recommend the exact patch shown in the issue at visual-test.yml:55-68 — using the single Ensure visual-test labels exist step with two gh label create … --force invocations and no if ! … | grep -q precheck.

Scope. This should be a single, narrow PR touching only visual-test.yml. Two follow-ups are worth filing but should not be bundled here:

1. Stale diagnosis prompt — .github/prompts/visual-test-diagnose.md needs to match reality: screenshot.ppm → screenshot.png, src/game/screens/home.zig → modules/game-ui/src/screens/home.zig, and remove the "PPM is then converted to PNG" line. Otherwise the diagnose agent will keep writing paths that no longer exist and waste runs on confused reports.
2. Same root cause on related issues — [Visual Test] Label pagination aborts workflow before screenshot capture #968 (run #30795834741) and [Visual Test] Label preflight pagination aborts workflow before screenshot capture on run #30887567592 #970 (run #30887567592) describe the identical failure mode. Once the patch lands, those should be closed with a reference to this fix.

Not requesting additional info from the reporter — the diagnosis is already at the level of file, line, signal, and reproduction that the team needs to merge a one-step fix.

opencode session  |  github run