[Visual Test] Workflow fails before game runs: 'Ensure visual-test label exists' step rejected by duplicate label

[MichaelFisher1997]

Summary

The visual-test schedule run is failing before the game ever starts. The failure happens in the workflow's Ensure visual-test label exists preflight step, not in the Vulkan/SDL/screenshot pipeline the diagnose prompt describes. There is no build-output.log for this run because the game command at .github/workflows/visual-test.yml:81 was skipped.

Workflow run: https://github.com/OpenStaticFish/ZigCraft/actions/runs/29809988559 (also reproduced on the previous scheduled run 29725595085, which used the same head SHA 8945da9).

Error output from the failing step

Step name in the Actions UI: Ensure visual-test label exists.

Observed step outcome:

Process completed with exit code 1.

Observed step state (from the run page):

Ensure visual-test label exists     failure
Setup Lavapipe Vulkan               skipped
Run menu screenshot capture         skipped
Upload build log artifact           skipped
Stop headless Wayland compositor    skipped
Run opencode failure diagnosis      failure

Weston itself started cleanly at 1280x720 and was terminated by the cleanup step after the preflight failure; weston.log contains no compositor error.

Diagnosis

The failure originates in .github/workflows/visual-test.yml:55-68, specifically the second guard and create at .github/workflows/visual-test.yml:62-65:

if ! gh label list --json name --jq '.[].name' | grep -q '^run-visual-test$'; then
  gh label create "run-visual-test" \
    --description "Run deterministic visual regression workflow on a PR" \
    --color "E06C75"
fi

gh label list returns at most 30 labels by default. The repository currently has 41 labels (curl https://api.github.com/repos/OpenStaticFish/ZigCraft/labels?per_page=100). With the default ordering, visual-test is at position 38 and run-visual-test is at position 35 — both outside the first-page window, so neither guard sees them, both gh label create calls fire, and GitHub rejects the second one because run-visual-test already exists. That duplicate-create rejection is what produces exit code 1.

This is purely a workflow-side preflight failure. None of the rendering paths referenced in the diagnose prompt were reached:

• initContext in modules/engine-graphics/src/rhi_vulkan.zig
• headless createSwapchain path in modules/engine-graphics/src/vulkan_swapchain.zig:127-174
• menu selection in src/game/app.zig:268-276
• HomeScreen.init in modules/game-ui/src/screens/home.zig:33-41
• captureFrame in modules/engine-graphics/src/rhi_vulkan.zig:415-420
• requestCapture / recordCapture / completeCapture in modules/engine-graphics/src/vulkan/screenshot.zig

Therefore the Vulkan instance/device, swapchain, shader, HomeScreen, and screenshot readback code paths have not been exercised by these failures; they remain unverified until the workflow actually reaches the game step.

Side note on the diagnose prompt

The current visual-test.yml invokes zig build run -Dscreenshot-path=screenshot.png -Dskip-present=true (line 81), not -Dscreenshot-path=screenshot.ppm as .github/prompts/visual-test-diagnose.md describes. PNG is supported by detectScreenshotFormat in modules/engine-graphics/src/vulkan/screenshot.zig:262-267; PPM is not. If the workflow is ever changed to write a .ppm, requestCapture would log screenshot: unsupported image path 'screenshot.ppm' (use .png, .jpg, .jpeg, .gif, or .webp) and the game would never capture a frame. Either keep the workflow on .png, add a PPM encoder to screenshot.zig, or update the diagnose prompt to match the actual command.

Failure origin

• File: .github/workflows/visual-test.yml
• Step: Ensure visual-test label exists
• Failing command: gh label create "run-visual-test" at .github/workflows/visual-test.yml:63
• Trigger: unpaginated gh label list guard at .github/workflows/visual-test.yml:62 (and the symmetric guard for visual-test at lines 57-61)
• Head SHA: 8945da9ecb9944048dd94f70a88b63733dcc1a9e (feat: ship high-detail live world map (#943))

Suggested fix

Make the label setup idempotent without depending on a paginated gh label list query. Either pass --force to gh label create, or check each label with gh label view first. Increasing the --limit is fragile because the label count will keep growing.

- name: Ensure visual-test labels exist
  run: |
    gh label create "visual-test" \
      --description "Issues from automated visual regression tests" \
      --color "E06C75" \
      --force
    gh label create "run-visual-test" \
      --description "Run deterministic visual regression workflow on a PR" \
      --color "E06C75" \
      --force
  env:
    GH_TOKEN: ${{ secrets.OPENCODE_PAT }}

After the fix, please re-run visual-test on dev so we can finally exercise the headless Lavapipe + screenshot path and confirm the actual rendering pipeline (which this issue has not been able to validate).

[github-actions]

Classification: Bug (CI/workflow preflight failure, not a code defect).

Validation: The diagnosis is accurate. Verified against the repo:

• .github/workflows/visual-test.yml:55-66 matches the reporter's quote exactly. The two gh label list --json name --jq '.[].name' guards are unpaginated, so with the current label set (visual-test at position 38, run-visual-test at position 35 per gh label list default ordering and the 41-label count the reporter pulled from the API), both guards miss and both gh label create calls fire. GitHub rejects the second because the label already exists, which is the exit code 1.
• .github/workflows/visual-test.yml:81 does invoke … -Dscreenshot-path=screenshot.png -Dskip-present=true, not .ppm. The reporter's side note is also correct: .github/prompts/visual-test-diagnose.md:7, :42, :43 describe PPM, while modules/engine-graphics/src/vulkan/screenshot.zig:39-40 and :246-247 reject unsupported paths and the allow-list at :262+ does not include PPM. So a future switch to .ppm would silently break captures.

Scope: Single, well-contained bug. The pipeline beyond the preflight is fine — no need to split.

Duplicates / related PRs: None found on dev (head 8945da9 matches the reporter's quote). No prior PR touches the label step or the diagnose prompt. Safe to fix directly.

Suggested implementation steps (no code change from this comment):

1. Replace the two if ! gh label list … | grep -q … ; then gh label create … blocks with two unconditional gh label create … --force calls under GH_TOKEN: ${{ secrets.OPENCODE_PAT }}. --force is the idiomatic gh flag for idempotent label upserts and removes the pagination fragility the reporter flagged.
2. Reconcile .github/prompts/visual-test-diagnose.md with the actual workflow: either change lines 7, 42, 43 from screenshot.ppm to screenshot.png, or extend detectScreenshotFormat in modules/engine-graphics/src/vulkan/screenshot.zig to encode PPM and switch the workflow to .ppm. Recommend the prompt-only fix — adding a PPM encoder just to keep a stale doc in sync is dead weight.
3. After merging on dev, re-run visual-test (workflow_dispatch or run-visual-test label on a PR) so the Lavapipe + screenshot pipeline is actually exercised for the first time and the menu-screenshot artifact / build-output.log are produced.

No additional info needed from the reporter.

opencode session  |  github run