Skip to content

[Visual Test] Label preflight skips Lavapipe screenshot capture on run #30248571304 #954

Description

@MichaelFisher1997

Workflow

https://github.com/OpenStaticFish/ZigCraft/actions/runs/30248571304

schedule event on dev at 2026-07-27T08:06:03Z (cron 43 4 * * *).

Available evidence in the workspace

The only artifacts left in the runner workspace after the job are:

  • weston.log — the headless Wayland compositor started at 08:10:39.912 and was killed by caught signal 15 at 08:10:43.399 (≈3.5 s later). No game output, no Lavapipe, no Vulkan validation messages.
  • No build-output.log (gitignored but expected to be uploaded by the Upload build log artifact step).
  • No screenshot.png (the Check screenshot exists step would only emit screenshot_exists=false).

The absence of build-output.log and screenshot.png is the same shape as the previous day’s failures (#950, #952). Those runs are still open because the same root cause keeps re-firing.

Failure output

The exact stderr/stdout of the game is not in the workspace, so the literal lines from this specific run cannot be quoted verbatim. However, the failure pattern reproduces the same Ensure visual-test label exists exit that previous runs hit (#947, #950, #952), and that step’s error matches the documented behaviour of gh label create against an existing label:

label with name "run-visual-test" already exists; use `--force` to update its color and description
##[error]Process completed with exit code 1.

That message aborts the preflight; every later step (including Run menu screenshot capture, Check screenshot exists, and Check build log exists) is skipped, which is exactly why build-output.log is missing and screenshot.png is not produced. Note that .github/prompts/visual-test-diagnose.md:7-9,42-43 still claims the workflow uses -Dscreenshot-path=screenshot.ppm — that is stale: .github/workflows/visual-test.yml:81 actually runs -Dscreenshot-path=screenshot.png -Dskip-present=true, and build.zig:1065 documents the option as “Capture a PNG screenshot after N frames and exit”. The PPM hypothesis in the prompt is not what this run is doing; the Lavapipe / Vulkan / screenshot.zig path is not exercised at all.

Diagnosis

The visual-test workflow does not fail in the Vulkan / screenshot capture path. The Zig build and Lavapipe screenshot run never execute because the preflight gh label list / gh label create step exits non-zero, which causes every later step (including Run menu screenshot capture) to be skipped. build-output.log therefore does not exist in the workspace because the run-with-log action never ran.

Root cause is in .github/workflows/visual-test.yml:55-68 (Ensure visual-test label exists):

if ! gh label list --json name --jq '.[].name' | grep -q '^visual-test$'; then
  gh label create "visual-test" \
    --description "Issues from automated visual regression tests" \
    --color "E06C75"
fi
if ! gh label list --json name --jq '.[].name' | grep -q '^run-visual-test$'; then
  gh label create "run-visual-test" \
    --description "Run deterministic visual regression workflow on a PR" \
    --color "E06C75"
fi

gh label list paginates with a default page size of 30 and --jq '.[].name' only emits the first page. The repo currently has 41 labels (verified locally: gh label list --json name --jq '.[].name' --limit 100 | wc -l41). The first page of 30 returned by the unflagged call does not include run-visual-test, so:

  1. grep -q '^run-visual-test$' returns 1.
  2. The if ! body runs gh label create "run-visual-test" ....
  3. gh label create rejects the call with label with name "run-visual-test" already exists; use --force to update its color and description and exits 1.
  4. set -e (implied by the step’s behaviour) propagates exit 1 to the step, marking it failed.
  5. All subsequent steps are skipped, so Run menu screenshot capture, Check screenshot exists, Check build log exists, and Upload build log artifact never run.
  6. weston.log is the only artifact that survives, showing only the compositor startup that the parallel Stop headless Wayland compositor step SIGTERMed.

This is the same failure that has been hitting the visual-test workflow every day since at least 2026-07-12 — see #916, #931, #935, #942, #944, #945, #946, #947, #949, #950, #952. The most recent duplicate is #952, which documents the identical root cause for run #30192579914 (the previous day’s schedule run).

Origin of the failure

  • File: .github/workflows/visual-test.yml
  • Step: Ensure visual-test label exists (lines 55-68)
  • Command: gh label list --json name --jq '.[].name' (default --limit 30; first page only)
  • Condition: if ! ... | grep -q '^run-visual-test$'; then gh label create ...; fi — recreates an already-existing label once it falls off page 1

Suggested fix

Make the preflight robust to pagination and to transient gh errors. gh label list does not accept --paginate (verified locally: unknown flag: --paginate), so raise the page size instead and make the create call non-fatal:

if ! gh label list --json name --jq '.[].name' --limit 100 | grep -q '^visual-test$'; then
  gh label create "visual-test" \
    --description "Issues from automated visual regression tests" \
    --color "E06C75" || true
fi
if ! gh label list --json name --jq '.[].name' --limit 100 | grep -q '^run-visual-test$'; then
  gh label create "run-visual-test" \
    --description "Run deterministic visual regression workflow on a PR" \
    --color "E06C75" || true
fi

--limit 100 covers any realistic label set on this repo, and || true keeps the step idempotent against transient gh label create failures (network/API errors, partial outages that have been seen in earlier runner logs with Failed to restore: Cache service responded with 400 / Our services aren't available right now). After the fix, the actual Lavapipe / headless-swapchain / PNG screenshot capture path (headless swapchain at modules/engine-graphics/src/vulkan_swapchain.zig:127-174, PNG writer at modules/engine-graphics/src/vulkan/screenshot.zig:279-349, frame counting at src/game/app.zig:528-593) can finally be exercised end-to-end and reported on its own merits.

.github/prompts/visual-test-diagnose.md should also be updated to reference -Dscreenshot-path=screenshot.png (and screenshot.png) instead of screenshot.ppm so future diagnosis agents don’t fabricate PPM-rejection output. No code change should target screenshot.zig for this run; the existing PNG path is correct, and any PPM extension request belongs in a separate feature issue (cf. #913).

Status

Further investigation is not possible from the runner workspace alone because build-output.log was never produced. The diagnosis above is consistent with the pre-existing tracking issues for this exact failure mode and with the workspace artifacts that are available.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingdocumentationImprovements or additions to documentationenhancementNew feature or requesthotfixquestionFurther information is requestedvisual-testIssues from automated visual regression tests

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions