[Lighting Overhaul][Phase 0] Establish visual baselines and repair pipeline contracts

[MichaelFisher1997]

Parent

Part of #898.

Goal

Create trustworthy visual/performance baselines and fix pipeline contract bugs that would invalidate comparisons in later phases.

Scope

• Add deterministic scenes/captures for noon, low sun, cave entrance, sealed cave, RGB emitter, foliage/cutout, water, and a cross-chunk corridor.
• Capture debug channels for skylight, RGB block light, AO, cascade index, caster coverage, shadow factor, and final output.
• Correct the sky shadow UBO to four matrices and align every CPU/GLSL field offset.
• Add a shader ABI test using SPIR-V reflection or equivalent generated offset validation.
• Transition all shadow array layers correctly and include the required early/late depth stages.
• Publish the actual allocated shadow resolution to cascade fitting and sampling.
• Recreate shadow resources/descriptors safely at a frame boundary when quality changes.
• Make the final display transform format-independent for sRGB and UNORM/headless targets.
• Record high-preset CPU/GPU timings before behavioral rewrites.

Out of scope

• Voxel propagation changes.
• New cascade fitting/sampling math.
• Artistic lighting retuning.

Acceptance criteria

• CPU and all shaders agree on shadow UBO size, offsets, and cascade count.
• Vulkan validation reports no shadow array layout or dependency errors.
• Runtime shadow resolution changes use matching physical resources and uniforms.
• Windowed sRGB and headless UNORM captures match within an agreed tolerance.
• Deterministic scene captures and baseline benchmark JSON are documented and repeatable.
• Debug channels make each later phase independently diagnosable.

Verification

nix develop --command zig build test
nix develop --command zig build -Dskip-present
nix develop --command zig build test-robustness
nix develop --command zig build run -Dskip-present -Dshadow-test-scene -Dscreenshot-path=screenshots/lighting-phase0-shadow.png -Dscreenshot-frame=180
nix develop --command zig build benchmark -Doptimize=ReleaseFast -Dbenchmark-preset=high -Dbenchmark-duration=60 -Dbenchmark-output=lighting-phase0-high.json

[github-actions]

Thanks for splitting the lighting overhaul into phases – this matches the audit in #898 cleanly. A few observations after validating the cited evidence and the open-PR backlog:

Audit validation

I confirmed the three High-severity findings Phase 0 is meant to repair. The sky/CPU shadow UBO mismatch is real:

• assets/shaders/vulkan/sky.frag:34-69 declares mat4 light_space_matrices[3] and vec4 cascade_splits, then later indexes cascade 3 (getVolShadow, line 58: int layer = 3) and reads shadows.cascade_splits[3], which the declared layout does not provide. With std140 packing the trailing vec4 shadow_texel_sizes reads back undefined-but-typically-zero on most drivers.
• modules/engine-graphics/src/vulkan/rhi_shadow_bridge.zig:4-9 mirrors a four-cascade layout (SHADOW_CASCADE_COUNT matrices, [4]f32 splits/sizes, plus shadow_params not present on the GPU side).

So shader and CPU already disagree on at least one stride and on cascade count, and the GPU struct is silently shorter than the CPU one – exactly the kind of mismatch that makes Phase 2 (CSM rewrite) comparisons meaningless. A shader-ABI test (Scope item 3 + Acceptance criterion 1) is the right way to lock this down.

Related / duplicate work

git log --oneline -10 shows only #897 (RD22 streaming) since the umbrella opened. No open PR touches the shadow UBO, vulkan_swapchain.zig display encoding, or the runtime-quality descriptor path, so there is no merge conflict risk to call out.

Scope suggestion – split into 2–3 PRs

The eight scope items in this issue touch three different subsystems, so I'd recommend landing them as a small PR series rather than one mega-PR. Suggested split, ordered by dependency:

1. PR-A: Pipeline-contract correctness (Scope items 3, 4, 5, 6, 7)
   • Fix the sky shadow UBO to four matrices, align every CPU/GLSL field offset, add the SPIR-V reflection-based ABI test, correct the array-layer transitions (including the missing early/late depth stages), publish the real allocated shadow resolution, and make the final display transform format-independent.
   • This is pure bug-fix territory and lands the validation gate that every later phase relies on. It is also the part that resolves the three High-severity rows of the audit table in [Lighting Overhaul][Umbrella] Rebuild voxel lighting, shadows, and scene composition #898.
2. PR-B: Deterministic baselines (Scope items 1, 2, 8)
   • Add the eight deterministic scenes/captures, the debug channels for skylight / RGB block light / AO / cascade index / caster coverage / shadow factor / final output, and record the high-preset CPU+GPU baseline JSON.
   • These need to land after PR-A so the baseline numbers are not invalidated by the contrast/encoding fixes that PR-A introduces.

Splitting along this line keeps each PR reviewable in one sitting, makes rollback surgical, and gives Phase 2 (#903) a stable comparison target.

Acceptance criterion asks

Two acceptance criteria are currently underspecified for QA:

• “Windowed sRGB and headless UNORM captures match within an agreed tolerance.” – please pick a concrete metric (ΔE2000, max-channel L_inf, or CIE76) and a numerical bound (e.g. ΔE2000 ≤ 1.5 mean, ≤ 4 max) so CI can gate on it instead of eyeballing.
• “Debug channels make each later phase independently diagnosable.” – could you enumerate the exact channels and the phase each one first depends on? Right now it reads like a wish list rather than a pass/fail contract.

Minor

• The -Dshadow-test-scene verification line reads -Dshadow-test-scene without =true. The build option is declared b.option(bool, …), so it is accepted as a flag, just confirming that matches intent.
• Consider adding zig fmt src/ and a nix develop --command zig build -Doptimize=ReleaseFast smoke run to the Verification block per the AGENTS.md checklist, so reviewers do not have to remember the convention.

Overall this is a clean, well-scoped Phase 0 – the only thing blocking confident sign-off is the two tolerance gaps above and the PR-A vs PR-B ordering.

[github-actions]

Posted a focused comment on #900. Summary of what I did:

• Classified as Phase 0 of the umbrella lighting overhaul ([Lighting Overhaul][Umbrella] Rebuild voxel lighting, shadows, and scene composition #898) — a mix of bug-fix and new baseline infrastructure.
• Validated by confirming the cited sky/CPU shadow UBO mismatch (sky.frag:34-69 declares 3 matrices but indexes [3]; rhi_shadow_bridge.zig:4-9 declares 4).
• Checked for related work — no open PRs touch shadow UBO, swapchain, or descriptor recreation, so no merge conflict risk.
• Suggested splitting scope into PR-A (pipeline-contract correctness: items 3–7) before PR-B (deterministic baselines: items 1, 2, 8), because the baselines would otherwise be invalidated by the contrast/encoding fixes.
• Flagged two underspecified acceptance criteria: the sRGB/UNORM match tolerance needs a concrete metric + numerical bound, and the "debug channels make each later phase independently diagnosable" criterion should enumerate channels per phase.
• Noted a couple of minor verification-block cleanups against AGENTS.md conventions.

opencode session  |  github run