feat(ltx-2.5): image conditioning at crf=0, and the VAE encoder load path that did not exist (#644)

[localai-bot]

Row 1 of the #644 LTX-2.5 full-port campaign: closes the image conditioning arm and builds the thing every arm was blocked on.

Spec 0c1390f8e, committed before any code: .agents/specs/ltx25-image-conditioning.md.

What was actually missing

The refusal at ltx2_video.cpp:1122 was honestly written and its anchors verify, but it led with the CRF round trip, and that is not what blocked anything. Re-verified against the current tree:

claim	check	result
the encoder MATH landed	Ltx2ConvVideoEncode	present, gated since phase L11
the placement landed	Ltx2ConditionVideoByLatentIndex	present, gated
the engine loads the DECODER filter only	ltx2_video.cpp:752	confirmed
an encoder key filter exists anywhere	grep -rn Ltx2VideoVaeEncoderKeyRules src include tests	no match
an encoder CONFIG parser exists anywhere	grep -rn ParseConvVideoEncoder src include tests	no match

So the encoder was a brick with no delivery route. preprocess short-circuits at if crf == 0: return image (media_io/decode.py:425-426), so a crf=0 path is upstream-legal and needs no codec — this box has no av module at all, which is the demonstration rather than the obstacle.

What this adds

• Ltx2VideoVaeEncoderKeyRules / Ltx2ParseConvVideoEncoderConfig / Ltx2CheckpointHasVideoEncoder — VAE_ENCODER_COMFY_KEYS_FILTER (video_vae/model_configurator.py:267-276) and VideoEncoderConfigurator over _prepare_video_encoder_kwargs (:37-69, 72-78).
• The pixel front-end — Ltx2DecodePpmRgb, Ltx2ResizeAndCenterCrop, Ltx2LoadImageAndPreprocess, Ltx2ResolveDefaultImageCrf (media_io/decode.py:46-79, 139-170, 413-435, resize.py:41-73, range_map.py:8-9, constants.py:36-37, 124, 126-133).
• The engine path — encoder loaded beside the decoder, and the image decoded, resized, encoded and placed per phase, before the noiser, because the two-stage recipe renders its stages at different resolutions.
• image_crf, the family's first per-generation extra, reachable through the existing ABI arrays with no ABI change, and through ltx2-gen --first-frame / --image-crf.

Both new TUs are additive files rather than additions to ltx2_loader.cpp, which a concurrent row of this campaign also needs to edit.

CRF 0 is out of distribution, and says so

An LTX-2.5 checkpoint resolves default_image_crf = 18 (constants.py:37/124/130-133), and the H.264 round trip is unported. The default therefore refuses — a caller has to ask for image_crf=0 knowingly and gets a render conditioned on uncompressed pixels. Both halves are stated rather than one of them.

The other three refusals now name different missing pieces

This campaign has had five refusals whose stated reason went stale, including the one replaced here. Each surviving message names the exact symbol a later reader can go and check:

• keyframe → the DiT's unported keyframes_abs_pos_embedding (the encoder and Ltx2ConditionVideoByKeyframe are both here now);
• reference image / video → the IC-LoRA's downscale_factor / temporal_scale_factor, which live in LoRA metadata this project does not read;
• reference audio → the AUDIO VAE's encoder key filter, which this row did not build.

Evidence

scripts/gen-ltx2-image-cond-goldens.py executes upstream ltx_core and ltx_pipelines at the pinned SHA fd4ded7f. The package __init__ files are bypassed and av / OpenImageIO are stubbed as tripwires, not shortcuts: preprocess(image, 0) is image is asserted by object identity (no stub can manufacture that) and preprocess(image, 18) is asserted to reach the stubbed codec and raise, so the branch is measured rather than read.

Bands derived from measurement, not picked (both tolerances set to 1e-12 and the reported worst read back):

section	space	worst	band
resize	0..255	6.10352e-05	2e-4
preprocess	[-1, 1]	4.76837e-07	2e-6
encoded latent / conditioned clean	latent	2.68221e-07	2e-6
noised latent	latent	1.78814e-07	2e-6

Six source mutations RED, each rebuilt and restored byte-for-byte (md5 verified). Green baseline test_ltx2_image_cond 15 / 198 and test_ltx2_video 32 / 550:

mutation	result
ceil → llround in the resize	image_cond 14/15, 197/198
encoder key rules → the DECODER's	image_cond 14/15, 195/198; video 31/32 (THREW; assertion COUNT moved to 517)
mask 1 - strength → strength	image_cond 13/15, 193/198
latent width from vae.out_channels	image_cond 14/15, 192/198; video 9/32, 36/53
CRF refusal removed	image_cond 14/15, 193/198; video 31/32
encode the image, never PLACE it	video 31/32, 548/550

Encoder weights are loaded AND used, proven separately: one tensor of the encoder half is perturbed in a second fixture (offset read from the safetensors header, not guessed) with the request byte-identical, and the conditioning digest must move. The first version of that check searched for the tensor NAME and flipped a byte a fixed distance past it — which lands in a neighbouring tensor's JSON — and it failed, which is the only reason that is a footnote rather than a false green.

Three of this row's own claims were REFUTED

Recorded in the spec's §8 rather than quietly fixed:

1. The ceil does not guard what upstream's comment says it guards. Swept every source/target pair in 3..40 → {16, 24}: no pair makes src * scale land just above an integer. It is still load-bearing, for the ordinary reason that it disagrees with round and floor, and the generator asserts that.
2. No golden here can see a resize/normalize ORDER swap. Resize is a convex combination and normalize is affine, so the two orders are equal in exact arithmetic; the f32 gap is 1.94e-07, below every band and not amplifiable. The order is mirrored because it is upstream's, and that is written down in three places instead of assumed covered.
3. Torch's bilinear cannot be matched bit for bit portably. Index map and lambdas were probed with basis images and match exactly; the residual is FMA contraction in torch's kernel, and this tree builds -ffp-contract=off.

One mutation SURVIVED and is analysed rather than hidden (spec §8.4): phase 0 runs at noise_scale = 1.0, where the mutated value is genuinely discarded — and separately, test_ltx2_video gates no value of the composed latent, which test_ltx2_image_cond does against executed upstream. That survivor is why the trace digest is taken over the tokens as written rather than over the encoder's output.

Gate

• BUILD_EXIT=0, 0 ENOSPC/BFD hits, CPU Release, gcc.
• ctest -N → 424; ctest -j 4 → 424/424 passed, 0 failed, exit 0 — denominator asserted against the listing.
• Focused: test_ltx2_image_cond 15 cases / 198 assertions, test_ltx2_video 32 / 550, both exit 0.
• scripts/agent-preflight.sh --staged → all gates green, including the new check-oracle-pins.py from policy(oracle): admit a pinned secondary oracle where vLLM implements nothing (#647) #650.
• Generator is byte-reproducible: md5 da4f7423…a439 before and after a regeneration.

Closes part of #644.

🤖 Generated with Claude Code