Skip to content

fix(ltx2): wire conv_in to correct widths in LTX2VideoUpBlock3d - #14308

Open
Solaris-star wants to merge 1 commit into
huggingface:mainfrom
Solaris-star:fix/14307-ltx2-conv-in-widths
Open

fix(ltx2): wire conv_in to correct widths in LTX2VideoUpBlock3d#14308
Solaris-star wants to merge 1 commit into
huggingface:mainfrom
Solaris-star:fix/14307-ltx2-conv-in-widths

Conversation

@Solaris-star

Copy link
Copy Markdown

Summary

Fixes #14307

Two independent defects in LTX2VideoUpBlock3d.__init__ made any non-nominal decoder_block_out_channels unloadable:

1. conv_in projected onto out_channels, but the upsampler expects out_channels * upscale_factor.

The upsampler is built with in_channels=out_channels * upscale_factor, so conv_in must project onto that same width. The guard should compare in_channels != out_channels * upscale_factor.

2. LTX2VideoDecoder3d divided output_channel by the current block's upsample_factor to compute input_channel.

The tensor arriving from the previous block is output_channel wide — no division needed. The division produced a fictitious width corresponding to no tensor in the graph.

Both defects are invisible on released LTX-2 checkpoints because the nominal decoder_block_out_channels happen to make conv_in unnecessary and the division a no-op.

Tests

Added tests/models/autoencoders/test_autoencoder_kl_ltx2.py with 3 tests:

  • test_conv_in_projects_to_upsampler_input_width — non-nominal widths trigger conv_in and forward pass succeeds
  • test_no_conv_in_when_widths_match — nominal case still works without conv_in
  • test_conv_in_with_no_upsampler — no-upsampler case works correctly
$ pytest tests/models/autoencoders/test_autoencoder_kl_ltx2.py -v
3 passed in 0.89s

Two independent defects in __init__ made any non-nominal
decoder_block_out_channels unloadable (huggingface#14307):

1. conv_in projected onto out_channels, but the upsampler that consumes
   its output expects out_channels * upscale_factor. Fix: project onto
   out_channels * upscale_factor and gate conv_in creation on
   in_channels != out_channels * upscale_factor.

2. LTX2VideoDecoder3d passed input_channel = output_channel //
   upsample_factor[i] to each up block, but the tensor arriving from the
   previous block is output_channel wide (no division needed). The
   division produced a fictitious width corresponding to no tensor in
   the graph. Fix: input_channel = output_channel.

Both defects are invisible on released LTX-2 checkpoints because the
nominal decoder_block_out_channels happen to make conv_in unnecessary
and the division a no-op. They surface immediately with any other
decoder width configuration.

Adds unit tests covering the conv_in projection, the no-conv_in
nominal case, and the no-upsampler case.

Fixes huggingface#14307
@github-actions github-actions Bot added fixes-issue models tests size/M PR with diff < 200 LOC and removed fixes-issue labels Jul 28, 2026

@ErenAta16 ErenAta16 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reproduced #14307 and confirmed both halves of the fix on CPU with a tiny AutoencoderKLLTX2Video, plus checked the nominal path is byte-for-byte unchanged.

Three decoder width patterns, latent_channels=4, upsample_factor=(2,2,2), decode() on a (1, 4, 3, 8, 8) latent:

decoder_block_out_channels conv_in count before after
(16, 32, 64) — nominal, each block 2× the last 0 decode OK (1, 3, 17, 64, 64) same
(32, 32, 32) — constant width 2 RuntimeError weight of size [16, 8, 3, 3, 3], expected input[1, 16, ...] decode OK (1, 3, 17, 64, 64)
(64, 32, 16) — decreasing 2 RuntimeError weight of size [16, 4, 3, 3, 3], expected input[1, 8, ...] decode OK (1, 3, 17, 64, 64)

So the conv_in branch is unreachable on the nominal pattern (0 built) and broken on every other one, exactly as the issue describes.

On the regression risk, since the second hunk changes input_channel for all blocks rather than just the conv_in ones: I diffed every decoder.up_blocks.* parameter shape in the nominal configuration before and after. 18 parameters, all identical. That's the part I'd have wanted checked before merging, and it holds.

The three new tests in tests/models/autoencoders/test_autoencoder_kl_ltx2.py pass (3 passed).

One note on the second hunk. The old input_channel = output_channel // upsample_factor[i] is only equal to the incoming width when block_out_channels[i-1] // upsample_factor[i-1] == block_out_channels[i] // upsample_factor[i] * upsample_factor[i] — i.e. exactly the nominal doubling pattern with a constant factor. Worth saying in the PR body, because it explains why the released checkpoints load: they sit on the one pattern where the wrong expression coincides with the right one.

Measured on main @ 9f6fc2c with the PR applied, torch CPU, Python 3.10.12.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

models size/M PR with diff < 200 LOC tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[LTX-2 VAE] LTX2VideoUpBlock3d.conv_in is wired to the wrong widths

2 participants