Add HunyuanVideo-1.5 contrib model

[jimburtoft]

Note: The below template includes items meant for model contributions only. For other contributions such as bug fixes, features, etc., only fill out the relevant portions of the form.

Description

HunyuanVideo-1.5 text-to-video generation on AWS Trainium. 8.33B DiT transformer with TP=4 parallelism and NKI flash attention, producing 480x848 video at ~55s with Classifier-Free Guidance.

Multi-component pipeline: DiT backbone (Neuron TP=4), 3D Causal VAE (Neuron tiled decode), byT5 glyph encoder (Neuron traced), Qwen2.5-VL 7B LLM text encoder (CPU bf16).

Model Information

Model Name: HunyuanVideo-1.5 (tencent/HunyuanVideo-1.5)

Model Architecture: 54 double-stream DiT blocks, hidden_size=2048, 16 attention heads, head_dim=128, flow matching with Euler scheduler

Purpose: Text-to-video generation (480x848 resolution, 5 frames)

Checklist

Required Components

• Accuracy Test (test/integration/test_model.py)

  • byT5 encoder + mapper: cosine similarity >= 0.999 vs CPU reference (measured: 1.000 / 0.9999)
  • VAE tile decode: finite output, correct shape (16x spatial upsampling)
  • E2E pipeline: produces 5 frames at 480x848 resolution
  • E2E performance: 50-step generation within 120s wall-clock

• README.md with the following sections:

  • Usage Example: Full setup from scratch with environment variables
  • Compatibility Matrix: Validated on trn2.3xlarge with SDK 2.28
  • Example Checkpoints: Links to all 3 required HuggingFace models
  • Testing Instructions: Commands for component and E2E test groups

• Source Code (src/)

  • 8 Python modules covering DiT TP wrapper, CPU preprocessor, VAE compilation, tiled VAE decode, byT5 tracing, negative embedding caching, E2E pipeline, and DiT compilation
  • Uses torch_neuronx.trace() + parallel_model_trace() for TP=4

Optional Components

• Unit Tests (CPU or Neuron-based)

Folder Structure

Confirm your contribution follows this structure:

/contrib/models/HunyuanVideo-1.5/
  README.md
  /src
    __init__.py
    dit_tp_wrapper.py
    dit_wrapper.py
    e2e_pipeline.py
    compile_vae_neuron.py
    tiled_vae_decode.py
    trace_byt5.py
    cache_neg_embeddings.py
    recompile_dit_masked.py
  /test
    __init__.py
    /integration
      __init__.py
      test_model.py
  /samples
    /neuron
      frame_0000.png
      frame_0002.png
      frame_0004.png
  /examples
    generate_video.py

Testing

How did you test this change?

All tests executed on trn2.3xlarge (LNC=2, 4 logical NeuronCores) with pre-compiled models.

Component tests (byT5 + VAE) run in-process; E2E tests run as subprocess.
Tests must be run in two groups to avoid NeuronCore contention:

pytest test/integration/test_model.py -k "ByT5 or VAE" -v  # 2 passed, 54s
pytest test/integration/test_model.py -k "E2E" -v           # 2 passed, 150s

Test Results:

Test	Result	Key Metric
byT5 encoder accuracy	PASSED	cos_sim = 1.000001
byT5 mapper accuracy	PASSED	cos_sim = 0.999947
VAE tile decode	PASSED	[1,3,5,128,128], 177ms, all finite
E2E frame generation (2 steps)	PASSED	5 frames at 480x848, 23.2s
E2E performance (50 steps)	PASSED	327ms/step, 60.8s wall-clock

Compatibility

Tested with:

• Neuron SDK Version(s): 2.28 (neuronx-cc 2.22, torch-neuronx 2.9.0)
• Instance Type(s): trn2.3xlarge (LNC=2)
• PyTorch Version: 2.9.0
• Python Version: 3.12.3

Additional Information

5 code patches required for Neuron compatibility:

1. Dtype mismatch in token_refiner (mask.float() -> mask.to(x.dtype))
2. FlexAttention replaced with F.scaled_dot_product_attention
3. NaN from SDPA mask expansion (key-only mask fix)
4. byT5 output dtype cast to bf16
5. RoPE rotate_half: interleaved-pair rotation (critical for visual quality)

Performance:

• DiT per-step: 327ms (TP=4, NKI flash attention)
• VAE tiled decode: 8.5s (45 tiles x 177ms)
• byT5 encode: 4.4ms
• E2E with CFG (50 steps): ~55s
• E2E without CFG: ~25s

Known limitations:

• LLM (Qwen2.5-VL 7B) runs on CPU (~14s) -- too large for single-core Neuron trace
• VAE requires monkey-patches for replication_pad3d compiler issue
• B=2 batched CFG is 4.2x slower than sequential (HBM bandwidth saturation)

Related Issues

N/A

vLLM Integration

• This model/feature is intended for use with vLLM
• Documentation includes vLLM registration instructions

---

By submitting this PR, I confirm that:

• I have read and followed the contributing guidelines
• This is a community contribution and may have limited testing compared to officially-supported models
• The code follows best practices and is well-documented
• All required components listed above are included