IndexTTS-2.5: port the vLLM-Omni TTS lane (GPT-2 AR talker + EnhancedCodec + S2Mel CFM/DiT + BigVGAN) — our first audio-generating model

[localai-bot]

What

Port IndexTTS-2.5, upstream-supported at
https://recipes.vllm.ai/IndexTeam/IndexTTS-2.5 and served by vLLM-Omni as:

vllm-omni serve IndexTeam/IndexTTS-2.5 \
  --omni \
  --trust-remote-code \
  --served-model-name IndexTeam/IndexTTS-2.5 \
  --port 8092

This would be our first audio-generating model. Every audio path we ship today
consumes audio (Parakeet, Voxtral, audio_processor.cpp); nothing synthesizes it,
and there is no /v1/audio/speech route.

Upstream anchors

vllm_omni/model_executor/models/indextts2/ registers two architectures:

Registry key	Module
IndexTTS2TalkerForConditionalGeneration	indextts2/indextts2_talker.py
IndexTTS2S2MelDecoder	indextts2/indextts2_s2mel_decoder.py

Deploy config: vllm_omni/deploy/indextts2_5.yaml (auto-selected by model_type).
Offline entry point: vllm_omni.model_executor.models.indextts2.end2end.

Pipeline

Stage 0 — a ~0.8B GPT-2 AR talker turns text + reference audio into mel codes.
Stage 1 — EnhancedCodec (2.5 replaces IndexTTS-2's RepCodec, with
use_gpt_latent=false), then an S2Mel CFM/DiT flow-matching decoder, then
BigVGAN, emitting 22.05 kHz mono WAV. Roughly 6 GB VRAM, so it fits GB10 with
enormous headroom.

Reference audio is mandatory. Upstream states IndexTTS-2 does not support
text-only synthesis, so the voice-cloning encoders (w2v-bert-2.0, MaskGCT
semantic codec, CAMPPlus speaker embedding) are required port surface, not an
optional extra. They download to checkpoints/hf_cache/ on first run and need
pinned revisions under the NAS checkpoint policy.

What we can reuse

• BigVGAN is already ported — src/vllm/model_executor/models/minimax_h3_audio_vae.cpp,
  gated at 4.2e-9 against the checkpoint's own remote code. It needs generalizing
  out of the minimax_h3_* namespace, which converges with LTX-2.5: port the 21B joint video+audio DiT and generalize the video seam #435's ask to
  generalize the video seam.
• WAV serialization — minimax_h3_wav.cpp.
• The flow-matching denoise loop and AdaLN/timestep machinery from the H3 lane.
• Conformer encoder (parakeet_encoder.cpp) — w2v-bert-2.0 is a Conformer.
• Mel front-end work in whisper_audio.cpp.
• The /v1/videos runner seam is the template for /v1/audio/speech.

New: the GPT-2 talker backbone (opt.cpp is the same family — learned absolute
positions, LayerNorm — so this is an additive delta, not a fresh transformer),
EnhancedCodec, S2Mel, and the three reference encoders.

Blocked on

#633 — there is no vllm-omni parity pin, and vllm-omni additionally requires vLLM
0.27.0+ against our 0.26.0.dev0 core pin. Until that lands there is no oracle this
row can legally be gated against.

Deviations to record, not silently inherit

• Upstream's Stage 0 uses plain vLLM sampling and deliberately does not
  reproduce IndexTeam's own num_beams=3 default. We mirror vLLM-Omni and record
  the divergence from the reference implementation.
• Upstream states a seed controls both AR sampling and per-request CFM noise, and
  that differing concurrent batch composition does not guarantee a bit-identical
  waveform. Any gate therefore runs at fixed seed, c1, fixed batch composition.
• The two upstream doc surfaces disagree on language support (recipe page:
  zh/en/ja/es/ar; vllm-omni docs: zh/en/zhen/ja/yue). Resolve against
  indextts2_5.yaml before anything reaches docs/FEATURES.md.

Licensing

The checkpoint is under a custom bilibili-model-license, not Apache-2.0. This
constrains which fixtures and goldens can be committed and needs checking before
any artifact lands in-tree.

Quantization

bf16 for v1. vLLM-Omni ships no quantized IndexTTS arm at all (the same situation
recorded for H3), and the model already fits GB10 unquantized. Every
unimplemented quant arm must refuse at load naming the missing piece, and be
recorded as owed in the model matrix.