Skip to content

[None][feat] DSv4: model, tokenizer, and integration coverage #15414

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
juney-nvidia merged 9 commits into from
Jun 28, 2026
Merged

[None][feat] DSv4: model, tokenizer, and integration coverage #15414

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
ade38d4
[None][feat] DSv4 model and API support
lfr-0531 Jun 16, 2026
71eefee
[None][test] DSv4 integration coverage and docs
lfr-0531 Jun 16, 2026
6ddbcea
Move DSv4 sparse config lowering to PR-9
lfr-0531 Jun 25, 2026
c878525
Clean up DSv4 PR-9 test list scope
lfr-0531 Jun 25, 2026
9dfe1a7
Restore quickstart advanced example scope
lfr-0531 Jun 25, 2026
ed1d403
Fix stale DSv4 FP4 indexer test entry
lfr-0531 Jun 26, 2026
8f2bae5
Remove unrelated PR-9 test infrastructure changes
lfr-0531 Jun 27, 2026
95fb33e
Fix DSA slot mapping recompute for draft replay
lfr-0531 Jun 27, 2026
8545210
Fix DSv4 sparse config review issues
lfr-0531 Jun 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 7 additions & 4 deletions docs/source/models/supported-models.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ The following is a table of supported models for the PyTorch backend:
| `DeepSeekV2ForCausalLM` [^5] | DeepSeek V2 | `deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct` |
| `DeepseekV3ForCausalLM` | DeepSeek-V3, Kimi-K2 | `deepseek-ai/DeepSeek-V3` |
| `DeepseekV32ForCausalLM` | DeepSeek-V3.2 | `deepseek-ai/DeepSeek-V3.2` |
| `DeepseekV4ForCausalLM` [^11] | DeepSeek-V4 | `deepseek-ai/DeepSeek-V4-Pro` |
| `ExaoneForCausalLM` [^5] | EXAONE 3.5 | `LGAI-EXAONE/EXAONE-3.5-32B-Instruct` |
| `Exaone4ForCausalLM` | EXAONE 4.0 | `LGAI-EXAONE/EXAONE-4.0-32B` |
| `ExaoneMoEForCausalLM` | K-EXAONE | `LGAI-EXAONE/K-EXAONE-236B-A23B` |
Expand All @@ -32,7 +33,7 @@ The following is a table of supported models for the PyTorch backend:
| `LlamaForCausalLM` | Llama 3.1, Llama 3, Llama 2, LLaMA | `meta-llama/Meta-Llama-3.1-70B` |
| `Llama4ForConditionalGeneration` | Llama 4 | `meta-llama/Llama-4-Scout-17B-16E-Instruct` |
| `MiniMaxM2ForCausalLM` [^5] | MiniMax M2/M2.1/M2.7 | `MiniMaxAI/MiniMax-M2.7` |
| `MiniMaxM3SparseForConditionalGeneration` [^11]| MiniMax-M3 | `MiniMaxAI/MiniMax-M3` |
| `MiniMaxM3SparseForConditionalGeneration` [^12]| MiniMax-M3 | `MiniMaxAI/MiniMax-M3` |
| `MistralForCausalLM` | Mistral | `mistralai/Mistral-7B-v0.1` |
| `MixtralForCausalLM` | Mixtral | `mistralai/Mixtral-8x7B-v0.1` |
| `MllamaForConditionalGeneration` | Llama 3.2 | `meta-llama/Llama-3.2-11B-Vision` |
Expand Down Expand Up @@ -63,6 +64,7 @@ Note: Support for other models may vary. Features marked "N/A" are not applicabl
| -------------------------------- | ----------------- | ---------- | -------------------------- | --------------------- | --------------- | --- | ---------------- | ----------------- | ------ | ------------- | ---------------- | -------------- | ------------------------ | --------------------- | --------------- |
| `DeepseekV3ForCausalLM` | Yes | Yes | Yes | Yes | Yes [^1] | Yes | No | No | No | Yes | Yes | Yes [^2] | N/A | Yes | Yes |
| `DeepseekV32ForCausalLM` | Yes | Yes | Yes | Yes | Yes | Yes | No | No | No | Yes | Yes | Yes | N/A | Yes | Yes |
| `DeepseekV4ForCausalLM` [^11] | Yes | Yes | Yes | Untested | Yes | Yes | No | No | No | Yes | Yes | Untested | Yes | Untested | Untested |
| `Glm4MoeForCausalLM` | Yes | Yes | Yes | Untested | Yes | Yes | No | No | No | Yes | Yes | Untested | N/A | Yes | Yes |
| `Qwen3MoeForCausalLM` | Yes | Yes | Yes | Yes | Yes | No | Yes | Yes | No | Yes | Yes | Yes | N/A | Yes | Yes |
| `Qwen3NextForCausalLM` [^3] | Yes | Yes | Yes | Untested | Yes | No | No | No | No | Yes | Yes | No | No | Untested | Untested |
Expand All @@ -73,7 +75,7 @@ Note: Support for other models may vary. Features marked "N/A" are not applicabl
| `NemotronHForCausalLM` | Yes | Yes | Yes | Yes | Yes | Yes | No | No | No | Yes | Yes | Yes | N/A | Untested | Untested |
| `Gemma4ForConditionalGeneration` | Untested | Yes | Untested | No | Yes | No | No | No | No | Yes | Untested | No | Yes | Untested | Untested |
| `Step3p7ForConditionalGeneration`| Yes | Yes | Yes | Untested | Untested | Yes | No | No | No | Yes | Untested | Untested | Yes | Untested | Untested |
| `MiniMaxM3SparseForConditionalGeneration` [^11] | Yes | Yes | Yes | Untested | Untested | No | No | No | No | Yes | Untested | No | N/A | Untested | Untested |
| `MiniMaxM3SparseForConditionalGeneration` [^12] | Yes | Yes | Yes | Untested | Untested | No | No | No | No | Yes | Untested | No | N/A | Untested | Untested |

[^1]: Chunked Prefill for MLA can only be enabled on SM100/SM103.
[^2]: KV cache reuse for MLA can only be enabled on SM90/SM100/SM103 and in BF16/FP8 KV cache dtype.
Expand All @@ -84,7 +86,8 @@ Note: Support for other models may vary. Features marked "N/A" are not applicabl
[^8]: Supports text and image inputs. The vision tower runs in BF16 even when the text decoder is quantized (FP8 block-scale or NVFP4). The text decoder is also usable standalone (text-only) via the `Step3p5ForCausalLM` architecture.
[^9]: Audio modality only supported on E2B/E4B variants.
[^10]: Audio requires a checkpoint with a `sound_config` and is supported only on the full (non-disaggregated) model path, not the EPD disaggregated path.
[^11]: Supports text, image, and video inputs over the block-sparse attention path. The model is served in BF16. The text decoder is also usable standalone (text-only) via the `MiniMaxM3SparseForCausalLM` architecture. KV cache reuse and MTP are not supported on the sparse-attention path in this release.
[^11]: DeepSeek-V4 is only supported on Blackwell GPUs (`SM100+`). See the [DeepSeek-V4 example README](../../../examples/models/core/deepseek_v4/README.md) for setup and parallelism.
[^12]: Supports text, image, and video inputs over the block-sparse attention path. The published MXFP8 checkpoint is dequantized on load so the runtime sees an effectively BF16 model. The text decoder is also usable standalone (text-only) via the `MiniMaxM3SparseForCausalLM` architecture. KV cache reuse and MTP are not supported on the sparse-attention path in this release.

# Multimodal Feature Support Matrix (PyTorch Backend)

Expand All @@ -105,7 +108,7 @@ Note: Support for other models may vary. Features marked "N/A" are not applicabl
| `Qwen3VLForConditionalGeneration` | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | L + I + V |
| `Qwen3VLMoeForConditionalGeneration` | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | L + I + V |
| `Step3p7ForConditionalGeneration` | Yes | Yes | Untested | Yes | Untested | Untested | Untested | Untested | L + I |
| `MiniMaxM3SparseForConditionalGeneration` [^11] | Yes | Yes | Untested | Yes | Untested | No | Untested | Untested | L + I + V |
| `MiniMaxM3SparseForConditionalGeneration` [^12] | Yes | Yes | Untested | Yes | Untested | No | Untested | Untested | L + I + V |

Note:
- L: Language
Expand Down
Loading
Loading