[None][feat] Support DeepSeek-V4 model - #14751

Closed

lfr-0531 wants to merge 58 commits into

NVIDIA:mainfrom

lfr-0531:user/fanrongl/dsv4_model

Commits on Jun 17, 2026

[None][feat] Add DeepSeek-V4 model support

heyuhhh
authored and
lfr-0531
committed
[TRTLLM-12338][feat] Lift TOKENIZER_ALIASES to module level in llmapi.llm_args (#13568)

nv-yna
authored and
lfr-0531
committed
[None][fix] Fix fused mHC RMS normalization (#13587)

mingyangHao
authored and
lfr-0531
committed
[None][fix] Cherry pick KV Cache Manager V2 fixes (#13597)

jiaganc
authored and
lfr-0531
committed
[None][fix] Remove MHC fused hidden-size guard (#13611)

mingyangHao
authored and
lfr-0531
committed
[None][feat] Enable EPLB for DeepSeek-V4 (#13595)

Barry-Delaney
authored and
lfr-0531
committed
[None][feat] tool template and parser for dsv4 (#13608)

authored and
lfr-0531
committed
[TRTLLM-12111][feat] Add V2 KV cache event support (#13589)

yizhang-nv
authored and
lfr-0531
committed
[None][feat] Fuse FP8 1x128 quantize + UE8M0 scale pack on SM100 (#13628)

lishicheng1996-nv
authored and
lfr-0531
committed
[None][feat] Add MLA dependency-aware overlap on DSv4 (compressor || q_b_proj+norm) (#13629)

lishicheng1996-nv
authored and
lfr-0531
committed
[None][fix] Change compressor linear dtype (#13605)

mingyangHao
authored and
lfr-0531
committed
[None][fix] Remove duplicate V2 KV cache allocation revert

heyuhhh
authored and
lfr-0531
committed
[None][infra] Update CI for DS v4 (#13604)

EmmaQiaoCh
authored and
lfr-0531
committed
[TRTLLM-12374][fix] Tighten DSv4 cache constraint floor to match warmup shapes (#13657)

authored and
lfr-0531
committed
[None][test] Add DeepSeek-V4 CI coverage (#13653)

Barry-Delaney
authored and
lfr-0531
committed
[TRTLLM-12383][fix] limit MHC TF32 pmap GEMM to SM100 (#13660)

mingyangHao
authored and
lfr-0531
committed
[None][chore] Apply pre-commit fixes

heyuhhh
authored and
lfr-0531
committed
[None][fix] Plumb swiglu_limit through DeepGEMM and TRTLLMGen FP8 fused MoE (#13767)

Barry-Delaney
authored and
lfr-0531
committed
[None][perf] Use bf16 custom router GEMM kernel for DeepSeek-V4 (#13646)

authored and
lfr-0531
committed
[None][perf] Optimize DeepSeek-V4 compressor BF16 input (#13761)

mingyangHao
authored and
lfr-0531
committed
[TRTLLM-12478][fix] Fix IMA about KVCacheManagerV2's overlap scheduler mode

heyuhhh
authored and
lfr-0531
committed
[None][fix] Fix fused MHC for DeepSeek-V4-Pro hidden size (#13771)

authored and
lfr-0531
committed
[None][fix] Use compressed lengths for DeepSeek-V4 indexer (#13802)

mingyangHao
authored and
lfr-0531
committed
[None][feat] Indexer topk opt (#13811)

pcastonguay
authored and
lfr-0531
committed
[None][fix] Gate DeepSeek V4 rotate activation (#13889)
lfr-0531
committed
[None][fix] Run DeepSeek V4 gate test on CUDA (#13932)
lfr-0531
committed
fix: route token sparse fmha to cubins

heyuhhh
authored and
lfr-0531
committed
test: add v4 sparse mla debug coverage

heyuhhh
authored and
lfr-0531
committed
[None][feat] Keep DSv4 o_a_proj as FP8, and port vLLM's fused_inv_rope_fp8_quant (#13938)

lishicheng1996-nv
and
lfr-0531
committed
[None][fix] Restore overlap headroom in seq slot pool sizing (#13966)

Shixiaowei02
authored and
lfr-0531
committed
[None][perf] mHC fused_hc kernel optimizations + DS-V4 entry-boundary RMSNorm fold-in (#13892)

mingyangHao
authored and
lfr-0531
committed
[None][test] Add DeepSeek V4 CI coverage (#13988)

lfr-0531
and
Tracin
committed
[None][feat] Support NVFP4 dsv4 (#14026)

authored and
lfr-0531
committed
[None][feat] Enable MEGAMOE_DEEPGEMM backend for DeepSeek V4 (#14129)

Barry-Delaney
authored and
lfr-0531
committed
[None][perf] DSV4 multistream improvement for attention (#14142)

liji-nv
authored and
lfr-0531
committed
[None][fix] Support DeepSeekV4 routing in perfect-router planner (#14074)

qiaoxj07
authored and
lfr-0531
committed
[TRTLLM-12478][fix] Synchronize prepare inputs H2D copy (#14128)

jiaganc
authored and
lfr-0531
committed
[None][perf] DSV4 compressor: enable 4-warp Phase 3 reduction for MTP… (#14124)

mingyangHao
authored and
lfr-0531
committed
[None][fix] Make indexer top-K launch policy device-aware (#13886)

mingyangHao
authored and
lfr-0531
committed
[None][fix] Update DeepGEMM revision (#14150)

lfr-0531
and
Barry-Delaney
committed
[TRTLLM-12589][fix] Reset MoE A2A dispatch state on warmup OOM (#14000)

Barry-Delaney
authored and
lfr-0531
committed
[TRTLLM-12316][feat] Integrate FP4 indexer for DSv4 (#13575)

mikeiovine
and
lfr-0531
committed
[None][perf] Add CUDA q_b norm for DeepSeek V4 (#13975)

mingyangHao
authored and
lfr-0531
committed
[None][feat] Enable 2 DSv4 perf optimizations by default (#14120)

lishicheng1996-nv
authored and
lfr-0531
committed
[None][fix] Handle DeepSeek-V4 fused A scale shape (#14149)
lfr-0531
committed
[None][fix] Restore DSV4 routed-expert swiglu_limit on TRTLLM-Gen (#14238)

Barry-Delaney
authored and
lfr-0531
committed
[None][fix] Avoid dp_size x ep_size double-count in MegaMoEDeepGemm SymmBuffer (#14213)

qiaoxj07
authored and
lfr-0531
committed
[None][feat] DSv4: enable GVR Heuristic Top-K for compress_ratio=4 (#14219)

authored and
lfr-0531
committed
[None][fix] DSv4: drop duplicate indexer_k_dtype kwarg on V4 sparse attn (#14321)

Tabrizian
authored and
lfr-0531
committed
[None][fix] Warn on experimental DeepSeek-V4 base checkpoints (#14299)
lfr-0531
committed
[TRTLLM-12112][feat] Support v2 KV cache stats (#13953)

yizhang-nv
authored and
lfr-0531
committed
[None][perf] Enable PDL for DeepGEMM and related kernels (#14311)

liji-nv
authored and
lfr-0531
committed
[TRTLLM-12732][fix] Fence V2 `_batched_migrate` behind `execution_stream` (#14245)

authored and
lfr-0531
committed
[None][chore] cap RoPE cos/sin table to runtime max_seq_len (#14241)

lancelly
authored and
lfr-0531
committed
[None][fix] DSv4 indexer: stable radix aux scratch for CUDA Graph safety (#14297)

longcheng-nv
authored and
lfr-0531
committed
[None][fix] Resolve DSv4 rebase fallout
lfr-0531
committed
[None][fix] Resolve DSv4 CI 43359 failures
lfr-0531
committed
[None][fix] Stabilize MegaMoE MPI bootstrap
lfr-0531
committed