Fix ep deployment issues

[CUHKSZzxy]

Related

DeepEP mode switching and buffer clear updates, credit to @SHshenhao
Those fixes in the current PR refer to the following PR

• update dlblas with deepep  #4126

And huge thanks to collaborators who upgraded DeepEP, DeepGEMM in DLBlas

• Sh/update deepep support DeepLink-org/DLBlas#86, @SHshenhao
• upgrade deep_gemm 2.1.1+c9f8b34 DeepLink-org/DLBlas#83, @zhaochaoxing

Modifications

1. Upgrade DeepEP / DeepGEMM / DLBlas / FlashMLA

• DeepEP -> v1.2.1
• DeepGEMM -> v2.1.1.post3
• DLBlas -> v0.0.6
• FlashMLA -> commit 1408756 (no official release, pick the latest one)

2. Bring back GDRCopy

• According to the official guide, to enable NVSHMEM IBGDA support, we can either (1) modify host driver config and reboot, or (2) install GDRCopy and load the gdrdrv kernel module. When modifying the driver is not an option, GDRCopy should be installed. Some concerns discussed in Questions about environment setups deepseek-ai/DeepEP#486

• vLLM dockerfile, SGLang dockerfile both install GDRCopy.

• vLLM ep kernels emphasize that enabling IBGDA is crucial for multinode DeepEP deployment. vLLM expert parallel deployment lists installing GDRCopy as a necessary step.

3. Expose DeepEP num_sms env var

Default DeepEP buffer num sms will raise the following errors on H200 multi-nodes. Therefore, we expose this environment variable to users for configuration. A feasible value on H200 is DEEPEP_BUFFER_NUM_SMS=16.

csrc/kernels/internode.cu:386, condition: ibgda_get_state()->num_rc_per_pe == num_channels or ibgda_get_state()->num_rc_per_pe >= num_sms

This is a known issue in deepep

• By default, in megatron, condition fais on ibgda_get_state()->num_rc_per_pe == num_channels || ibgda_get_state()->num_rc_per_pe >= num_sms  deepseek-ai/DeepEP#226

4. Fix DeepEP mode in CUDA graph

Flip DeepEP mode between prefill and decode, and also clear the buffer (performed by the DLBLas side when setting to low latency). Otherwise, it will trigger CUDA illegal memory access in deepep or the following deepgemm kernel, as known in

• Manually flip deepep_mode for cuda_graph sgl-project/sglang#11666

5. Others

• Add some deep_gemm CUDA dependencies
• Pin torch version to avoid build / runtime version mismatch (leads to undefined symbol for deep_gemm)
• Add vim
• Add some comments

[lvhan028]

@CUHKSZzxy test_docker workflow failed

[lvhan028]

Will it affect on A100 platform?

[CUHKSZzxy]

GDRCopy is used in the case of DeepEP, theoretically wont affect non-Hopper devices. But haven't tested on A100 yet

[grimoire]

LGTM