Skip to content

[ascend]adapt for s1-pro dp*tp+ep#4380

Merged
lvhan028 merged 2 commits intoInternLM:mainfrom
DeepLink-org:multinodes_ep
Mar 4, 2026
Merged

[ascend]adapt for s1-pro dp*tp+ep#4380
lvhan028 merged 2 commits intoInternLM:mainfrom
DeepLink-org:multinodes_ep

Conversation

@yao-fengchen
Copy link
Copy Markdown
Collaborator

No description provided.

@yao-fengchen yao-fengchen changed the title adapt for s1-pro dp*tp+ep [ascend]adapt for s1-pro dp*tp+ep Feb 28, 2026
@jinminxi104 jinminxi104 requested a review from Copilot March 4, 2026 06:18
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adapts the dlinfer (Ascend) backend to support dp*tp+ep parallelism configurations. It makes two changes:

  1. Adds FopeRotaryEmbeddingImpl (Fractional Rotary Positional Embedding) support to the dlinfer rotary embedding builder, reusing the default backend's implementation.
  2. Enables group-based router support (n_groups) in DlinferSoftmaxTopKImpl by removing the prior NotImplementedError and instead forwarding n_groups to moe_metadata during the forward pass.

Changes:

  • Add FopeRotaryEmbeddingImpl support for RopeType.Fope in the dlinfer rotary embedding builder.
  • Remove the NotImplementedError for group routing in DlinferSoftmaxTopKImpl, storing n_groups and propagating it to moe_metadata during inference.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
lmdeploy/pytorch/backends/dlinfer/rotary_embedding.py Imports FopeRotaryEmbeddingImpl from the default backend and adds a RopeType.Fope case to DlinferRotaryEmbeddingBuilder.build, mirroring the default builder.
lmdeploy/pytorch/backends/dlinfer/moe.py Replaces the NotImplementedError for grouped routing with a stored n_groups field and assigns it to moe_metadata.router_n_groups in forward.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@jinminxi104 jinminxi104 marked this pull request as ready for review March 4, 2026 08:29
@lvhan028 lvhan028 merged commit f54a2f6 into InternLM:main Mar 4, 2026
4 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants