[TRTLLM-14609][chore] Remove ENABLE_CONFIGURABLE_MOE escape hatch and remaining MoE legacy relics by xxi-nv · Pull Request #17009 · NVIDIA/TensorRT-LLM

xxi-nv · 2026-07-29T14:09:10Z

Summary

Final cleanup after the per-backend legacy MoE path removals (#16861-#16865). ConfigurableMoE has been the only execution path for the wrapped backends for months, so the ENABLE_CONFIGURABLE_MOE escape hatch and the state it kept alive are dead code. This PR removes them.

Description

create_moe: drop the ENABLE_CONFIGURABLE_MOE gate and the unreachable fallback-warning branch (CutlassFusedMoE is in the wrapped tuple, so that sub-branch could never run). WideEPMoE / TritonFusedMoE / VanillaMoE keep using create_moe_backend.
CutlassFusedMoE, TRTLLMGenFusedMoE: drop the never-read alltoall state (alltoall_method_type, alltoall_workspace, alltoall_prepare_workspace, use_low_precision_combine, moe_a2a) and the enable_alltoall override that always returned False — MoE.enable_alltoall already returns False.
without_comm: remove the vestigial parameter. No backend has read it since the comm-construction blocks were deleted; communication is owned by the ConfigurableMoE wrapper.
TRTLLMOpBackend: drop the unused MoeAlltoAll handle. The Python MoeAlltoAll class itself stays — AutoDeploy fused-MoE custom ops still construct it, so moe_alltoall.py is not removable.
Drop the now-nonexistent env var from the disaggregated perf-sanity configs and test docstrings.

No public API or user-facing behavior change: every removed symbol was either unread or a constant.

Test Coverage

tests/unittest/_torch/modules/moe/test_moe_module.py — full file, no -k filter, on GB200 (OCI, 100-real build at this commit):

Suite	Result
`test_configurable_moe_single_gpu` + `test_trtllm_gen_fp32_routing_bias`	253 passed, 847 skipped, 0 failed
`test_configurable_moe_multi_gpu` + `test_configurable_moe_multi_gpu_eplb` (5 ranks)	412 passed, 1058 skipped, 0 failed

The single-GPU numbers are byte-for-byte identical to a baseline run of the same command on the parent commit (main), executed against the same compiled binaries — this diff is Python-only, so the C++/CUDA artifacts are shared between the two runs.

PR Checklist

PR title uses the [TRTLLM-14609][chore] prefix
Commit is DCO signed off
Tests pass on GB200
No public API change

Dev Engineer Review

Removed the ENABLE_CONFIGURABLE_MOE gate and legacy fallback path; supported backends now consistently use ConfigurableMoE.
Removed the obsolete without_comm parameter and unused all-to-all state from MoE implementations and backend wiring.
Retained the Python MoeAlltoAll custom op while removing the unused TRTLLM backend handle.
Updated performance configurations and documentation to remove obsolete environment-variable references.
No public API or user-facing behavior changes are expected. Review should confirm all external callers have migrated from the removed constructor parameter.

QA Engineer Review

Modified tests/unittest/_torch/lora/test_moe_lora_model_path.py.
Updated documentation for test_configurable_moe_forward_impl_forwards_lora_params_to_scheduler; no test logic was changed.
No test-list files were modified, and coverage-list mapping was not indicated.
MoE single- and multi-GPU unit suites passed on GB200 with zero failures.

Verdict: sufficient

… remaining MoE legacy relics Final cleanup after the per-backend legacy MoE path removals (NVIDIA#16861-NVIDIA#16865). ConfigurableMoE has been the only execution path for the wrapped backends for months, so the env escape hatch and the state it kept alive are now dead. - create_moe: drop the ENABLE_CONFIGURABLE_MOE gate and the unreachable fallback-warning branch (CutlassFusedMoE is in the wrapped tuple, so that sub-branch could never run). WideEPMoE / TritonFusedMoE / VanillaMoE keep create_moe_backend. - CutlassFusedMoE, TRTLLMGenFusedMoE: drop the never-read alltoall state (alltoall_method_type, alltoall_workspace, alltoall_prepare_workspace, use_low_precision_combine, moe_a2a) and the enable_alltoall override that was always False; MoE.enable_alltoall already returns False. - Remove the vestigial without_comm parameter: no backend has read it since the comm construction blocks were deleted. - TRTLLMOpBackend: drop the unused MoeAlltoAll handle. The Python MoeAlltoAll class stays; AutoDeploy fused-MoE custom ops still construct it. - Drop the now-nonexistent env var from the disaggregated perf configs and test docstrings. Signed-off-by: xxi <xxi@nvidia.com>

xxi-nv · 2026-07-29T14:10:35Z

/bot run

coderabbitai · 2026-07-29T14:16:11Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: ff406c40-e724-45d5-ad58-852789310e25

📥 Commits

Reviewing files that changed from the base of the PR and between f20ea65 and f3b2b10.

📒 Files selected for processing (16)

tensorrt_llm/_torch/modules/fused_moe/configurable_moe.py
tensorrt_llm/_torch/modules/fused_moe/create_moe.py
tensorrt_llm/_torch/modules/fused_moe/fused_moe_cute_dsl.py
tensorrt_llm/_torch/modules/fused_moe/fused_moe_cutlass.py
tensorrt_llm/_torch/modules/fused_moe/fused_moe_deepgemm.py
tensorrt_llm/_torch/modules/fused_moe/fused_moe_densegemm.py
tensorrt_llm/_torch/modules/fused_moe/fused_moe_trtllm_gen.py
tensorrt_llm/_torch/modules/fused_moe/mega_moe/mega_moe_cute_dsl.py
tensorrt_llm/_torch/modules/fused_moe/mega_moe/mega_moe_deepgemm.py
tensorrt_llm/_torch/modules/fused_moe/moe_op_backend.py
tests/microbenchmarks/bench_moe_comm.py
tests/scripts/perf-sanity/disaggregated/gb200_deepseek-v32-fp4_32k4k_con256_ctx1_dep8_gen1_dep8_eplb0_mtp0_ccb-NIXL.yaml
tests/scripts/perf-sanity/disaggregated/gb300_deepseek-v32-fp4_32k4k_con256_ctx1_dep8_gen1_dep8_eplb0_mtp0_ccb-NIXL.yaml
tests/scripts/perf/disaggregated/gb200_deepseek-v32-fp4_32k4k_con256_ctx1_dep8_gen1_dep8_eplb0_mtp0_ccb-NIXL.yaml
tests/scripts/perf/disaggregated/gb300_deepseek-v32-fp4_32k4k_con256_ctx1_dep8_gen1_dep8_eplb0_mtp0_ccb-NIXL.yaml
tests/unittest/_torch/lora/test_moe_lora_model_path.py

💤 Files with no reviewable changes (6)

tensorrt_llm/_torch/modules/fused_moe/moe_op_backend.py
tensorrt_llm/_torch/modules/fused_moe/mega_moe/mega_moe_deepgemm.py
tensorrt_llm/_torch/modules/fused_moe/fused_moe_cute_dsl.py
tensorrt_llm/_torch/modules/fused_moe/mega_moe/mega_moe_cute_dsl.py
tensorrt_llm/_torch/modules/fused_moe/fused_moe_deepgemm.py
tests/microbenchmarks/bench_moe_comm.py

Walkthrough

The MoE factory now selects ConfigurableMoE directly for supported backends, removes without_comm from backend construction APIs, and eliminates legacy all-to-all state wiring. Related benchmarks, performance configurations, and LoRA test documentation are updated.

Changes

Configurable MoE backend cleanup

Layer / File(s)	Summary
Factory and wrapper routing `tensorrt_llm/_torch/modules/fused_moe/create_moe.py`, `tensorrt_llm/_torch/modules/fused_moe/configurable_moe.py`	Removes environment-variable gating and `without_comm` propagation, directly wrapping the configured backend classes with `ConfigurableMoE`.
Backend constructor and communication state cleanup `tensorrt_llm/_torch/modules/fused_moe/fused_moe_.py`, `tensorrt_llm/_torch/modules/fused_moe/mega_moe/`, `tensorrt_llm/_torch/modules/fused_moe/moe_op_backend.py`	Removes `without_comm`, legacy all-to-all fields, cached enablement properties, and `MoeAlltoAll` wiring while updating remaining constructor parameters.
Benchmark and configuration updates `tests/microbenchmarks/bench_moe_comm.py`, `tests/scripts/perf/disaggregated/.yaml`, `tests/unittest/_torch/lora/test_moe_lora_model_path.py`	Removes `ENABLE_CONFIGURABLE_MOE=1` from worker environments, updates benchmark construction, and documents the current LoRA forwarding path.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant MoEFactory
  participant ConfigurableMoE
  participant MoEBackend
  MoEFactory->>ConfigurableMoE: select supported backend
  ConfigurableMoE->>MoEBackend: create with init_load_balancer=False
  MoEBackend-->>ConfigurableMoE: return initialized backend
  ConfigurableMoE-->>MoEFactory: return configured MoE module

Possibly related PRs

NVIDIA/TensorRT-LLM#16861: Modifies related legacy all-to-all wiring in CutlassFusedMoE.
NVIDIA/TensorRT-LLM#16862: Changes related communication-state construction in TRTLLMGenFusedMoE.
NVIDIA/TensorRT-LLM#16863: Updates adjacent legacy communication methods in CuteDslFusedMoE.

Suggested labels: api-breaking

Suggested reviewers: brnguyen2

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 54.55% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is concise and accurately summarizes the main cleanup: removing ENABLE_CONFIGURABLE_MOE and legacy MoE remnants.
Description check	✅ Passed	The description follows the template and includes summary, detailed changes, test coverage, and checklist items.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands.}

tensorrt-cicd · 2026-07-29T14:16:53Z

PR_Github #62529 [ run ] triggered by Bot. Commit: f3b2b10 Link to invocation

tensorrt-cicd · 2026-07-29T16:31:41Z

PR_Github #62529 [ run ] completed with state SUCCESS. Commit: f3b2b10
/LLM/main/L0_MergeRequest_PR pipeline #50672 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

xxi-nv · 2026-07-30T00:22:44Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-30T00:31:22Z

PR_Github #62637 [ run ] triggered by Bot. Commit: f3b2b10 Link to invocation

tensorrt-cicd · 2026-07-30T07:55:37Z

PR_Github #62637 [ run ] completed with state SUCCESS. Commit: f3b2b10
/LLM/main/L0_MergeRequest_PR pipeline #50777 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

xxi-nv · 2026-07-30T07:58:08Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-30T08:06:22Z

PR_Github #62736 [ run ] triggered by Bot. Commit: f3b2b10 Link to invocation

tensorrt-cicd · 2026-07-30T10:03:09Z

PR_Github #62736 [ run ] completed with state FAILURE. Commit: f3b2b10
/LLM/main/L0_MergeRequest_PR pipeline #50870 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

xxi-nv · 2026-07-30T10:03:55Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-30T10:09:53Z

PR_Github #62758 [ run ] triggered by Bot. Commit: f3b2b10 Link to invocation

xxi-nv requested review from a team as code owners July 29, 2026 14:09

xxi-nv requested review from Shixiaowei02, allisonlim-nv, dc3671, leslie-fang25, nv-ananjappa, pcastonguay and rosong11 July 29, 2026 14:09

github-actions Bot assigned xxi-nv Jul 29, 2026

leslie-fang25 approved these changes Jul 30, 2026

View reviewed changes

dc3671 approved these changes Jul 30, 2026

View reviewed changes

Uh oh!

Conversation

xxi-nv commented Jul 29, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Description

Test Coverage

PR Checklist

Dev Engineer Review

QA Engineer Review

Uh oh!

xxi-nv commented Jul 29, 2026

Uh oh!

coderabbitai Bot commented Jul 29, 2026

Walkthrough

Changes

Sequence Diagram(s)

❌ Failed checks (1 warning)

Uh oh!

tensorrt-cicd commented Jul 29, 2026

Uh oh!

tensorrt-cicd commented Jul 29, 2026

Uh oh!

xxi-nv commented Jul 30, 2026

Uh oh!

tensorrt-cicd commented Jul 30, 2026

Uh oh!

tensorrt-cicd commented Jul 30, 2026

Uh oh!

xxi-nv commented Jul 30, 2026

Uh oh!

tensorrt-cicd commented Jul 30, 2026

Uh oh!

tensorrt-cicd commented Jul 30, 2026

Uh oh!

xxi-nv commented Jul 30, 2026

Uh oh!

tensorrt-cicd commented Jul 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

xxi-nv commented Jul 29, 2026 •

edited by coderabbitai Bot

Loading