[#13819][feat] AutoDeploy: Qwen3.5 MoE (VLM) MTP

[govind-ramnarayan]

fixes: #13819

Parts to this change:

1. Support VLMs with speculative decoding in AutoDeploy. This is nontrivial because speculative decoding previously assumed that the target and draft were both GraphModules - now it needs to tolerate the target model being an eager nn.Module that wraps a GraphModule. We try to reach some mix of generality and not overengineering here. Main touch points are the factory code in eagle.py and modeling code in modeling_eagle.py, wherever we need to combine information from submodules into one (e.g. quantization configs in the factory) or split kwargs per submodule (when dispatching args to submodules in the model.forward()).
2. Add modeling code + tests for Qwen3.5 MoE + MTP with the sharding IR.
3. Support gated delta net models in speculative decoding.
4. Support disabling quantization on MTP heads while the base model has it. This feature also needed to be made robust to VLM's nontrivial export modules. Since we do MTP and Eagle in the same modeling code, when the MTP head is excluded from quantization, we need to map that exclusion to our own submodule prefix. It used to be "model", which could then conflict with existing exclusion-instructions for the target model, so we renamed it to "eagle_drafter" to avoid this conflict.

And then various unit and smoke tests.

The main e2e test is a qwen 3.5 moe nvfp4 + mtp accuracy test that also checks the acceptance rate at the other end.

Summary by CodeRabbit

Release Notes

• New Features

  • Added support for Qwen3.5 MoE 35B with MTP (Multi-Token Prediction) speculative decoding configuration and deployment.
  • Enhanced Eagle drafter model architecture to support additional model variants and improved speculative decoding integration.

• Tests

  • Added comprehensive test coverage for MTP speculative decoding functionality, accuracy validation, and export workflows.

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

[govind-ramnarayan]

/bot run --stage-list "DGX_B200-8_GPUs-AutoDeploy-Post-Merge-1"

[tensorrt-cicd]

PR_Github #53427 [ run ] triggered by Bot. Commit: 8f6842f Link to invocation

[govind-ramnarayan]

/bot run --stage-list "DGX_B200-8_GPUs-AutoDeploy-Post-Merge-1"

[tensorrt-cicd]

PR_Github #53439 [ run ] triggered by Bot. Commit: 318c6ba Link to invocation

[tensorrt-cicd]

PR_Github #53427 [ run ] completed with state ABORTED. Commit: 8f6842f

Link to invocation

[tensorrt-cicd]

PR_Github #53439 [ run ] completed with state FAILURE. Commit: 318c6ba
/LLM/main/L0_MergeRequest_PR pipeline #42607 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[govind-ramnarayan]

/bot run --stage-list "DGX_B200-8_GPUs-AutoDeploy-Post-Merge-1"

[tensorrt-cicd]

PR_Github #53616 [ run ] triggered by Bot. Commit: 318c6ba Link to invocation

[tensorrt-cicd]

PR_Github #53616 [ run ] completed with state SUCCESS. Commit: 318c6ba
/LLM/main/L0_MergeRequest_PR pipeline #42762 (Partly Tested) completed with status: 'SUCCESS'

CI Report

Link to invocation

[govind-ramnarayan]

/bot run --disable-fail-fast

[govind-ramnarayan]

PR_Github #53616 [ run ] completed with state SUCCESS. Commit: 318c6ba /LLM/main/L0_MergeRequest_PR pipeline #42762 (Partly Tested) completed with status: 'SUCCESS'

CI Report

Link to invocation

Note: Verified that this ran the new Qwen3.5 MoE nvfp4 + mtp test and passed. Will assume this is true unless drastic changes are made to the PR and just try to get pre-merge green

[tensorrt-cicd]

PR_Github #53740 [ run ] triggered by Bot. Commit: 318c6ba Link to invocation

[tensorrt-cicd]

PR_Github #53740 [ run ] completed with state SUCCESS. Commit: 318c6ba
/LLM/main/L0_MergeRequest_PR pipeline #42866 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation