[None][fix] Register Multimodal Placeholders for Qwen3.5 MoE VLM Serving

[anurags25]

Summary

PR #12114 fixed model internals (mRoPE, chunked prefill, expert routing) but multimodal placeholders were not registers so the serving layer was unable to process multimodal requests. This change completes the fix for #12271.

• Register MULTIMODAL_PLACEHOLDER_REGISTRY entry for qwen3_5_moe in the AutoDeploy custom model, so the serving layer can resolve image/video placeholder tokens during chat message parsing
• Introduce _Qwen35MoeVLMConfig subclass in config_utils.py to preserve model_type = "qwen3_5_moe" through the config normalization path (_Qwen35ConfigCompat converts to Qwen3NextConfig, which loses the original model_type)
• Placeholder tokens (<|vision_start|><|image_pad|><|vision_end|>, <|vision_start|><|video_pad|><|vision_end|>) match the PyTorch backend's qwen3_vl_moe registration

Test plan

• trtllm-serve <qwen3.5-35b-a3b> --backend=_autodeploy --config=<config.yaml> starts without error
• Text-only chat completions return correct output
• Multimodal chat completions (image_url) return correct image descriptions
• Dense Qwen3.5 variants (non-MoE) are unaffected by the config_utils change

Summary by CodeRabbit

• New Features
  • Added support for Qwen3.5 Mixture of Experts (MoE) model variant.
  • Enhanced multimodal handling for Qwen3.5 MoE, including proper image and video placeholder token mapping and content formatting support.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 06ffd74b-48c5-40a2-9f99-15b0ead01970

📥 Commits

Reviewing files that changed from the base of the PR and between 86f33e6 and 6b2b5dd.

📒 Files selected for processing (2)

• tensorrt_llm/_torch/auto_deploy/models/custom/modeling_qwen3_5_moe.py
• tensorrt_llm/_torch/pyexecutor/config_utils.py

---

📝 Walkthrough

Walkthrough

This PR adds multimodal support for Qwen3.5 MoE by introducing a config subclass to preserve the model type identifier during normalization and registering multimodal placeholder metadata with conditional imports.

Changes

Qwen3.5 MoE multimodal support

Layer / File(s)	Summary
Config class and model type preservation tensorrt_llm/_torch/pyexecutor/config_utils.py	_Qwen35MoeVLMConfig subclass forces model_type = "qwen3_5_moe" during normalization, and load_pretrained_config is updated to use this class instead of the base transformers.Qwen3NextConfig for qwen3_5_moe variants.
Multimodal imports and placeholder metadata registration tensorrt_llm/_torch/auto_deploy/models/custom/modeling_qwen3_5_moe.py	Optional multimodal imports expanded to include placeholder registry and metadata types with fallbacks to None, and runtime registration adds image/video placeholder token mappings and string formatting metadata for the "qwen3_5_moe" model.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Suggested reviewers

• hnover-nv
• galagam

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: registering multimodal placeholders for Qwen3.5 MoE VLM serving, which directly reflects the primary objective of the PR.
Description check	✅ Passed	The PR description provides a comprehensive summary, explains the issue and solution, lists test coverage with checkmarks, and addresses the PR checklist items appropriately.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[bmarimuthu-nv]

Thanks @anurags25 for the contribution!

[bmarimuthu-nv]

/bot run

[tensorrt-cicd]

PR_Github #52857 [ run ] triggered by Bot. Commit: 6b2b5dd Link to invocation

[tensorrt-cicd]

PR_Github #52857 [ run ] completed with state SUCCESS. Commit: 6b2b5dd
/LLM/main/L0_MergeRequest_PR pipeline #42110 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[bmarimuthu-nv]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #52962 [ run ] triggered by Bot. Commit: 6b2b5dd Link to invocation

[tensorrt-cicd]

PR_Github #52962 [ run ] completed with state SUCCESS. Commit: 6b2b5dd
/LLM/main/L0_MergeRequest_PR pipeline #42203 completed with status: 'SUCCESS'

CI Report

Link to invocation

[moraxu]

@anurags25 , just a note that I reverted your tensorrt_llm/_torch/pyexecutor/config_utils.py changes in my pending #14599 because I introduced a composite-config dispatch and moved Qwen35ConfigCompat to tensorrt_llm/_torch/models/modeling_qwen3_5.py