[None][feat] AutoDeploy: enable attention-DP request balancer in SuperV3-MTP config

[MrGeva]

Summary

Enables the attention-DP request balancer (PyExecutor._balance_adp_requests) for the SuperV3-MTP AutoDeploy config by adding a top-level attention_dp_config block to the shipped examples/auto_deploy/model_registry/configs/super_v3_mtp.yaml:

attention_dp_config:
  enable_balance: true
  batching_wait_iters: 10
  timeout_iters: 16

Why

Under attention-DP, each MoE all-to-all is a collective that synchronizes all DP ranks every step. The rank holding the most sequences becomes a straggler that gates the collective, inflating inter-token latency. The balancer redistributes per-rank decode load so no single rank gates the collective.

Impact

• +64–111% output-token throughput at concurrency 128/256 on SuperV3-MTP NVFP4 attention-DP serving (SPEED-Bench).
• Trade-off: defers prefill, raising TTFT at high concurrency. Tune batching_wait_iters / timeout_iters when first-token latency is the priority.

Scope / safety

• Config-only change. No source-code default changes — the balancer stays opt-in. Configs that do not set attention_dp_config are unaffected.
• No-op unless attention-DP is active (multi-GPU with enable_attention_dp). Single-GPU and attn_dp_off paths are unchanged (the balancer requires tp_size > 1).
• The TestNemotronSuperV3::test_accuracy matrix drives enable_attention_dp via a detect_sharding kwarg override, so both attn_dp_off and attn_dp_on variants continue to exercise this config correctly.

Test Coverage

Covered by existing accuracy/test_llm_api_autodeploy.py::TestNemotronSuperV3::test_accuracy[*-4-attn_dp_on-trtllm].

🤖 Generated with Claude Code

Summary by CodeRabbit

Release Notes

• New Features
  • Added attention-DP request balancer configuration for SuperV3 MTP speculative decoding, enabling optimized decode load balancing across distributed ranks with tunable batching and timeout parameters.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 2b7b14e9-ef7c-49e8-82c0-981998c8fc4f

📥 Commits

Reviewing files that changed from the base of the PR and between 484e6c9 and 9528981.

📒 Files selected for processing (1)

• examples/auto_deploy/model_registry/configs/super_v3_mtp.yaml

---

📝 Walkthrough

Walkthrough

SuperV3 MTP configuration now includes an attention_dp_config section that enables request balancing for attention-distributed-pipeline decode with timing parameters batching_wait_iters: 10 and timeout_iters: 16. The configuration includes comments documenting that balancing is inactive unless attention-DP is enabled.

Changes

Attention-DP Request Balancer Configuration

Layer / File(s)	Summary
Attention-DP request balancer configuration with batching thresholds examples/auto_deploy/model_registry/configs/super_v3_mtp.yaml	Adds attention_dp_config block with enable_balance: true, batching_wait_iters: 10, and timeout_iters: 16 parameters, along with documentation describing the balancer behavior and its dependency on attention-DP being active.

Estimated Code Review Effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Possibly Related PRs

• NVIDIA/TensorRT-LLM#14943: Updates ADEngine to propagate attention_dp_config from ad_config instead of hardcoded defaults, directly implementing the YAML fields added in this PR.

Suggested Reviewers

• galagam
• HuiGao-NV
• symphonylyh

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and specifically describes the main change: enabling an attention-DP request balancer in the SuperV3-MTP config, following the required format with ticket ID, type, and concise summary.
Description check	✅ Passed	The PR description includes a clear Summary section explaining what was changed, Why section explaining the rationale, Impact section with performance metrics, Scope/safety section addressing safety concerns, and Test Coverage section. All critical information is present and well-organized.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}