[None][fix] trtllm-serve VisualGen: bind HTTP port only on rank 0 in multi-rank launch

[luyiyun1021]

Summary by CodeRabbit

• Bug Fixes
  • Improved server startup reliability when running in multi-rank launch setups.
  • Non-primary ranks now avoid trying to bind the same network port, reducing startup conflicts and race conditions.

Description

In external multi-rank launch (torchrun / srun), launch_visual_gen_server binds the HTTP socket before constructing VisualGen, but rank>0 processes only exit into worker mode inside VisualGen.__init__. On a multi-GPU node every local rank therefore races to bind the same host:port and all but one die with EADDRINUSE, so no server ever becomes ready. This is a regression from #14355 (nvbugs/6143883), which moved the bind ahead of the minutes-long pipeline init to reserve the port; that is correct for a single server rank but makes every rank bind under multi-rank-per-node launch.

Fix: before the early bind, detect external launch and let rank>0 construct VisualGen (which dispatches to the worker loop and never returns) and return without touching the port. Rank 0 and single-node (mp.Process) keep the early-bind behavior unchanged.

Test Coverage

Verified on a 16-GPU (4 nodes × 4) LTX-2 distributed serve on GB200 (NVFP4, cfg2 × ulysses8, VAE-parallel 8): before the fix all 16 ranks fail with [Errno 98] Address already in use; after the fix rank 0 binds, ranks 1-15 become workers, and the server reaches Application startup complete and serves generations.

PR Checklist

• PR description clearly explains what and why.

• PR Follows TRT-LLM CODING GUIDELINES.

• Test cases are provided for new code paths.

• Any new dependencies have been scanned for license and vulnerabilities.

• CODEOWNERS updated if ownership changes.

• Documentation updated as needed.

• Update tava architecture diagram if significant design change.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 5faeaec3-2dac-42c5-a54f-00e4c12069f6

📥 Commits

Reviewing files that changed from the base of the PR and between 0b0fd54 and 55455db.

📒 Files selected for processing (1)

• tensorrt_llm/commands/serve.py

---

📝 Walkthrough

Walkthrough

The launch_visual_gen_server function in serve.py now checks for external multi-rank launch and the current rank. Non-zero ranks initialize VisualGen and return early, skipping HTTP socket binding and FastAPI server startup to avoid port conflicts.

Changes

VisualGen Rank Gating

Layer / File(s)	Summary
Early-return rank gate for non-zero ranks tensorrt_llm/commands/serve.py	Added detection of external multi-rank launch; if the current rank is non-zero, VisualGen is constructed and the function returns before binding the HTTP listening socket or starting the server.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is concise, specific, and accurately summarizes the main fix to VisualGen port binding in multi-rank launches.
Description check	✅ Passed	The description follows the template with Description, Test Coverage, and PR Checklist sections and provides sufficient detail.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[luyiyun1021]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #58622 [ run ] triggered by Bot. Commit: 55455db Link to invocation

[tensorrt-cicd]

PR_Github #58622 [ run ] completed with state FAILURE. Commit: 55455db
/LLM/main/L0_MergeRequest_PR pipeline #47210 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[luyiyun1021]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #58680 [ run ] triggered by Bot. Commit: 55455db Link to invocation

[tensorrt-cicd]

PR_Github #58680 [ run ] completed with state SUCCESS. Commit: 55455db
/LLM/main/L0_MergeRequest_PR pipeline #47266 completed with status: 'SUCCESS'

CI Report

Link to invocation