[https://nvbugs/5979673][fix] Pin nixl-cu13==0.9.0 instead of nixl meta-package by Shixiaowei02 · Pull Request #14567 · NVIDIA/TensorRT-LLM

Shixiaowei02 · 2026-05-26T09:53:35Z

The nixl PyPI package became a meta-package whose only dependency is nixl-cu12 with no version constraint, so nixl==0.9.0 resolves to nixl-cu12==<latest> (currently 1.1.0). The 1.1.0 bindings reference a new 3-arg overload of nixlAgent::prepXferDlist (with nixlAgentOptionalArgs) that does not exist in the source-built 0.9.0 libnixl.so shipped in the container, causing the test_agent unit test to fail at import time with an undefined-symbol error.

Pin nixl-cu13==0.9.0 directly so:

the wheel's _bindings.so only depends on the 4-arg prepXferDlist, which matches both the wheel's bundled libnixl and the source-built 0.9.0 libnixl;
the wheel variant aligns with the CUDA 13 container instead of cu12.

Summary by CodeRabbit

Chores
- Updated a development dependency to its CUDA 13–specific build to align dev tooling with CUDA 13 environments.

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

Shixiaowei02 · 2026-05-26T09:56:19Z

/bot run --disable-fail-fast

coderabbitai · 2026-05-26T09:56:28Z

📝 Walkthrough

Walkthrough

Replaces the development dependency nixl==0.9.0 with the CUDA 13-specific package nixl-cu13==0.9.0 in requirements-dev.txt.

Changes

CUDA 13 Dependency Update

Layer / File(s)	Summary
CUDA 13-specific nixl dependency `requirements-dev.txt`	Package dependency updated from `nixl==0.9.0` to `nixl-cu13==0.9.0`.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~2 minutes

Suggested reviewers

limin2021
chzblych
lfr-0531

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	PR description provides detailed context and solution, but does not follow the provided template with clearly filled-in sections like 'Description' and 'Test Coverage'.	Fill in the template sections: add a brief explanation in the 'Description' section and list relevant tests in 'Test Coverage' section that validate this dependency pinning change.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: pinning nixl-cu13==0.9.0 instead of the nixl meta-package, which directly addresses the dependency resolution issue described in the PR objectives.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

tensorrt-cicd · 2026-05-26T10:02:45Z

PR_Github #50341 [ run ] triggered by Bot. Commit: c510e75 Link to invocation

tensorrt-cicd · 2026-05-26T15:11:43Z

PR_Github #50341 [ run ] completed with state SUCCESS. Commit: c510e75
/LLM/main/L0_MergeRequest_PR pipeline #39868 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Shixiaowei02 · 2026-05-27T01:33:28Z

/bot run --stage-list "A10-PyTorch-1" --disable-fail-fast

tensorrt-cicd · 2026-05-27T01:39:09Z

PR_Github #50417 [ run ] triggered by Bot. Commit: 5cacbca Link to invocation

Shixiaowei02 · 2026-05-27T03:16:08Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-05-27T03:21:37Z

PR_Github #50438 [ run ] triggered by Bot. Commit: 4ab94b7 Link to invocation

tensorrt-cicd · 2026-05-27T03:28:00Z

PR_Github #50438 [ run ] completed with state FAILURE. Commit: 4ab94b7

Link to invocation

Shixiaowei02 · 2026-05-27T05:05:14Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-05-27T05:10:53Z

PR_Github #50461 [ run ] triggered by Bot. Commit: 120a83f Link to invocation

tensorrt-cicd · 2026-05-27T09:45:02Z

PR_Github #50461 [ run ] completed with state SUCCESS. Commit: 120a83f
/LLM/main/L0_MergeRequest_PR pipeline #39977 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

tensorrt-cicd · 2026-05-29T05:34:37Z

PR_Github #50976 [ run ] triggered by Bot. Commit: 747e3be Link to invocation

tensorrt-cicd · 2026-05-29T10:26:22Z

PR_Github #50976 [ run ] completed with state SUCCESS. Commit: 747e3be
/LLM/main/L0_MergeRequest_PR pipeline #40430 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Shixiaowei02 · 2026-06-01T09:51:05Z

/bot run --disable-fail-fast

The trtllm dev environment used to install the meta `nixl` package, which provides a top-level `nixl/__init__.py` shim that redirects to `nixl_cu13` / `nixl_cu12`. After NVIDIA#14436 switched the dependency to `nixl-cu13` directly, the meta shim is no longer present, so `from nixl import nixl_agent, ...` in `tensorrt_llm/_torch/disaggregation/nixl/_agent_py.py` raises `ModuleNotFoundError: No module named 'nixl'` at import time, which in turn breaks `test_agent_multi_backends::test_run_with_different_env[1]` during collection of the spawned subprocess pytest. Resolve the symbols from `nixl_cu13` (or `nixl_cu12`) first and fall back to the meta-package only when neither CUDA-specific package is installed. Signed-off-by: Shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>

This case (TRTLLM_USE_PY_NIXL_KVCACHE=1) was waived under nvbugs/5979673 because the meta `nixl` package pulled the latest `nixl-cu12` wheel, whose `_bindings.so` referenced symbols absent from the source-built 0.9.0 `libnixl.so`. With `nixl-cu13==0.9.0` directly pinned in requirements-dev.txt, the bindings and the libnixl in the container are ABI-consistent again, so the test is expected to pass. Signed-off-by: Shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>

tensorrt-cicd · 2026-06-01T09:56:25Z

PR_Github #51355 [ run ] triggered by Bot. Commit: b768245 Link to invocation

Shixiaowei02 · 2026-06-01T09:57:39Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-06-01T10:03:42Z

PR_Github #51356 [ run ] triggered by Bot. Commit: 7b73fda Link to invocation

tensorrt-cicd · 2026-06-01T10:08:08Z

PR_Github #51355 [ run ] completed with state ABORTED. Commit: b768245

Link to invocation

tensorrt-cicd · 2026-06-01T10:52:37Z

PR_Github #51356 [ run ] completed with state SUCCESS. Commit: 7b73fda
/LLM/main/L0_MergeRequest_PR pipeline #40767 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Shixiaowei02 · 2026-06-01T11:20:53Z

/bot run --disable-fail-fast

Shixiaowei02 · 2026-06-01T14:25:02Z

/bot run --disable-fail-fast

Shixiaowei02 · 2026-06-02T03:03:08Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-06-02T03:08:51Z

PR_Github #51485 [ run ] triggered by Bot. Commit: af5beae Link to invocation

tensorrt-cicd · 2026-06-02T07:32:32Z

PR_Github #51485 [ run ] completed with state SUCCESS. Commit: af5beae
/LLM/main/L0_MergeRequest_PR pipeline #40891 completed with status: 'SUCCESS'
Pipeline passed with automatic retried tests. Check the rerun report for details.

CI Report

Link to invocation

Shixiaowei02 · 2026-06-03T01:38:13Z

/bot skip --comment "CI had already passed before handling the waive file conflicts."

tensorrt-cicd · 2026-06-03T01:43:39Z

PR_Github #51720 [ skip ] triggered by Bot. Commit: b5da4cf Link to invocation

tensorrt-cicd · 2026-06-03T01:52:57Z

PR_Github #51720 [ skip ] completed with state SUCCESS. Commit: b5da4cf
Skipping testing for commit b5da4cf

Link to invocation

Shixiaowei02 · 2026-06-04T05:31:35Z

Moved to #14939

Shixiaowei02 requested a review from a team as a code owner May 26, 2026 09:53

github-actions Bot assigned Shixiaowei02 May 26, 2026

Shixiaowei02 requested review from bo-nv, chuangz0, xinhe-nv, yiqingy0 and zackyoray May 26, 2026 09:54

Shixiaowei02 force-pushed the user/xiaoweis/pin-nixl-cu13 branch from 4e43b56 to c510e75 Compare May 26, 2026 09:54

chuangz0 approved these changes May 26, 2026

View reviewed changes

roiedanino approved these changes May 26, 2026

View reviewed changes

zackyoray approved these changes May 26, 2026

View reviewed changes

Shixiaowei02 force-pushed the user/xiaoweis/pin-nixl-cu13 branch from c510e75 to 5cacbca Compare May 27, 2026 01:32

Shixiaowei02 force-pushed the user/xiaoweis/pin-nixl-cu13 branch from 5cacbca to 4ab94b7 Compare May 27, 2026 03:16

Shixiaowei02 force-pushed the user/xiaoweis/pin-nixl-cu13 branch from 4ab94b7 to 120a83f Compare May 27, 2026 05:04

xinhe-nv approved these changes May 27, 2026

View reviewed changes

Shixiaowei02 requested a review from a team as a code owner May 29, 2026 03:51

Shixiaowei02 requested review from dongxuy04 and leslie-fang25 May 29, 2026 03:51

Shixiaowei02 force-pushed the user/xiaoweis/pin-nixl-cu13 branch 2 times, most recently from c842a11 to b768245 Compare June 1, 2026 09:46

Shixiaowei02 added 2 commits June 1, 2026 02:56

Shixiaowei02 force-pushed the user/xiaoweis/pin-nixl-cu13 branch from b768245 to 7b73fda Compare June 1, 2026 09:57

Shixiaowei02 force-pushed the user/xiaoweis/pin-nixl-cu13 branch from 7b73fda to dcf3fde Compare June 1, 2026 11:20

Shixiaowei02 force-pushed the user/xiaoweis/pin-nixl-cu13 branch from dcf3fde to af5beae Compare June 2, 2026 03:02

Shixiaowei02 enabled auto-merge (squash) June 3, 2026 01:38

Shixiaowei02 marked this pull request as draft June 3, 2026 01:40

auto-merge was automatically disabled June 3, 2026 01:40
Pull request was converted to draft

Shixiaowei02 force-pushed the user/xiaoweis/pin-nixl-cu13 branch from b5da4cf to 7b73fda Compare June 4, 2026 05:21

Shixiaowei02 closed this Jun 4, 2026

Shixiaowei02 deleted the user/xiaoweis/pin-nixl-cu13 branch June 4, 2026 05:31

Uh oh!

Conversation

Shixiaowei02 commented May 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Description

Test Coverage

PR Checklist

GitHub Bot Help

Uh oh!

Shixiaowei02 commented May 26, 2026

Uh oh!

coderabbitai Bot commented May 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Suggested reviewers

❌ Failed checks (1 warning)

Uh oh!

tensorrt-cicd commented May 26, 2026

Uh oh!

tensorrt-cicd commented May 26, 2026

Uh oh!

Shixiaowei02 commented May 27, 2026

Uh oh!

tensorrt-cicd commented May 27, 2026

Uh oh!

Shixiaowei02 commented May 27, 2026

Uh oh!

tensorrt-cicd commented May 27, 2026

Uh oh!

tensorrt-cicd commented May 27, 2026

Uh oh!

Shixiaowei02 commented May 27, 2026

Uh oh!

tensorrt-cicd commented May 27, 2026

Uh oh!

tensorrt-cicd commented May 27, 2026

Uh oh!

tensorrt-cicd commented May 29, 2026

Uh oh!

tensorrt-cicd commented May 29, 2026

Uh oh!

Shixiaowei02 commented Jun 1, 2026

Uh oh!

tensorrt-cicd commented Jun 1, 2026

Uh oh!

Shixiaowei02 commented Jun 1, 2026

Uh oh!

tensorrt-cicd commented Jun 1, 2026

Uh oh!

tensorrt-cicd commented Jun 1, 2026

Uh oh!

tensorrt-cicd commented Jun 1, 2026

Uh oh!

Shixiaowei02 commented Jun 1, 2026

Uh oh!

Shixiaowei02 commented Jun 1, 2026

Uh oh!

Shixiaowei02 commented Jun 2, 2026

Uh oh!

tensorrt-cicd commented Jun 2, 2026

Uh oh!

tensorrt-cicd commented Jun 2, 2026

Uh oh!

Shixiaowei02 commented Jun 3, 2026

Uh oh!

tensorrt-cicd commented Jun 3, 2026

Uh oh!

tensorrt-cicd commented Jun 3, 2026

Uh oh!

Shixiaowei02 commented Jun 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Shixiaowei02 commented May 26, 2026 •

edited

Loading

coderabbitai Bot commented May 26, 2026 •

edited

Loading