[TRTLLMINF-127][infra] Upgrade dependencies for NGC PyTorch 26.05 stack - #15087
Conversation
📝 WalkthroughWalkthroughThis PR consolidates framework ecosystem upgrades (CUDA 13.2.1, PyTorch 2.11.0/2.12.0, TensorRT 10.16.1) across Docker images, installation scripts, and CI configuration; refactors C++ CUDA memory initialization for consistency; adds CMake compiler compatibility safeguards; migrates PyTorch compilation backend to compile_fx_inner; and updates test waivers and configuration. ChangesVersion Upgrade and Infrastructure
C++ CUDA Virtual Memory Initialization
Build Configuration and Compiler Compatibility
PyTorch Compilation Backend Migration
Test and CI Configuration Updates
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (6)
cpp/include/tensorrt_llm/runtime/virtualMemory.h (1)
2-2:⚠️ Potential issue | 🟠 Major | ⚡ Quick winUpdate the NVIDIA copyright year for this modified file.
This file was meaningfully modified in this PR, so the header year should be updated to the latest modification year.
As per coding guidelines, “update year on modified files” and include the latest meaningful modification year in NVIDIA copyright headers.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@cpp/include/tensorrt_llm/runtime/virtualMemory.h` at line 2, Update the copyright header in cpp/include/tensorrt_llm/runtime/virtualMemory.h to reflect the latest modification year (change "2025" to "2026"); locate the top-of-file copyright comment block in virtualMemory.h and replace the year only, preserving the existing NVIDIA CORPORATION wording and formatting.Source: Coding guidelines
cpp/tensorrt_llm/runtime/virtualMemory.cpp (1)
2-2:⚠️ Potential issue | 🟠 Major | ⚡ Quick winPlease bump the copyright year to the latest modification year.
This source file is changed in the PR, so the NVIDIA header year should reflect the latest meaningful modification.
As per coding guidelines, modified source files must carry an NVIDIA copyright header updated to the latest meaningful modification year.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@cpp/tensorrt_llm/runtime/virtualMemory.cpp` at line 2, Update the NVIDIA copyright header year in cpp/tensorrt_llm/runtime/virtualMemory.cpp to the latest modification year (2026); locate the file-level header comment at the top of virtualMemory.cpp and replace "2025" with "2026" so the file's copyright reflects the most recent meaningful change.Source: Coding guidelines
cpp/tests/unit_tests/common/cudaDriverWrapperTest.cpp (1)
2-2:⚠️ Potential issue | 🟠 Major | ⚡ Quick winHeader year is stale for a modified test file.
Please update the NVIDIA copyright year/range to include the latest meaningful modification year.
As per coding guidelines, all modified source files (including
.cpptests) must have NVIDIA copyright headers updated to the latest meaningful modification year.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@cpp/tests/unit_tests/common/cudaDriverWrapperTest.cpp` at line 2, The file's copyright header shows a stale year range ("2022-2024"); update the header block at the top of the file to include the current meaningful modification year by extending or replacing the year/range (e.g., change "2022-2024" to include 2026 or the correct end year), ensuring the comment format and spacing remain identical and that the updated string appears where the existing copyright line is located.Source: Coding guidelines
cpp/tests/unit_tests/runtime/virtualMemoryTest.cpp (1)
2-2:⚠️ Potential issue | 🟠 Major | ⚡ Quick winUpdate copyright year on this modified test file.
Since this file has meaningful changes in the PR, the header year should be updated to the latest modification year.
As per coding guidelines, modified TensorRT-LLM source files (
.py/.cpp/.h/.cu/.cuh) must carry an NVIDIA copyright header with the latest meaningful modification year.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@cpp/tests/unit_tests/runtime/virtualMemoryTest.cpp` at line 2, Update the copyright header year in the file's top comment from 2025 to the current modification year (2026); edit the header line containing "Copyright (c) 2025, NVIDIA CORPORATION." to read "Copyright (c) 2026, NVIDIA CORPORATION." so the modified test file's header reflects the latest meaningful change.Source: Coding guidelines
scripts/rename_docker_images.py (1)
1-7:⚠️ Potential issue | 🟠 Major | ⚡ Quick winAdd the required NVIDIA OSS copyright header to this modified Python file.
This source file is modified in this PR but does not include the required NVIDIA Apache-2.0 header block with the latest modification year.
Proposed fix
#!/usr/bin/env python3 +# +# Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + import argparse as _apAs per coding guidelines, all TensorRT-LLM source files (including
.py) must carry the NVIDIA copyright header with the latest meaningful modification year.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/rename_docker_images.py` around lines 1 - 7, Add the required NVIDIA Apache-2.0 copyright header block (with the latest meaningful modification year) to the top of this Python module so it precedes the existing shebang ("#!/usr/bin/env python3") and imports; ensure the full multi-line copyright and license text used across the repo is included verbatim and retains the shebang as the first executable line after the header, updating the year to reflect this PR's modification.Source: Coding guidelines
jenkins/L0_Test.groovy (1)
4388-4395:⚠️ Potential issue | 🟠 Major | ⚡ Quick win
PY312-DLFWnow builds in one image but version-tags from another.After switching this config to
LLM_ROCKYLINUX8_PY312_DOCKER_IMAGE, the laterversionLocal = "ngcpytorch..."logic still parsesLLM_DOCKER_IMAGE. That decouples the wheel's local version suffix from the actual DLFW runtime image used by this job and can make the pip-install sanity check validate the wrong stack label. Derive the suffix from the DLFW image for this config instead of the globalLLM_DOCKER_IMAGE.Suggested fix
- if (isDlfw) { - // Extract PyTorch version from LLM_DOCKER_IMAGE. e.g. pytorch-26.02 -> 2602 - def matcher = LLM_DOCKER_IMAGE =~ /:pytorch-(\d+)\.(\d+)-/ + if (isDlfw) { + // Derive the stack suffix from the DLFW runtime image used by this sanity job. + def matcher = values[5] =~ /\/pytorch:(\d+)\.(\d+)-/ if (!matcher.find()) { - error "Failed to extract PyTorch version from LLM_DOCKER_IMAGE: ${LLM_DOCKER_IMAGE}" + error "Failed to extract PyTorch version from DLFW image: ${values[5]}" } versionLocal = "ngcpytorch${matcher.group(1)}${matcher.group(2)}" }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@jenkins/L0_Test.groovy` around lines 4388 - 4395, The version suffix derivation still uses LLM_DOCKER_IMAGE even when the job uses LLM_ROCKYLINUX8_PY312_DOCKER_IMAGE and DLFW_IMAGE is the runtime that builds PY312-DLFW; update the logic that sets versionLocal to detect when the job image equals LLM_ROCKYLINUX8_PY312_DOCKER_IMAGE (or when PY312 DLFW is in use) and parse the local version suffix from DLFW_IMAGE instead of LLM_DOCKER_IMAGE. Locate the code that computes versionLocal and replace or branch that parsing to use DLFW_IMAGE (keeping existing parsing code) when the selected image variable is LLM_ROCKYLINUX8_PY312_DOCKER_IMAGE so the wheel suffix matches the actual DLFW runtime.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@jenkins/L0_Test.groovy`:
- Around line 4069-4071: The PR removes the only Verl job from launchTestJobs
(the commented-out "DGX_B200-4_GPUs-Verl-Post-Merge-1" entry), which eliminates
Verl CI coverage; restore coverage by either re-enabling that original Verl
entry in launchTestJobs or replace it with a smaller waived/smoke Verl stage
(e.g., add a single-GPU or waived "Verl" entry) so the Verl path is still
exercised while nvbugs/6236818 is blocked; locate the launchTestJobs array and
update the entries (referencing "DGX_B200-4_GPUs-Verl-Post-Merge-1" or a new
name like "Verl-Smoke"/"l0_verl_smoke") accordingly.
- Line 2256: The shell invocation sets NVIDIA_TRITON_SERVER_VERSION=26.04 which
is out of sync with the runtime tag r26.05; update the exported env var in the
sh call so the check_test_list.py run uses the same Triton version as the
runtime (e.g., change NVIDIA_TRITON_SERVER_VERSION=26.04 to 26.05), or
preferably read the runtime tag variable if one exists and pass that (ensure the
sh command that runs python3 ${llmSrc}/scripts/check_test_list.py uses
NVIDIA_TRITON_SERVER_VERSION matching the runtime tag).
- Line 1: The Jenkinsfile is pinned to a mutable personal branch via the
`@Library` annotation (bloom-jenkins-shared-lib@emma/update_nsc_login_node) which
creates a fragile dependency; either merge the shared-lib changes into the
canonical reviewed ref and update the `@Library` annotation to use that stable ref
(e.g., main or a release tag), or replace the branch name with an immutable
commit SHA or tag in the `@Library` declaration so L0 no longer depends on a
mutable personal branch.
In `@README.md`:
- Line 12: Update the Torch badge in README.md to reflect the new default stack
by replacing the existing
"[](https://pytorch.org)"
string with the matching version
"[](https://pytorch.org)";
ensure this matches the TORCH_VERSION="2.12.0" value used in
docker/common/install_pytorch.sh so the README and install script remain
consistent.
---
Outside diff comments:
In `@cpp/include/tensorrt_llm/runtime/virtualMemory.h`:
- Line 2: Update the copyright header in
cpp/include/tensorrt_llm/runtime/virtualMemory.h to reflect the latest
modification year (change "2025" to "2026"); locate the top-of-file copyright
comment block in virtualMemory.h and replace the year only, preserving the
existing NVIDIA CORPORATION wording and formatting.
In `@cpp/tensorrt_llm/runtime/virtualMemory.cpp`:
- Line 2: Update the NVIDIA copyright header year in
cpp/tensorrt_llm/runtime/virtualMemory.cpp to the latest modification year
(2026); locate the file-level header comment at the top of virtualMemory.cpp and
replace "2025" with "2026" so the file's copyright reflects the most recent
meaningful change.
In `@cpp/tests/unit_tests/common/cudaDriverWrapperTest.cpp`:
- Line 2: The file's copyright header shows a stale year range ("2022-2024");
update the header block at the top of the file to include the current meaningful
modification year by extending or replacing the year/range (e.g., change
"2022-2024" to include 2026 or the correct end year), ensuring the comment
format and spacing remain identical and that the updated string appears where
the existing copyright line is located.
In `@cpp/tests/unit_tests/runtime/virtualMemoryTest.cpp`:
- Line 2: Update the copyright header year in the file's top comment from 2025
to the current modification year (2026); edit the header line containing
"Copyright (c) 2025, NVIDIA CORPORATION." to read "Copyright (c) 2026, NVIDIA
CORPORATION." so the modified test file's header reflects the latest meaningful
change.
In `@jenkins/L0_Test.groovy`:
- Around line 4388-4395: The version suffix derivation still uses
LLM_DOCKER_IMAGE even when the job uses LLM_ROCKYLINUX8_PY312_DOCKER_IMAGE and
DLFW_IMAGE is the runtime that builds PY312-DLFW; update the logic that sets
versionLocal to detect when the job image equals
LLM_ROCKYLINUX8_PY312_DOCKER_IMAGE (or when PY312 DLFW is in use) and parse the
local version suffix from DLFW_IMAGE instead of LLM_DOCKER_IMAGE. Locate the
code that computes versionLocal and replace or branch that parsing to use
DLFW_IMAGE (keeping existing parsing code) when the selected image variable is
LLM_ROCKYLINUX8_PY312_DOCKER_IMAGE so the wheel suffix matches the actual DLFW
runtime.
In `@scripts/rename_docker_images.py`:
- Around line 1-7: Add the required NVIDIA Apache-2.0 copyright header block
(with the latest meaningful modification year) to the top of this Python module
so it precedes the existing shebang ("#!/usr/bin/env python3") and imports;
ensure the full multi-line copyright and license text used across the repo is
included verbatim and retains the shebang as the first executable line after the
header, updating the year to reflect this PR's modification.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 7d1657c2-96eb-487f-978c-9f9a2bb50974
📒 Files selected for processing (24)
README.mdconstraints.txtcpp/include/tensorrt_llm/runtime/virtualMemory.hcpp/tensorrt_llm/deep_ep/CMakeLists.txtcpp/tensorrt_llm/flash_mla/CMakeLists.txtcpp/tensorrt_llm/runtime/virtualMemory.cppcpp/tests/unit_tests/common/cudaDriverWrapperTest.cppcpp/tests/unit_tests/runtime/virtualMemoryTest.cppdocker/Dockerfile.multidocker/Makefiledocker/common/install_cuda_toolkit.shdocker/common/install_pytorch.shdocker/common/install_tensorrt.shdocs/source/legacy/reference/support-matrix.mdjenkins/Build.groovyjenkins/L0_Test.groovyjenkins/current_image_tags.propertiesrequirements.txtscripts/rename_docker_images.pytensorrt_llm/_torch/compilation/backend.pytensorrt_llm/_torch/compilation/piecewise_optimizer.pytests/integration/defs/verl/verl_config.ymltests/integration/test_lists/waives.txttests/unittest/auto_deploy/singlegpu/custom_ops/moe/test_trtllm_moe.py
💤 Files with no reviewable changes (1)
- constraints.txt
EmmaQiaoCh
changed the title
|
/bot run --stage-list "Build-Docker-Images" |
|
PR_Github #52696 [ run ] triggered by Bot. Commit: |
|
/bot run --stage-list "Build-Docker-Images" |
|
PR_Github #52786 [ run ] triggered by Bot. Commit: |
|
PR_Github #52696 [ run ] completed with state |
|
PR_Github #52786 [ run ] completed with state
|
|
/bot run --stage-list "Build-Docker-Images" |
|
PR_Github #52921 [ run ] triggered by Bot. Commit: |
|
PR_Github #52921 [ run ] completed with state
|
EmmaQiaoCh
changed the title
|
/bot run --stage-list "Build-Docker-Images" |
|
PR_Github #53307 [ run ] triggered by Bot. Commit: |
|
PR_Github #53307 [ run ] completed with state
|
|
PR_Github #58844 [ run ] triggered by Bot. Commit: |
|
PR_Github #58844 [ run ] completed with state
|
|
/bot run --only-multi-gpu-test --disable-fail-fast |
|
PR_Github #58874 [ run ] triggered by Bot. Commit: |
|
/bot run --post-merge --only-multi-gpu-test --disable-fail-fast |
|
PR_Github #58910 [ run ] triggered by Bot. Commit: |
|
PR_Github #58874 [ run ] completed with state
|
Signed-off-by: EmmaQiaoCh <qqiao@nvidia.com>
|
/bot run --post-merge --disable-fail-fast |
|
PR_Github #58968 [ run ] triggered by Bot. Commit: |
|
PR_Github #58910 [ run ] completed with state |
|
PR_Github #58968 [ run ] completed with state
|
Signed-off-by: EmmaQiaoCh <qqiao@nvidia.com>
|
/bot run --stage-list "GB200-4_GPUs-PyTorch-Post-Merge-1,DGX_H100-4_GPUs-PyTorch-DeepSeek-1,DGX_H200-4_GPUs-PyTorch-Post-Merge-1,DGX_B200-4_GPUs-PyTorch-Post-Merge-3,DGX_H100-4_GPUs-PyTorch-DeepSeek-1 ,RTXPro6000D-4_GPUs-PyTorch-Post-Merge-1 ,RTXPro6000D-4_GPUs-PyTorch-Post-Merge-2" --disable-fail-fast |
|
PR_Github #59081 [ run ] triggered by Bot. Commit: |
Co-authored-by: Yanchao Lu <yanchaol@nvidia.com> Signed-off-by: Emma Qiao <qqiao@nvidia.com>
|
PR_Github #59081 [ run ] completed with state
|
Signed-off-by: EmmaQiaoCh <qqiao@nvidia.com>
|
/bot skip --comment "Waive the failed cases in CI and replaying jobs" |
|
PR_Github #59205 [ skip ] triggered by Bot. Commit: |
|
PR_Github #59205 [ skip ] completed with state |
…GB300 NVFP4 GEMM heuristic regression cuBLAS 13.4.1.2 (introduced by PR NVIDIA#15087 in the 26.05 stack upgrade) picks a slower NVFP4 GEMM plan on GB300 for the GLM-5 dense-attention prefill shapes, driving output_token_throughput -32% (4.07 -> 2.75) and per-request TTFT +42% (247 ms -> 352 ms) on aggr_upload-ctx_only-gb300_glm-5-fp4_1k1k_con1_ctx1_dep2_gen1_tep4_eplb0_mtp3_ccb-NIXL. This is a surgical alternative to NVIDIA#16601, which fully reverted the 26.05 -> 26.04 stack (container tag + cuBLAS + cuDNN + NCCL + torch upper bound in a single blob). Per reviewer feedback, we prefer to pin cuBLAS specifically while keeping the newer NGC base image, so downstream fixes in cuDNN 9.22, NCCL 2.30, and torch 2.13a0 are retained. Only docker/common/install_tensorrt.sh changes: CUBLAS_VER="13.4.1.2-1" -> "13.4.0.1-1" The header/lib symlink guards added in NVIDIA#15087 (CUBLAS_HDR_DIR / CUBLAS_LIB_DIR checks) are already conditional (if [ -d ... ]) so they no-op cleanly when the older cuBLAS is installed. No other code paths change. Signed-off-by: Chenfei Zhang <chenfeiz@nvidia.com>
3 participants
Summary by CodeRabbit
Chores
Bug Fixes
Features
Description
Test Coverage
PR Checklist
Please review the following before submitting your PR:
PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either
api-compatibleorapi-breaking. Forapi-breaking, includeBREAKINGin the PR title.Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.
GitHub Bot Help
To see a list of available CI bot commands, please comment
/bot help.