[https://nvbugs/6287721][chore] Unwaive a test for update weights by shikicloud · Pull Request #16102 · NVIDIA/TensorRT-LLM

shikicloud · 2026-07-08T05:17:33Z

Summary by CodeRabbit

Chores
- Updated integration test waivers to account for additional test cases in GPU-related update-weight workflows.
- Adjusted existing waiver entries as part of the same test list update.

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

shikicloud · 2026-07-08T05:18:16Z

/bot run --stage-list "DGX_B200-4_GPUs-PyTorch-Ray-1"

coderabbitai · 2026-07-08T05:19:36Z

📝 Walkthrough

Walkthrough

Two new SKIP/waive entries were added to the integration test waiver list for update_weights-related unit tests, and an existing multi-gpu nvfp4 waiver entry was adjusted in place.

Changes

Waiver List Update

Layer / File(s)	Summary
Add update_weights test waivers `tests/integration/test_lists/waives.txt`	Added a waiver for `test_llm_update_weights_fp8` on multi-gpu Qwen3-8B referencing nvbugs/6287721, and added a single-gpu waiver for `test_llm_update_weights_with_quant_config[fp8-Qwen3-30B-A3B]` referencing nvbugs/6280570 while adjusting the surrounding existing nvfp4 waiver entry.

Estimated code review effort: 1 (Trivial) | ~2 minutes

Possibly related PRs

NVIDIA/TensorRT-LLM#16093: Both PRs modify waives.txt to add waive SKIP entries for failing update_weights CI cases, including ray orchestrator tests.
NVIDIA/TensorRT-LLM#16085: Both PRs modify the same waiver list file with SKIP/waive entries for specific unittest failures.
NVIDIA/TensorRT-LLM#15585: Both PRs update SKIP/waive entries in waives.txt for specific unit tests to control CI skipping.

Suggested reviewers: dongxuy04, pcastonguay, mzweilz

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description is just the template and lacks the required issue, solution, and test coverage details.	Add a short Description of the problem and fix, list relevant Test Coverage, and complete the checklist with actual PR-specific details.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title matches the change by naming the NVBugs ticket and unwaiving an update-weights test.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands.}

tensorrt-cicd · 2026-07-08T05:24:02Z

PR_Github #58151 [ run ] triggered by Bot. Commit: 2e0fedd Link to invocation

Superjomn · 2026-07-08T05:29:45Z

 unittest/_torch/ray_orchestrator/multi_gpu/test_llm_update_weights_multi_gpu.py::test_llm_partial_update_weights_nvfp4[auto-Qwen3/Qwen3-8B] SKIP (https://nvbugs/6372690)
 unittest/_torch/ray_orchestrator/multi_gpu/test_llm_update_weights_multi_gpu.py::test_llm_partial_update_weights_nvfp4[fp8-Qwen3/Qwen3-30B-A3B] SKIP (https://nvbugs/6372690)
 unittest/_torch/ray_orchestrator/multi_gpu/test_llm_update_weights_multi_gpu.py::test_llm_partial_update_weights_nvfp4[fp8-Qwen3/Qwen3-8B] SKIP (https://nvbugs/6372690)
+unittest/_torch/ray_orchestrator/multi_gpu/test_llm_update_weights_multi_gpu.py::test_llm_update_weights_fp8[Qwen3/Qwen3-8B-Qwen3/Qwen3-8B-FP8] SKIP (https://nvbugs/6287721)


If skipping both tests is as expected from a functionality perspective, then you may use pytest.skip in the code. If not, and both should be TODOs, then keep it.

Superjomn

LGTM

Signed-off-by: Shiki Wu <shikiw@oci-hsg-cs-001-vscode-01.cm.cluster>

shikicloud · 2026-07-08T06:15:39Z

/bot run --stage-list "DGX_B200-4_GPUs-PyTorch-Ray-1"

tensorrt-cicd · 2026-07-08T06:21:04Z

PR_Github #58176 [ run ] triggered by Bot. Commit: a31e9e0 Link to invocation

tensorrt-cicd · 2026-07-08T06:24:31Z

PR_Github #58151 [ run ] completed with state ABORTED. Commit: 2e0fedd

Link to invocation

tensorrt-cicd · 2026-07-08T07:51:46Z

PR_Github #58176 [ run ] completed with state FAILURE. Commit: a31e9e0
/LLM/main/L0_MergeRequest_PR pipeline #46826 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

shikicloud · 2026-07-08T14:13:44Z

/bot run --stage-list "DGX_B200-4_GPUs-PyTorch-Ray-1"

Signed-off-by: Shiki Wu <shikiw@oci-hsg-cs-001-vscode-01.cm.cluster>

shikicloud · 2026-07-08T15:01:52Z

/bot run --stage-list "DGX_B200-4_GPUs-PyTorch-Ray-1"

tensorrt-cicd · 2026-07-08T15:08:17Z

PR_Github #58252 [ run ] triggered by Bot. Commit: ad85689 Link to invocation

tensorrt-cicd · 2026-07-08T16:37:42Z

PR_Github #58252 [ run ] completed with state FAILURE. Commit: ad85689
/LLM/main/L0_MergeRequest_PR pipeline #46893 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Signed-off-by: Shiki Wu <shikiw@nvidia.com>

shikicloud · 2026-07-08T16:48:37Z

/bot run --stage-list "DGX_B200-4_GPUs-PyTorch-Ray-1"

tensorrt-cicd · 2026-07-08T16:55:17Z

PR_Github #58267 [ run ] triggered by Bot. Commit: 3f55b11 Link to invocation

tensorrt-cicd · 2026-07-08T18:39:21Z

PR_Github #58267 [ run ] completed with state SUCCESS. Commit: 3f55b11
/LLM/main/L0_MergeRequest_PR pipeline #46906 (Partly Tested) completed with status: 'SUCCESS'

CI Report

Link to invocation

Superjomn · 2026-07-13T07:40:25Z

The unwaived tests have passed: https://prod.blsm.nvidia.com/sw-tensorrt-llm-github-2/blue/organizations/jenkins/LLM%2Fmain%2FL0_Test-x86_64-Multi-GPU/detail/L0_Test-x86_64-Multi-GPU/2273/tests

Superjomn · 2026-07-13T07:40:43Z

/bot skip --comment "the unwaived tests passed"

tensorrt-cicd · 2026-07-13T07:46:37Z

PR_Github #58929 [ skip ] triggered by Bot. Commit: 3f55b11 Link to invocation

tensorrt-cicd · 2026-07-13T08:00:19Z

PR_Github #58929 [ skip ] completed with state SUCCESS. Commit: 3f55b11
Skipping testing for commit 3f55b11

Link to invocation

github-actions Bot assigned shikicloud Jul 8, 2026

shikicloud changed the title ~~[https://nvbugs/6287721][chore] Unwaive update weights test for specific models~~ [https://nvbugs/6287721][chore] Waive update weights test for specific models Jul 8, 2026

Superjomn reviewed Jul 8, 2026

View reviewed changes

Superjomn approved these changes Jul 8, 2026

View reviewed changes

unwaive test

a31e9e0

Signed-off-by: Shiki Wu <shikiw@oci-hsg-cs-001-vscode-01.cm.cluster>

shikicloud force-pushed the waive-nvbugs branch from 2e0fedd to a31e9e0 Compare July 8, 2026 06:13

shikicloud changed the title ~~[https://nvbugs/6287721][chore] Waive update weights test for specific models~~ [https://nvbugs/6287721][chore] Unwaive a test for update weights Jul 8, 2026

rebase

ad85689

Signed-off-by: Shiki Wu <shikiw@oci-hsg-cs-001-vscode-01.cm.cluster>

[https://nvbugs/6287721][test] Unwaive FP8 8B update-weight test

3f55b11

Signed-off-by: Shiki Wu <shikiw@nvidia.com>

Superjomn enabled auto-merge (squash) July 13, 2026 07:41

Uh oh!

Conversation

shikicloud commented Jul 8, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Description

Test Coverage

PR Checklist

GitHub Bot Help

Uh oh!

shikicloud commented Jul 8, 2026

Uh oh!

coderabbitai Bot commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

tensorrt-cicd commented Jul 8, 2026

Uh oh!

Superjomn Jul 8, 2026

Choose a reason for hiding this comment

Uh oh!

Superjomn left a comment

Choose a reason for hiding this comment

Uh oh!

shikicloud commented Jul 8, 2026

Uh oh!

tensorrt-cicd commented Jul 8, 2026

Uh oh!

tensorrt-cicd commented Jul 8, 2026

Uh oh!

tensorrt-cicd commented Jul 8, 2026

Uh oh!

shikicloud commented Jul 8, 2026

Uh oh!

shikicloud commented Jul 8, 2026

Uh oh!

tensorrt-cicd commented Jul 8, 2026

Uh oh!

tensorrt-cicd commented Jul 8, 2026

Uh oh!

shikicloud commented Jul 8, 2026

Uh oh!

tensorrt-cicd commented Jul 8, 2026

Uh oh!

tensorrt-cicd commented Jul 8, 2026

Uh oh!

Superjomn commented Jul 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Superjomn commented Jul 13, 2026

Uh oh!

tensorrt-cicd commented Jul 13, 2026

Uh oh!

tensorrt-cicd commented Jul 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

shikicloud commented Jul 8, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jul 8, 2026 •

edited

Loading

Superjomn commented Jul 13, 2026 •

edited

Loading