Skip to content

[None][test] Waive 11 failed cases for main in QA CI - #15506

Merged
xinhe-nv merged 6 commits into
NVIDIA:mainfrom
tensorrt-cicd:trtllm-ci-report/waive-20260620-192222
Jun 23, 2026
Merged

[None][test] Waive 11 failed cases for main in QA CI#15506
xinhe-nv merged 6 commits into
NVIDIA:mainfrom
tensorrt-cicd:trtllm-ci-report/waive-20260620-192222

Conversation

@tensorrt-cicd

@tensorrt-cicd tensorrt-cicd commented Jun 20, 2026

Copy link
Copy Markdown
Collaborator

Auto-generated Waive PR

Created by: TensorRT LLM CI Report (requested by qa@nvidia.com)
Target branch: main
Jenkins build: https://prod.blsm.nvidia.com/swqa-tensorrt-qa-test/job/LLM_FUNCTION_CLUSTER_TEST/1555/
Bug(s): 6316980, 6316981, 6317600, 6329165, 6331421, 6344883, 6344884

Waive entries added

full:GB300/accuracy/test_llm_api_pytorch.py::TestGPTOSS::test_w4_4gpus[v2_kv_cache_no_reuse-tp4-trtllm-fp8] SKIP (https://nvbugs/6316980)
full:GB300/accuracy/test_llm_api_autodeploy.py::TestNemotronSuperV3::test_mtp[nvfp4_ws4_80gb-trtllm] SKIP (https://nvbugs/6316981)
full:GB300/accuracy/test_llm_api_pytorch.py::TestQwen3_5_35B_A3B::test_fp8_moe_dflash SKIP (https://nvbugs/6317600)
full:GB300/accuracy/test_llm_api_pytorch.py::TestQwen3_5_4B::test_dflash SKIP (https://nvbugs/6344883)
full:GB300/disaggregated/test_auto_scaling.py::test_worker_restart[http-round_robin] SKIP (https://nvbugs/6344884)
full:GB300/accuracy/test_llm_api_pytorch.py::TestGPTOSS::test_w4_4gpus[v2_kv_cache-ep4-trtllm-fp8] SKIP (https://nvbugs/6316980)
full:GB300/accuracy/test_llm_api_pytorch.py::TestQwen3_5_397B_A17B::test_nvfp4[tep4_trtllm] SKIP (https://nvbugs/6317600)
full:GB300/accuracy/test_llm_api_pytorch_multimodal.py::TestQwen2_5_VL_7B::test_auto_dtype SKIP (https://nvbugs/6331421)
full:GB300/accuracy/test_llm_api_autodeploy.py::TestNemotronSuperV3::test_mtp[bf16_ws4_180gb-trtllm] SKIP (https://nvbugs/6316981)
full:GB300/accuracy/test_llm_api_autodeploy.py::TestNemotronNanoV3::test_accuracy[nvfp4-1-attn_dp_off-trtllm] SKIP (https://nvbugs/6329165)
full:GB300/accuracy/test_llm_api_autodeploy.py::TestNemotronSuperV3::test_mtp[fp8_ws4_80gb-trtllm] SKIP (https://nvbugs/6316981)

This PR was auto-generated by TensorRT LLM CI Report. Please review the waive entries before merging.

Summary by CodeRabbit

Release Notes

This release contains internal testing infrastructure updates only. No user-facing features, bug fixes, or changes have been introduced.

@xinhe-nv
xinhe-nv marked this pull request as ready for review June 23, 2026 03:43
@xinhe-nv
xinhe-nv enabled auto-merge (squash) June 23, 2026 03:43
@coderabbitai

coderabbitai Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

Adds 11 new SKIP waiver lines to tests/integration/test_lists/waives.txt in the GB300 section (lines 209–226) for failing tests across test_llm_api_autodeploy.py, test_llm_api_pytorch.py, test_llm_api_pytorch_multimodal.py, and disaggregated/test_auto_scaling.py, with at least one updated nvbugs reference.

Changes

GB300 Waiver Additions

Layer / File(s) Summary
GB300 waiver entries
tests/integration/test_lists/waives.txt
Adds SKIP waivers for TestNemotronSuperV3::test_mtp (bf16/fp8/nvfp4 trtllm configs), TestGPTOSS::test_w4_4gpus (v2_kv_cache ep4/tp4 fp8), TestQwen3_5_35B_A3B::test_fp8_moe_dflash (updated nvbugs ref), TestQwen3_5_397B_A17B::test_nvfp4[tep4_trtllm], TestQwen3_5_4B::test_dflash, TestQwen2_5_VL_7B::test_auto_dtype (updated nvbugs ref), and test_worker_restart[http-round_robin].

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Possibly related PRs

  • NVIDIA/TensorRT-LLM#15337: Modifies the same waives.txt file and adds overlapping TestGPTOSS::test_w4_4gpus waivers with similar kv-cache and parallelism parameter variants.
  • NVIDIA/TensorRT-LLM#15341: Modifies the same waives.txt to add/update the waiver for TestQwen2_5_VL_7B::test_auto_dtype with nvbugs reference updates.
  • NVIDIA/TensorRT-LLM#15342: Modifies the same waives.txt to add/adjust SKIP waivers for TestQwen3_5_35B_A3B::test_fp8_moe_dflash and TestQwen2_5_VL_7B::test_auto_dtype.

Suggested reviewers

  • crazydemo
  • jieli-matrix
  • StanleySun639
  • LarryXFly
  • xinhe-nv
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and specifically describes the main change: waiving 11 failed test cases for the main branch in QA CI.
Description check ✅ Passed The PR description is auto-generated and provides context, bug references, and a clear list of waived entries, though it deviates from the template structure.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/integration/test_lists/waives.txt`:
- Around line 209-226: Remove all duplicate waive entries in the waives.txt file
where the same test nodeid and nvbug reference appear multiple times.
Specifically, remove one instance of each duplicate pair:
TestNemotronNanoV3::test_accuracy[nvfp4-1-attn_dp_off-trtllm],
TestNemotronSuperV3::test_mtp[bf16_ws4_180gb-trtllm],
TestNemotronSuperV3::test_mtp[fp8_ws4_80gb-trtllm],
TestNemotronSuperV3::test_mtp[nvfp4_ws4_80gb-trtllm],
TestGPTOSS::test_w4_4gpus[v2_kv_cache-ep4-trtllm-fp8], and
TestGPTOSS::test_w4_4gpus[v2_kv_cache_no_reuse-tp4-trtllm-fp8]. Additionally,
consolidate entries that have the same test nodeid but different nvbug
references by keeping only one entry with the most recent or correct nvbug
reference. For TestQwen3_5_35B_A3B::test_fp8_moe_dflash and
TestQwen2_5_VL_7B::test_auto_dtype, determine which nvbug reference is correct
and remove the redundant entry. Ensure the file maintains one entry per unique
test nodeid to pass the waive list check.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 278b275a-95e2-48d8-baa0-c2d4666da041

📥 Commits

Reviewing files that changed from the base of the PR and between 3448424 and 45c069e.

📒 Files selected for processing (1)
  • tests/integration/test_lists/waives.txt

Comment thread tests/integration/test_lists/waives.txt Outdated
@xinhe-nv
xinhe-nv force-pushed the trtllm-ci-report/waive-20260620-192222 branch from 45c069e to 335ebe5 Compare June 23, 2026 03:47
@xinhe-nv
xinhe-nv force-pushed the trtllm-ci-report/waive-20260620-192222 branch from 335ebe5 to e31cd9c Compare June 23, 2026 03:49
tensorrt-cicd and others added 5 commits June 23, 2026 03:50
Signed-off-by: GitLab CI Bot <gitlab-ci@nvidia.com>
Signed-off-by: GitLab CI Bot <gitlab-ci@nvidia.com>
Bug(s): 6316980, 6316981, 6317600, 6329165, 6331421, 6344883, 6344884
Requested by: qa@nvidia.com
Jenkins build: https://prod.blsm.nvidia.com/swqa-tensorrt-qa-test/job/LLM_FUNCTION_CLUSTER_TEST/1555/

Signed-off-by: tensorrt-cicd <90828364+tensorrt-cicd@users.noreply.github.com>
Signed-off-by: GitLab CI Bot <gitlab-ci@nvidia.com>
Signed-off-by: GitLab CI Bot <gitlab-ci@nvidia.com>
GitLab CI Bot
fix
Signed-off-by: GitLab CI Bot <gitlab-ci@nvidia.com>
@xinhe-nv
xinhe-nv force-pushed the trtllm-ci-report/waive-20260620-192222 branch from 84a070f to e019f69 Compare June 23, 2026 03:50
@xinhe-nv

Copy link
Copy Markdown
Collaborator

/bot skip --comment "Only waive tests"

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

PR_Github #55145 [ skip ] triggered by Bot. Commit: 90630de Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

PR_Github #55145 [ skip ] completed with state SUCCESS. Commit: 90630de
Skipping testing for commit 90630de

Link to invocation

@xinhe-nv
xinhe-nv merged commit 2ce56a3 into NVIDIA:main Jun 23, 2026
7 checks passed
@xinhe-nv
xinhe-nv deleted the trtllm-ci-report/waive-20260620-192222 branch June 23, 2026 04:18
xinhe-nv added a commit to tensorrt-cicd/TensorRT-LLM that referenced this pull request Jun 24, 2026
Signed-off-by: tensorrt-cicd <90828364+tensorrt-cicd@users.noreply.github.com>
Co-authored-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com>
Signed-off-by: GitLab CI Bot <gitlab-ci@nvidia.com>
BrianLi23 pushed a commit to BrianLi23/TensorRT-LLM that referenced this pull request Jul 9, 2026
Signed-off-by: tensorrt-cicd <90828364+tensorrt-cicd@users.noreply.github.com>
Co-authored-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants