Skip to content

[None][test] Waive 3 failed cases for main in QA CI - #15319

Merged
jieli-matrix merged 2 commits into
NVIDIA:mainfrom
tensorrt-cicd:trtllm-ci-report/waive-20260612-190633
Jun 20, 2026
Merged

[None][test] Waive 3 failed cases for main in QA CI#15319
jieli-matrix merged 2 commits into
NVIDIA:mainfrom
tensorrt-cicd:trtllm-ci-report/waive-20260612-190633

Conversation

@tensorrt-cicd

@tensorrt-cicd tensorrt-cicd commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator

Auto-generated Waive PR

Created by: TensorRT LLM CI Report (requested by qa@nvidia.com)
Target branch: main
Jenkins build: https://prod.blsm.nvidia.com/swqa-tensorrt-qa-test/job/LLM_FUNCTION_TEST/2242/
Bug(s): 6313060, 6313072, 6313076

Waive entries added

full:RTX_PRO_6000_Blackwell_Server_Edition/accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_bfloat16[mtp_nextn=0-attention_dp=False-cuda_graph=False-overlap_scheduler=False-torch_compile=False-enable_chunked_prefill=False-v2_kv_cache=True] SKIP (https://nvbugs/6313060)
full:RTX_PRO_6000_Blackwell_Server_Edition/accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_bfloat16[mtp_nextn=0-attention_dp=False-cuda_graph=False-overlap_scheduler=False-torch_compile=True-enable_chunked_prefill=False-v2_kv_cache=False] SKIP (https://nvbugs/6313072)
full:RTX_PRO_6000_Blackwell_Server_Edition/accuracy/test_llm_api_pytorch.py::TestQwen3_5_35B_A3B::test_fp8[enable_block_reuse=False] SKIP (https://nvbugs/6313076)

This PR was auto-generated by TensorRT LLM CI Report. Please review the waive entries before merging.

Summary by CodeRabbit

  • Tests
    • Added test waivers for specific model variants (DeepSeekV3Lite and Qwen 3.5 35B) on Blackwell RTX_PRO_6000 hardware with various optimization settings to handle known test conditions.

@jieli-matrix

Copy link
Copy Markdown
Collaborator

close as triage with a new PR

@jieli-matrix jieli-matrix reopened this Jun 13, 2026
@jieli-matrix
jieli-matrix marked this pull request as ready for review June 13, 2026 09:35
@jieli-matrix
jieli-matrix force-pushed the trtllm-ci-report/waive-20260612-190633 branch from 4c71ece to 2b56fb5 Compare June 13, 2026 09:35
@jieli-matrix jieli-matrix self-assigned this Jun 13, 2026
@jieli-matrix

Copy link
Copy Markdown
Collaborator

/bot run --stage-list ""

@coderabbitai

coderabbitai Bot commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

Added three test waiver entries to skip integration tests on RTX_PRO_6000 Blackwell hardware: two DeepSeekV3Lite bfloat16 configurations with different parameter combinations and one Qwen3_5_35B_A3B fp8 configuration, each linked to specific NVBUG references.

Changes

RTX_PRO_6000 Blackwell Test Waivers

Layer / File(s) Summary
RTX_PRO_6000 Blackwell test waivers
tests/integration/test_lists/waives.txt
Added two DeepSeekV3Lite bfloat16 SKIP waivers with mtp_nextn=0 and differing v2_kv_cache/torch_compile configurations, and one Qwen3_5_35B_A3B fp8 waiver with enable_block_reuse=False, all for RTX_PRO_6000 Blackwell tied to NVBUG references.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~2 minutes

Possibly related PRs

Suggested reviewers

  • jieli-matrix
  • xinhe-nv
  • LarryXFly
  • StanleySun639
  • crazydemo
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name Status Explanation Resolution
Description check ❓ Inconclusive The description provides clear context about the auto-generated PR, including Jenkins build reference and three bug references, but omits required sections from the template. While the auto-generated format is acceptable, consider adding a brief 'Description' section explaining the waiver rationale and a 'Test Coverage' section noting which tests are waived.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and specifically summarizes the main change: waiving 3 failed test cases for the main branch in QA CI.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/integration/test_lists/waives.txt`:
- Around line 224-225: The waiver's bug ID is incorrect: update the first
DeepSeek waiver reference from 6313072 to 6313060 so it matches the PR
assignment; specifically edit the entry for
TestDeepSeekV3Lite::test_bfloat16[mtp_nextn=0-attention_dp=False-cuda_graph=False-overlap_scheduler=False-torch_compile=False-enable_chunked_prefill=False-v2_kv_cache=True]
to use 6313060 instead of 6313072.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 6b679010-da51-4d03-8dce-892439586087

📥 Commits

Reviewing files that changed from the base of the PR and between bb32597 and 2b56fb5.

📒 Files selected for processing (1)
  • tests/integration/test_lists/waives.txt

Comment thread tests/integration/test_lists/waives.txt
@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

PR_Github #54037 [ run ] triggered by Bot. Commit: 2b56fb5 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

PR_Github #54037 [ run ] completed with state SUCCESS. Commit: 2b56fb5
/LLM/main/L0_MergeRequest_PR pipeline #43121 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@jieli-matrix
jieli-matrix enabled auto-merge (squash) June 15, 2026 06:58
@jieli-matrix
jieli-matrix force-pushed the trtllm-ci-report/waive-20260612-190633 branch from 2b56fb5 to 8ab7019 Compare June 15, 2026 06:58
@jieli-matrix

Copy link
Copy Markdown
Collaborator

/bot run --stage-list ""

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

PR_Github #54247 [ run ] triggered by Bot. Commit: 8ab7019 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

PR_Github #54247 [ run ] completed with state SUCCESS. Commit: 8ab7019
/LLM/main/L0_MergeRequest_PR pipeline #43321 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@jieli-matrix
jieli-matrix force-pushed the trtllm-ci-report/waive-20260612-190633 branch from 8ab7019 to 44b0b39 Compare June 16, 2026 01:48
@jieli-matrix

Copy link
Copy Markdown
Collaborator

/bot run --stage-list ""

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

PR_Github #54409 [ run ] triggered by Bot. Commit: 44b0b39 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

PR_Github #54409 [ run ] completed with state FAILURE. Commit: 44b0b39
/LLM/main/L0_MergeRequest_PR pipeline #43476 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@jieli-matrix
jieli-matrix force-pushed the trtllm-ci-report/waive-20260612-190633 branch from 44b0b39 to 0a37901 Compare June 16, 2026 06:34
@jieli-matrix

Copy link
Copy Markdown
Collaborator

/bot run --stage-list ""

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

PR_Github #54513 [ run ] triggered by Bot. Commit: 0a37901 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

PR_Github #54513 [ run ] completed with state SUCCESS. Commit: 0a37901
/LLM/main/L0_MergeRequest_PR pipeline #43571 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@jieli-matrix

Copy link
Copy Markdown
Collaborator

/bot run --stage-list ""

@jieli-matrix
jieli-matrix force-pushed the trtllm-ci-report/waive-20260612-190633 branch from 0a37901 to 9115358 Compare June 17, 2026 02:25
@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

PR_Github #54716 [ run ] triggered by Bot. Commit: 9115358 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

PR_Github #54716 [ run ] completed with state FAILURE. Commit: 9115358
/LLM/main/L0_MergeRequest_PR pipeline #43742 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@jieli-matrix
jieli-matrix force-pushed the trtllm-ci-report/waive-20260612-190633 branch from 9115358 to 475bbe4 Compare June 18, 2026 06:58
@jieli-matrix

Copy link
Copy Markdown
Collaborator

/bot run --stage-list ""

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

PR_Github #54911 [ run ] triggered by Bot. Commit: 475bbe4 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

PR_Github #54911 [ run ] completed with state SUCCESS. Commit: 475bbe4
/LLM/main/L0_MergeRequest_PR pipeline #43914 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

@jieli-matrix
jieli-matrix force-pushed the trtllm-ci-report/waive-20260612-190633 branch from 475bbe4 to c80e0b1 Compare June 19, 2026 06:21
@jieli-matrix

Copy link
Copy Markdown
Collaborator

/bot run --stage-list ""

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

PR_Github #54962 [ run ] triggered by Bot. Commit: c80e0b1 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

PR_Github #54962 [ run ] completed with state SUCCESS. Commit: c80e0b1
/LLM/main/L0_MergeRequest_PR pipeline #43962 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

@jieli-matrix
jieli-matrix force-pushed the trtllm-ci-report/waive-20260612-190633 branch 2 times, most recently from 8c7d189 to f217afc Compare June 20, 2026 04:34
tensorrt-cicd and others added 2 commits June 20, 2026 12:48
Bug(s): 6313060, 6313072, 6313076
Requested by: qa@nvidia.com
Jenkins build: https://prod.blsm.nvidia.com/swqa-tensorrt-qa-test/job/LLM_FUNCTION_TEST/2242/

Signed-off-by: tensorrt-cicd <90828364+tensorrt-cicd@users.noreply.github.com>
Signed-off-by: Jie Li <76780849+jieli-matrix@users.noreply.github.com>
@jieli-matrix
jieli-matrix force-pushed the trtllm-ci-report/waive-20260612-190633 branch from f217afc to de910d4 Compare June 20, 2026 04:48
@jieli-matrix

Copy link
Copy Markdown
Collaborator

/bot skip --comment "Only waive tests"

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

PR_Github #54978 [ skip ] triggered by Bot. Commit: de910d4 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

PR_Github #54978 [ skip ] completed with state SUCCESS. Commit: de910d4
Skipping testing for commit de910d4

Link to invocation

@jieli-matrix
jieli-matrix merged commit 53b392e into NVIDIA:main Jun 20, 2026
7 checks passed
xinhe-nv pushed a commit to tensorrt-cicd/TensorRT-LLM that referenced this pull request Jun 23, 2026
Signed-off-by: tensorrt-cicd <90828364+tensorrt-cicd@users.noreply.github.com>
Signed-off-by: Jie Li <76780849+jieli-matrix@users.noreply.github.com>
Co-authored-by: Jie Li <76780849+jieli-matrix@users.noreply.github.com>
Signed-off-by: GitLab CI Bot <gitlab-ci@nvidia.com>
xinhe-nv pushed a commit to tensorrt-cicd/TensorRT-LLM that referenced this pull request Jun 24, 2026
Signed-off-by: tensorrt-cicd <90828364+tensorrt-cicd@users.noreply.github.com>
Signed-off-by: Jie Li <76780849+jieli-matrix@users.noreply.github.com>
Co-authored-by: Jie Li <76780849+jieli-matrix@users.noreply.github.com>
Signed-off-by: GitLab CI Bot <gitlab-ci@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants