Skip to content

[None][infra] Waive 4 failed cases for main in post-merge - #13984

Merged
xinhe-nv merged 10 commits into
NVIDIA:mainfrom
xinhe-nv:trtllm-ci-report/waive-20260511-085554
May 12, 2026
Merged

[None][infra] Waive 4 failed cases for main in post-merge#13984
xinhe-nv merged 10 commits into
NVIDIA:mainfrom
xinhe-nv:trtllm-ci-report/waive-20260511-085554

Conversation

@xinhe-nv

@xinhe-nv xinhe-nv commented May 11, 2026

Copy link
Copy Markdown
Collaborator

Auto-generated Waive PR

Created by: TensorRT LLM CI Report (requested by qa@nvidia.com)
Target branch: main
Bug(s): 6094068, 6162121, 6162128, 6162328, 6162564, 6162618, 6163030, 6163032, 6163033, 6163047

Waive entries added

accuracy/test_llm_api_pytorch.py::TestQwen3NextInstruct::test_nvfp4[tp4ep4_adp_on-trtllm] SKIP (https://nvbugs/6094068)
accuracy/test_llm_api_pytorch.py::TestNemotronV3Super::test_auto_dtype_4gpus[4-1-True-True-False] SKIP (https://nvbugs/6163032)
accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_fp8_block_scales_4gpus[tp4-mtp_nextn=0-fp8kv=False-attention_dp=False-cuda_graph=True-overlap_scheduler=False-torch_compile=False-sampler_async_worker=True] SKIP (https://nvbugs/6163030)
accuracy/test_llm_api_pytorch.py::TestMistralLarge3_675B::test_fp8[latency_moe_deepgemm] SKIP (https://nvbugs/6163033)

Already waived (skipped)

  • accuracy/test_llm_api_autodeploy.py::TestModelRegistryAccuracy::test_autodeploy_from_registry[deepseek-ai_DeepSeek-R1-0528-True]
  • accuracy/test_llm_api_pytorch.py::TestMistralLarge3_675B::test_nvfp4_4gpus[latency_moe_trtllm_eagle]
  • accuracy/test_llm_api_pytorch.py::TestMistralLarge3_675B::test_nvfp4_4gpus[latency_moe_trtllm]
  • accuracy/test_llm_api_pytorch.py::TestMiniMaxM2::test_4gpus[attention_dp=False-cuda_graph=True-overlap_scheduler=True-tp_size=4-ep_size=4]
  • accuracy/test_llm_api_pytorch.py::TestKimiK2::test_nvfp4[8gpus]

This PR was auto-generated by TensorRT LLM CI Report. Please review the waive entries before merging.

Summary by CodeRabbit

  • Tests
    • Updated test automation configuration to refine which integration tests are excluded from continuous runs. Changes include new exclusions for specific GPU compute configurations and quantization scenarios while removing obsolete waiver entries, enhancing overall test pipeline efficiency and optimizing resource allocation across automated validation workflows.

Review Change Stack

Bug(s): 6094068, 6162121, 6162128, 6162328, 6162564, 6162618, 6163030, 6163032, 6163033, 6163047
Requested by: qa@nvidia.com

Signed-off-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com>
@xinhe-nv

Copy link
Copy Markdown
Collaborator Author

/bot run --skip-test

@coderabbitai

coderabbitai Bot commented May 11, 2026

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: bfc2eba1-6197-4e41-8963-da678c9fbb4c

📥 Commits

Reviewing files that changed from the base of the PR and between 9547230 and d951873.

📒 Files selected for processing (1)
  • tests/integration/test_lists/waives.txt

📝 Walkthrough

Walkthrough

The waives.txt integration test waiver file is updated with four new entries. One entry skips a TestDeepSeekV3Lite::test_fp8_block_scales_4gpus variant with async worker enabled. A second change replaces multiple prior waivers with updated entries for TestMistralLarge3_675B (fp8/nvfp4 latency-moe), TestNemotronV3Super (auto-dtype), and TestQwen3NextInstruct (nvfp4/trtllm) configurations.

Changes

Test Waiver Configuration Updates

Layer / File(s) Summary
Waived Test Entries
tests/integration/test_lists/waives.txt
Added waiver for TestDeepSeekV3Lite::test_fp8_block_scales_4gpus[tp4-mtp_nextn=0, cuda_graph=True, sampler_async_worker=True] (nvbugs/6163030). Replaced waivers for TestMistralLarge3_675B latency-moe configurations (fp8 and nvfp4 variants), added TestNemotronV3Super auto-dtype waiver, and added TestQwen3NextInstruct nvfp4/trtllm waiver while removing prior TestNemotronNas and TestQwen3NextInstruct variants.

🎯 1 (Trivial) | ⏱️ ~3 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description is largely incomplete compared to the template. It lacks sections for 'Description' explaining the issue/solution, 'Test Coverage' listing relevant tests, and the PR Checklist is not filled out. Add 'Description' section explaining why these tests are being waived, complete 'Test Coverage' section, and mark the PR Checklist items as reviewed/checked.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and specifically identifies the purpose: waiving 4 failed test cases for the main branch in post-merge, matching the actual changes in the waives.txt file.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #47710 [ run ] triggered by Bot. Commit: d951873 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #47710 [ run ] completed with state SUCCESS. Commit: d951873
/LLM/main/L0_MergeRequest_PR pipeline #37606 (Partly Tested) completed with status: 'SUCCESS'

CI Report

Link to invocation

@xinhe-nv

Copy link
Copy Markdown
Collaborator Author

/bot reuse-pipeline

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #47853 [ reuse-pipeline ] triggered by Bot. Commit: 9b4d6d7 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #47853 [ reuse-pipeline ] completed with state SUCCESS. Commit: 9b4d6d7
Reusing PR_Github #47710 (Partly Tested) for commit 9b4d6d7

Link to invocation

xinhe-nv added 3 commits May 12, 2026 11:38
Signed-off-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com>
Signed-off-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com>
@xinhe-nv

Copy link
Copy Markdown
Collaborator Author

/bot reuse-pipeline

Signed-off-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com>
@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #47870 [ reuse-pipeline ] triggered by Bot. Commit: aa4ee84 Link to invocation

@xinhe-nv

Copy link
Copy Markdown
Collaborator Author

/bot reuse-pipeline

@xinhe-nv

Copy link
Copy Markdown
Collaborator Author

/bot reuse-pipeline

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #47875 [ reuse-pipeline ] triggered by Bot. Commit: f15b314 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #47870 [ reuse-pipeline ] completed with state ABORTED. Commit: aa4ee84
Can't reuse PR_Github #47710 (Partly Tested) with status: SUCCESS

Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #47880 [ reuse-pipeline ] triggered by Bot. Commit: f15b314 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #47875 [ reuse-pipeline ] completed with state ABORTED. Commit: f15b314
Can't reuse PR_Github #0 with status: UNKNOWN

Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #47880 [ reuse-pipeline ] completed with state SUCCESS. Commit: f15b314
Reusing PR_Github #47710 (Partly Tested) for commit f15b314

Link to invocation

@xinhe-nv
xinhe-nv merged commit 5e7f231 into NVIDIA:main May 12, 2026
6 checks passed
@xinhe-nv
xinhe-nv deleted the trtllm-ci-report/waive-20260511-085554 branch May 12, 2026 05:17
yufeiwu-nv pushed a commit to yufeiwu-nv/TensorRT-LLM that referenced this pull request May 19, 2026
Signed-off-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants