Skip to content

[None][infra] Waive 11 failed cases for main in post-merge 2760 - #15003

Merged
EmmaQiaoCh merged 2 commits into
NVIDIA:mainfrom
ZhanruiSunCh:trtllm-ci-report/waive-20260605-101150
Jun 5, 2026
Merged

[None][infra] Waive 11 failed cases for main in post-merge 2760#15003
EmmaQiaoCh merged 2 commits into
NVIDIA:mainfrom
ZhanruiSunCh:trtllm-ci-report/waive-20260605-101150

Conversation

@ZhanruiSunCh

@ZhanruiSunCh ZhanruiSunCh commented Jun 5, 2026

Copy link
Copy Markdown
Collaborator

Auto-generated Waive PR

Created by: TensorRT LLM CI Report (requested by @EmmaQiaoCh)
Target branch: main
Bug(s): 6272556, 6272573, 6272598, 6272608, 6272644, 6272653, 6272656, 6272668, 6272673

Waive entries added

accuracy/test_llm_api_autodeploy.py::TestModelRegistryAccuracy::test_autodeploy_from_registry[deepseek-ai_DeepSeek-R1-0528-True] SKIP (https://nvbugs/6272656)
accuracy/test_llm_api_autodeploy.py::TestQwen3_5_397B_MoE::test_nvfp4[8] SKIP (https://nvbugs/6272668)
accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_nvfp4_4gpus[moe_backend=CUTLASS-mtp_nextn=0-tp4-fp8kv=True-attention_dp=True-cuda_graph=True-overlap_scheduler=True-low_precision_combine=False-torch_compile=True] SKIP (https://nvbugs/6272673)
examples/test_visual_gen_multi_gpu.py::test_wan22_t2v_lpips_against_golden_multi_gpu[attn2d_2x2] SKIP (https://nvbugs/6272644)
examples/test_visual_gen_multi_gpu.py::test_wan22_t2v_lpips_against_golden_multi_gpu[cfg2_ulysses2] SKIP (https://nvbugs/6272644)
examples/test_visual_gen_multi_gpu.py::test_wan22_t2v_lpips_against_golden_multi_gpu[ulysses2_ring2] SKIP (https://nvbugs/6272644)
examples/test_visual_gen_multi_gpu.py::test_wan22_t2v_lpips_against_golden_multi_gpu[ulysses4] SKIP (https://nvbugs/6272644)
unittest/_torch/multimodal/test_mm_encoder_standalone.py::test_single_request_chat_multiple_images[pd_disagg-qwen3_30b_a3b_fp8] SKIP (https://nvbugs/6272573)
verl/test_verl_cases.py::test_trtllm_abort SKIP (https://nvbugs/6272653)

Already waived (skipped)

  • unittest/_torch/modeling/test_modeling_step3p7.py::TestStep3p7Checkpoint::test_config_and_weight_accounting_1_nvfp4

This PR was auto-generated by TensorRT LLM CI Report. Please review the waive entries before merging.

Summary by CodeRabbit

  • Chores
    • Updated test waiver entries across multiple test suites to accommodate additional test cases, including autodeploy, PyTorch, visual generation, and other test scenarios. Added 11 new waiver entries to improve test infrastructure management.

Bug(s): 6272556, 6272573, 6272598, 6272608, 6272644, 6272653, 6272656, 6272668, 6272673
Requested by: @EmmaQiaoCh

Signed-off-by: ZhanruiSunCh <184402041+ZhanruiSunCh@users.noreply.github.com>
@coderabbitai

coderabbitai Bot commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

Updated tests/integration/test_lists/waives.txt by adding eleven new skip/waiver entries for failing test cases across six distinct test suites and modules (accuracy/autodeploy, accuracy/pytorch, examples/visual_gen_multi_gpu, executor, torch/visual_gen, and verl).

Changes

Test Waiver Additions

Layer / File(s) Summary
Waiver entries for failing test cases across multiple suites
tests/integration/test_lists/waives.txt
Added skip/waiver entries for failing test cases: autodeploy DeepSeek R1 and Qwen3_5 MoE nvfp4; pytorch TestDeepSeekV3Lite nvfp4_4gpus with torch_compile; four visual_gen_multi_gpu wan22_t2v_lpips parameterizations; executor test; torch multimodal encoder and THOP MoE autotune cases; visual_gen wan22_t2v_pipeline; and verl trtllm_abort.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Possibly related PRs

  • NVIDIA/TensorRT-LLM#14854: Both PRs update the same tests/integration/test_lists/waives.txt by adding/changing SKIP entries for overlapping failing test case parameterizations—especially TestDeepSeekV3Lite::test_nvfp4_4gpus.
  • NVIDIA/TensorRT-LLM#14896: Both PRs update tests/integration/test_lists/waives.txt by adding SKIP/waiver entries for failing cases in accuracy/test_llm_api_pytorch.py with different specific parametrizations.
  • NVIDIA/TensorRT-LLM#14802: Both PRs update tests/integration/test_lists/waives.txt with overlapping SKIP/waiver additions for the same failing integration tests in accuracy/test_llm_api_autodeploy.py and accuracy/test_llm_api_pytorch.py.

Suggested reviewers

  • chzblych
  • EmmaQiaoCh
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The PR description is incomplete and does not fully follow the required template. Critical sections like 'Description', 'Test Coverage', and PR Checklist are missing or largely unfilled. Add a proper 'Description' section explaining the waiver entries and their purpose, include 'Test Coverage' section, and complete the PR Checklist items. Ensure the PR title follows the format: [ticket/bug][type] Summary.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The PR title clearly and specifically describes the main change: waiving 11 failed test cases with a reference to the post-merge context.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/integration/test_lists/waives.txt`:
- Line 330: The waiver entry uses a directory path "unittest/_torch/executor"
which won't match collected tests because apply_waives_ut only supports exact
file/function/nodeid matches; update the waiver list to reference explicit test
file paths or nodeids under that directory (for example
"unittest/_torch/executor/<test_file>.py" or specific nodeids) so
apply_waives_ut will correctly skip the intended tests, and verify against the
parser logic in tests/unittest/conftest.py::apply_waives_ut when choosing names.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 01412b83-c8e5-46cf-be2e-59e48bd9a738

📥 Commits

Reviewing files that changed from the base of the PR and between fdcdcb3 and de8f6b2.

📒 Files selected for processing (1)
  • tests/integration/test_lists/waives.txt

Comment thread tests/integration/test_lists/waives.txt Outdated
Signed-off-by: Emma Qiao <qqiao@nvidia.com>
@EmmaQiaoCh

Copy link
Copy Markdown
Collaborator

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #52328 [ run ] triggered by Bot. Commit: 6951b07 Link to invocation

@EmmaQiaoCh
EmmaQiaoCh enabled auto-merge (squash) June 5, 2026 10:29
@EmmaQiaoCh

Copy link
Copy Markdown
Collaborator

/bot skip --comment "Previous CI passed check test list"

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #52339 [ skip ] triggered by Bot. Commit: 6951b07 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #52328 [ run ] completed with state ABORTED. Commit: 6951b07

Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #52339 [ skip ] completed with state SUCCESS. Commit: 6951b07
Skipping testing for commit 6951b07

Link to invocation

@EmmaQiaoCh
EmmaQiaoCh merged commit 2336e47 into NVIDIA:main Jun 5, 2026
11 checks passed
fbxai pushed a commit to fbxai/TensorRT-LLM that referenced this pull request Jun 5, 2026
…IA#15003)

Signed-off-by: ZhanruiSunCh <184402041+ZhanruiSunCh@users.noreply.github.com>
Signed-off-by: Emma Qiao <qqiao@nvidia.com>
Co-authored-by: Emma Qiao <qqiao@nvidia.com>
Signed-off-by: NVFB <186336021+NVFB@users.noreply.github.com>
2ez4bz pushed a commit to 2ez4bz/TensorRT-LLM that referenced this pull request Jun 8, 2026
…IA#15003)

Signed-off-by: ZhanruiSunCh <184402041+ZhanruiSunCh@users.noreply.github.com>
Signed-off-by: Emma Qiao <qqiao@nvidia.com>
Co-authored-by: Emma Qiao <qqiao@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants