[None][infra] Waive 12 failed cases for main in post-merge 2749

[ZhanruiSunCh]

Auto-generated Waive PR

Created by: TensorRT LLM CI Report (requested by @EmmaQiaoCh)
Target branch: main
Bug(s): 6144270, 6156233, 6191524, 6248744, 6248757, 6248764, 6248769, 6248776, 6248783, 6248827, 6248837

Waive entries added

accuracy/test_llm_api_autodeploy.py::TestModelRegistryAccuracy::test_autodeploy_from_registry[google_gemma-3-1b-it-False] SKIP (https://nvbugs/6248764)
accuracy/test_llm_api_autodeploy.py::TestModelRegistryAccuracy::test_autodeploy_from_registry[mistralai_Ministral-8B-Instruct-2410-False] SKIP (https://nvbugs/6248769)
accuracy/test_llm_api_autodeploy.py::TestNemotronUltraV3::test_accuracy[nvfp4-8] SKIP (https://nvbugs/6248757)
accuracy/test_llm_api_pytorch.py::TestDeepSeekR1::test_fp8_blockscale[throughput_mtp_trtllm] SKIP (https://nvbugs/6191524)
accuracy/test_llm_api_pytorch.py::TestGPTOSS::test_dflash SKIP (https://nvbugs/6156233)
accuracy/test_llm_api_pytorch.py::TestKimiK25::test_nvfp4[tp8] SKIP (https://nvbugs/6248837)
accuracy/test_llm_api_pytorch.py::TestKimiK25::test_nvfp4[tp8_attn_dp] SKIP (https://nvbugs/6144270)
accuracy/test_llm_api_pytorch.py::TestMistralLarge3_675B::test_nvfp4_4gpus[latency_moe_trtllm] SKIP (https://nvbugs/6248827)
accuracy/test_llm_api_pytorch.py::TestQwen3_30B_A3B_Instruct_2507::test_skip_softmax_attention_4gpus[target_sparsity_0.5-fp8kv=True] SKIP (https://nvbugs/6248783)
accuracy/test_llm_api_pytorch_multimodal.py::TestNemotron_Nano_12B_V2_VL::test_auto_dtype[forced_chunked_prefill] SKIP (https://nvbugs/6248744)
unittest/llmapi/test_llm_pytorch.py::test_nemotron_nas_lora[None] SKIP (https://nvbugs/6248776)
unittest/llmapi/test_llm_pytorch.py::test_nemotron_nas_lora[cuda_graph_config0] SKIP (https://nvbugs/6248776)

---

This PR was auto-generated by TensorRT LLM CI Report. Please review the waive entries before merging.

Summary by CodeRabbit

• Tests
  • Expanded test waiver list with additional entries for parameterized test configurations across multiple model variants and their related scenarios. New waivers improve test execution tracking for various model types, their integration tests, and accuracy testing cases across different configurations and deployment platforms.

[EmmaQiaoCh]

/bot run

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR adds twelve new SKIP waiver entries to the test exclusion list, primarily for autodeploy registry accuracy tests and various model-specific parameterized test cases that are currently failing and need to be skipped from the CI suite.

Changes

Test Waiver Updates

Layer / File(s)

Summary

Add test skip waivers for various accuracy and unit tests tests/integration/test_lists/waives.txt

Added SKIP entries for TestModelRegistryAccuracy autodeploy tests with Nemotron/Qwen3 variants (lines 18–25), TestGPTOSS dflash test (line 72), TestKimiK25 nvfp4 tests with tensor parallelism (lines 86–87), TestMistralLarge3_675B nvfp4 latency test (line 125), TestQwen3_30B sparsity attention test (line 134), TestNemotron_Nano_12B_V2_VL multimodal dtype test (line 143), and test_nemotron_nas_lora unit test configurations (lines 374–375).

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Possibly related PRs

• NVIDIA/TensorRT-LLM#14737: Both PRs modify tests/integration/test_lists/waives.txt by adding SKIP waiver entries for accuracy/test_llm_api_autodeploy.py::TestModelRegistryAccuracy::test_autodeploy_from_registry test parameterizations.
• NVIDIA/TensorRT-LLM#14615: Both PRs modify tests/integration/test_lists/waives.txt by adding SKIP entries for overlapping accuracy/test_llm_api_autodeploy.py Nemotron autodeploy test scenarios.
• NVIDIA/TensorRT-LLM#13829: Both PRs modify tests/integration/test_lists/waives.txt by adding new SKIP waiver entries for failing test cases at the waiver-list level.

Suggested reviewers

• chzblych
• EmmaQiaoCh

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name	Status	Explanation	Resolution
Description check	❓ Inconclusive	The description is auto-generated but lacks some template sections like 'Description', 'Test Coverage', and PR Checklist confirmation despite being a waive PR.	Consider adding brief descriptions of what prompted the waivers and confirming the PR checklist items, or clarify if this format is acceptable for auto-generated waive PRs.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly follows the required format with [None][infra] prefix and accurately describes adding 12 waive entries for failed test cases.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

🧹 Nitpick comments (1)

tests/integration/test_lists/waives.txt (1)

18-19: ⚡ Quick win

Add explicit follow-up tracking for waived coverage gaps.

Waive syntax/format looks correct, but coverage is now insufficient for the newly skipped paths in:

• accuracy/test_llm_api_autodeploy.py
• accuracy/test_llm_api_pytorch.py
• accuracy/test_llm_api_pytorch_multimodal.py
• unittest/llmapi/test_llm_pytorch.py

Please add/confirm a linked follow-up tracker per file (owner + target unwaive milestone) outside this PR so these 12 SKIPs don’t become permanent.

Based on learnings: "In tests/integration/test_lists/waives.txt, prefer the short nvbug URL format https://nvbugs/XXXX ... ensure consistency by using the short form in this file."

Also applies to: 22-22, 25-25, 72-72, 86-87, 125-125, 134-134, 143-143, 374-375

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/integration/test_lists/waives.txt` around lines 18 - 19, Update
tests/integration/test_lists/waives.txt to include explicit follow-up tracker
links and owner+target unwaive milestone references for each waived test group
listed (specifically for accuracy/test_llm_api_autodeploy.py,
accuracy/test_llm_api_pytorch.py, accuracy/test_llm_api_pytorch_multimodal.py,
and unittest/llmapi/test_llm_pytorch.py) and replace long nvbug URLs with the
short form (https://nvbugs/XXXX) for the existing waived entries (including
those around lines 22,25,72,86-87,125,134,143,374-375); ensure each waived line
has a short nvbug reference plus an owner and target milestone noted in a
consistent comment format so maintainers can track and unwaive later.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tests/integration/test_lists/waives.txt`:
- Around line 18-19: Update tests/integration/test_lists/waives.txt to include
explicit follow-up tracker links and owner+target unwaive milestone references
for each waived test group listed (specifically for
accuracy/test_llm_api_autodeploy.py, accuracy/test_llm_api_pytorch.py,
accuracy/test_llm_api_pytorch_multimodal.py, and
unittest/llmapi/test_llm_pytorch.py) and replace long nvbug URLs with the short
form (https://nvbugs/XXXX) for the existing waived entries (including those
around lines 22,25,72,86-87,125,134,143,374-375); ensure each waived line has a
short nvbug reference plus an owner and target milestone noted in a consistent
comment format so maintainers can track and unwaive later.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 34602a06-481a-48e6-b836-4864de3330bb

📥 Commits

Reviewing files that changed from the base of the PR and between 5f5b772 and c31560f.

📒 Files selected for processing (1)

• tests/integration/test_lists/waives.txt

[tensorrt-cicd]

PR_Github #51290 [ run ] triggered by Bot. Commit: c31560f Link to invocation

[EmmaQiaoCh]

/bot skip --comment "The previous CI passed check test list"

[tensorrt-cicd]

PR_Github #51326 [ skip ] triggered by Bot. Commit: c31560f Link to invocation

[tensorrt-cicd]

PR_Github #51290 [ run ] completed with state ABORTED. Commit: c31560f

Link to invocation

[tensorrt-cicd]

PR_Github #51326 [ skip ] completed with state SUCCESS. Commit: c31560f
Skipping testing for commit c31560f

Link to invocation