[None][infra] Waive 8 failed cases for main in post-merge 2738

[ZhanruiSunCh]

Auto-generated Waive PR

Created by: TensorRT LLM CI Report (requested by @mzweilz)
Target branch: main
Bug(s): 6221022, 6221024, 6221483, 6227203, 6227472

Waive entries added

accuracy/test_llm_api_autodeploy.py::TestNemotronSuperV3::test_mtp[nvfp4_ws4_80gb-trtllm] SKIP (https://nvbugs/6221483)
accuracy/test_llm_api_autodeploy.py::TestNemotronUltraV3::test_accuracy[nvfp4-4] SKIP (https://nvbugs/6221483)
perf/test_perf_sanity.py::test_e2e[aggr_upload-dynamo_k25_thinking_fp4_blackwell-k25_thinking_fp4_tep8_adp_2k1k] SKIP (https://nvbugs/6227472)
perf/test_perf_sanity.py::test_e2e[aggr_upload-k25_thinking_fp4_blackwell-k25_thinking_fp4_dep8_8k1k] SKIP (https://nvbugs/6227472)
perf/test_perf_sanity.py::test_e2e[aggr_upload-k25_thinking_fp4_blackwell-k25_thinking_fp4_tep8_32k8k] SKIP (https://nvbugs/6227472)
perf/test_perf_sanity.py::test_e2e[disagg_upload-gen_only-gb200_deepseek-r1-fp4_128k8k_con128_ctx1_pp8_gen1_dep16_eplb0_mtp1_ccb-NIXL] SKIP (https://nvbugs/6221024)
perf/test_perf_sanity.py::test_e2e[disagg_upload-gen_only-gb200_kimi-k25-thinking-fp4_8k1k_con4096_ctx1_dep4_gen1_dep16_eplb0_mtp0_ccb-NIXL] SKIP (https://nvbugs/6221022)
test_e2e.py::test_openai_kv_cache_contamination SKIP (https://nvbugs/6227203)

---

This PR was auto-generated by TensorRT LLM CI Report. Please review the waive entries before merging.

Summary by CodeRabbit

• Tests
  • Updated test waivers for integration and performance test scenarios across various configurations.

---

Note: This is an internal testing infrastructure update with no visible impact on end-user functionality.

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR adds eight new test waiver entries across tests/integration/test_lists/waives.txt to skip failing test cases in four distinct areas: Nemotron model configurations, performance sanity tests with aggregated upload scenarios, performance sanity tests with disaggregated upload generation-only variants, and an OpenAI KV cache contamination test.

Changes

Test Waivers

Layer / File(s)	Summary
Test case waivers for failing scenarios tests/integration/test_lists/waives.txt	Four separate waiver entries added to disable failing test cases: Nemotron SuperV3 MTP and UltraV3 accuracy tests, perf_sanity aggr_upload K25-thinking configurations, perf_sanity disagg_upload-gen_only for GB200 DeepSeek and Kimi K25 thinking variants, and OpenAI KV cache contamination test.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Possibly related PRs

• NVIDIA/TensorRT-LLM#14542: Both PRs modify the same waiver list file by adding SKIP entries for specific test cases.
• NVIDIA/TensorRT-LLM#14516: Both PRs update waiver entries with overlapping coverage of perf_sanity aggr_upload and disagg_upload test configurations.
• NVIDIA/TensorRT-LLM#14405: Both PRs modify the same waiver list with overlapping perf_sanity test_e2e entries for aggr_upload and disagg_upload-gen_only scenarios.

Suggested reviewers

• mzweilz

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and specifically describes the main change: adding 8 waive entries for failed CI test cases, with the issue reference.
Description check	✅ Passed	The description covers the essential information (auto-generated nature, target branch, bug references, and specific waive entries), though it deviates from the standard template structure.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

🧹 Nitpick comments (1)

tests/integration/test_lists/waives.txt (1)

313-315: 💤 Low value

Performance test coverage temporarily reduced for K25 thinking configurations.

These three perf sanity tests cover K25 thinking model configurations on Blackwell with different parameter settings (tep8_adp, dep8, tep8 with various context/output lengths). Waiving them removes performance regression detection for these configurations until the underlying issues tracked in NVBug 6227472 are resolved.

As per coding guidelines: Check whether perf test coverage is present for performance-sensitive paths, especially for attention kernels, MoE routing, KV cache management, scheduler, and batching logic.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/integration/test_lists/waives.txt` around lines 313 - 315, The waivers
removed three perf/test_perf_sanity.py::test_e2e cases for K25 thinking configs
(mentioned as SKIP referencing NVBug 6227472), which reduces coverage of
performance-sensitive code paths; restore coverage by adding targeted perf tests
that exercise the same K25 "thinking" configurations or by adding focused
micro-benchmarks for the attention kernels, MoE routing, KV cache management,
scheduler, and batching logic used by test_e2e, and update the test suite so
these new tests (or re-enabled parametrized variants of test_e2e) run even while
the NVBug 6227472 root cause is being addressed. Ensure the tests explicitly
reference the same model/config parameters from the skipped test_e2e cases so
they cover tep8_adp, dep8, and tep8 context/output length variants.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tests/integration/test_lists/waives.txt`:
- Around line 313-315: The waivers removed three
perf/test_perf_sanity.py::test_e2e cases for K25 thinking configs (mentioned as
SKIP referencing NVBug 6227472), which reduces coverage of performance-sensitive
code paths; restore coverage by adding targeted perf tests that exercise the
same K25 "thinking" configurations or by adding focused micro-benchmarks for the
attention kernels, MoE routing, KV cache management, scheduler, and batching
logic used by test_e2e, and update the test suite so these new tests (or
re-enabled parametrized variants of test_e2e) run even while the NVBug 6227472
root cause is being addressed. Ensure the tests explicitly reference the same
model/config parameters from the skipped test_e2e cases so they cover tep8_adp,
dep8, and tep8 context/output length variants.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: f1e9b445-a5eb-40d1-ace2-833a9b9b2834

📥 Commits

Reviewing files that changed from the base of the PR and between 37079f6 and 6555713.

📒 Files selected for processing (1)

• tests/integration/test_lists/waives.txt

[yiqingy0]

/bot skip --comment "waive tests"

[tensorrt-cicd]

PR_Github #50468 [ skip ] triggered by Bot. Commit: 6555713 Link to invocation

[tensorrt-cicd]

PR_Github #50468 [ skip ] completed with state SUCCESS. Commit: 6555713
Skipping testing for commit 6555713

Link to invocation