Skip to content

[None][infra] Waive 1 failed cases for main in post-merge - #12390

Closed
xinhe-nv wants to merge 1 commit into
NVIDIA:mainfrom
xinhe-nv:trtllm-ci-report/waive-20260320-053615
Closed

[None][infra] Waive 1 failed cases for main in post-merge#12390
xinhe-nv wants to merge 1 commit into
NVIDIA:mainfrom
xinhe-nv:trtllm-ci-report/waive-20260320-053615

Conversation

@xinhe-nv

@xinhe-nv xinhe-nv commented Mar 20, 2026

Copy link
Copy Markdown
Collaborator

Auto-generated Waive PR

Created by: TensorRT LLM CI Report (requested by qa@nvidia.com)
Target branch: main
Bug(s): 5756008

Waive entries added

TestLlama3_1_8BInstruct::test_guided_decoding_4gpus[llguidance] SKIP (https://nvbugs/5756008)

This PR was auto-generated by TensorRT LLM CI Report. Please review the waive entries before merging.

Summary by CodeRabbit

  • Tests
    • Updated test configuration to skip a specific test scenario pending resolution.

Note: This release contains no end-user visible changes. The update is limited to internal test infrastructure.

Bug(s): 5756008
Requested by: qa@nvidia.com

Signed-off-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com>
@coderabbitai

coderabbitai Bot commented Mar 20, 2026

Copy link
Copy Markdown
Contributor
📝 Walkthrough

Walkthrough

Added a single waiver entry to the test skip list for TestLlama3_1_8BInstruct::test_guided_decoding_4gpus[llguidance], marking the test configuration to be skipped with reference to the nvbugs URL.

Changes

Cohort / File(s) Summary
Test Waiver Update
tests/integration/test_lists/waives.txt
Added one SKIP waiver entry for a specific test configuration variant.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~2 minutes

Possibly related PRs

Suggested reviewers

  • EmmaQiaoCh
  • chzblych
🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description is largely incomplete, missing key sections from the template such as 'Description', 'Test Coverage', and the PR Checklist, though it provides auto-generated waive information. Add the missing 'Description' and 'Test Coverage' sections, and include the PR Checklist with at least a basic confirmation of review.
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately describes the main change: adding a waiver entry for a failed test case in the infrastructure, following the repository's title template format with ticket reference and type.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
📝 Coding Plan
  • Generate coding plan for human review comments

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@tests/integration/test_lists/waives.txt`:
- Line 375: The waiver entry at line containing
"TestLlama3_1_8BInstruct::test_guided_decoding_4gpus[llguidance]" is missing the
required file path prefix; update that waiver string to include the full path
prefix "accuracy/test_llm_api.py::" so it matches the other entry for the same
test class (e.g., change
"TestLlama3_1_8BInstruct::test_guided_decoding_4gpus[llguidance]" to
"accuracy/test_llm_api.py::TestLlama3_1_8BInstruct::test_guided_decoding_4gpus[llguidance]")
to ensure the test framework recognizes the waiver.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: e2f46dda-a078-44f3-a62f-af1760c63756

📥 Commits

Reviewing files that changed from the base of the PR and between 7dd0865 and 90cb053.

📒 Files selected for processing (1)
  • tests/integration/test_lists/waives.txt

perf/test_perf_sanity.py::test_e2e[disagg_upload-gen_only-gb200_deepseek-r1-fp4_1k1k_con2048_ctx2_dep4_gen1_dep16_eplb0_mtp3_ccb-UCX] SKIP (https://nvbugs/5996645)
perf/test_perf_sanity.py::test_e2e[disagg_upload-e2e-gb200_deepseek-r1-fp4_1k1k_con2048_ctx2_dep4_gen1_dep16_eplb288_mtp3_ccb-UCX] SKIP (https://nvbugs/5997090)
perf/test_perf_sanity.py::test_e2e[aggr_upload-deepseek_v32_fp4_blackwell-v32_fp4_tep8_mtp3_8k1k] SKIP (https://nvbugs/5997092)
TestLlama3_1_8BInstruct::test_guided_decoding_4gpus[llguidance] SKIP (https://nvbugs/5756008)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Missing file path prefix in waiver entry.

The new waiver entry is missing the file path prefix accuracy/test_llm_api.py:: that appears in the similar entry at line 121 for the same test class. All other entries in this file include their full path (e.g., examples/, accuracy/, perf/, etc.).

Compare:

  • Line 121: accuracy/test_llm_api.py::TestLlama3_1_8BInstruct::test_guided_decoding_4gpus[xgrammar]
  • Line 375: TestLlama3_1_8BInstruct::test_guided_decoding_4gpus[llguidance] (missing path)

Without the correct path prefix, the waiver may not be recognized by the test framework.

🔧 Proposed fix
-TestLlama3_1_8BInstruct::test_guided_decoding_4gpus[llguidance] SKIP (https://nvbugs/5756008)
+accuracy/test_llm_api.py::TestLlama3_1_8BInstruct::test_guided_decoding_4gpus[llguidance] SKIP (https://nvbugs/5756008)
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
TestLlama3_1_8BInstruct::test_guided_decoding_4gpus[llguidance] SKIP (https://nvbugs/5756008)
accuracy/test_llm_api.py::TestLlama3_1_8BInstruct::test_guided_decoding_4gpus[llguidance] SKIP (https://nvbugs/5756008)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/integration/test_lists/waives.txt` at line 375, The waiver entry at
line containing
"TestLlama3_1_8BInstruct::test_guided_decoding_4gpus[llguidance]" is missing the
required file path prefix; update that waiver string to include the full path
prefix "accuracy/test_llm_api.py::" so it matches the other entry for the same
test class (e.g., change
"TestLlama3_1_8BInstruct::test_guided_decoding_4gpus[llguidance]" to
"accuracy/test_llm_api.py::TestLlama3_1_8BInstruct::test_guided_decoding_4gpus[llguidance]")
to ensure the test framework recognizes the waiver.

@xinhe-nv xinhe-nv closed this Mar 20, 2026
@xinhe-nv
xinhe-nv deleted the trtllm-ci-report/waive-20260320-053615 branch March 20, 2026 06:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant