Skip to content

[None][test] update bug ids in waives - #14946

Merged
xinhe-nv merged 2 commits into
NVIDIA:mainfrom
xinhe-nv:update-wavie
Jun 4, 2026
Merged

[None][test] update bug ids in waives#14946
xinhe-nv merged 2 commits into
NVIDIA:mainfrom
xinhe-nv:update-wavie

Conversation

@xinhe-nv

@xinhe-nv xinhe-nv commented Jun 4, 2026

Copy link
Copy Markdown
Collaborator

Summary by CodeRabbit

  • Tests
    • Updated integration test waiver configurations and associated tracking references.

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

  • PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

  • PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

  • Test cases are provided for new code paths (see test instructions)

  • If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

  • Any new dependencies have been scanned for license and vulnerabilities

  • CODEOWNERS updated if ownership changes

  • Documentation updated as needed

  • Update tava architecture diagram if there is a significant design change in PR.

  • The reviewers assigned automatically/manually are appropriate for the PR.

  • Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>
@xinhe-nv
xinhe-nv requested a review from yuxianq June 4, 2026 07:03
@xinhe-nv
xinhe-nv enabled auto-merge (squash) June 4, 2026 07:03
@xinhe-nv

xinhe-nv commented Jun 4, 2026

Copy link
Copy Markdown
Collaborator Author

/bot run --stage-list ""

@coderabbitai

coderabbitai Bot commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

Updates to tests/integration/test_lists/waives.txt refresh NVBug issue references for test waivers across multiple TestDeepSeekV3Lite configurations (bfloat16, fp8_block_scales, nvfp4 variants) and one TestLlama3_1_8BInstruct fp8 configuration with TRTLLM backend and torch_compile enabled.

Changes

Test waiver NVBug reference updates

Layer / File(s) Summary
DeepSeekV3Lite and Llama waiver NVBug updates
tests/integration/test_lists/waives.txt
NVBug issue references for TestDeepSeekV3Lite test configurations (bfloat16_4gpus, fp8_block_scales_4gpus tp2pp2, nvfp4_4gpus CUTLASS variants) and TestLlama3_1_8BInstruct test_fp8_4gpus tp4 configuration are updated to point to different NVBug IDs.

🎯 1 (Trivial) | ⏱️ ~5 minutes

Possibly related PRs

  • NVIDIA/TensorRT-LLM#14854: Both PRs only update tests/integration/test_lists/waives.txt, changing SKIP/waiver entries for the same test classes/parameterizations (e.g., TestDeepSeekV3Lite and TestLlama3_1_8BInstruct) along with the associated NVBug IDs.
  • NVIDIA/TensorRT-LLM#14857: Both PRs modify the same waiver file tests/integration/test_lists/waives.txt to update/insert NVBug-based skip entries for integration tests, so the change sets are directly related.
  • NVIDIA/TensorRT-LLM#14866: Both PRs update the same tests/integration/test_lists/waives.txt entries for accuracy/test_llm_api_pytorch.py (notably TestDeepSeekV3Lite and TestLlama3_1_8BInstruct), changing the skip/waiver NVBug references for those specific test configurations.

Suggested reviewers

  • LarryXFly
  • jieli-matrix
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The PR description is entirely the template with no actual content filled in—the Description, Test Coverage sections are empty and only the checklist box is marked. Fill in the Description section explaining why bug IDs were updated, and the Test Coverage section listing relevant tests affected by these waiver changes.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately describes the primary change: updating bug IDs in the waives.txt file for integration test configurations.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
tests/integration/test_lists/waives.txt (1)

45-48: QA coverage status: sufficient for this waiver-metadata PR; needs follow-up outside this PR for unwaive readiness.

For this PR scope (bug-id refresh in tests/integration/test_lists/waives.txt), coverage is sufficient because no executable test content changed.
Follow-up outside this PR (when each NVBug is resolved): update tests/integration/test_lists/waives.txt by removing/adjusting these exact entries and re-run the affected suites in accuracy/test_llm_api_pytorch.py for:

  • TestDeepSeekV3Lite::test_bfloat16_4gpus variants (Line 45-48)
  • TestDeepSeekV3Lite::test_fp8_block_scales_4gpus variants (Line 53-56)
  • TestDeepSeekV3Lite::test_nvfp4_4gpus CUTLASS variants (Line 64-66)
  • TestLlama3_1_8BInstruct::test_bfloat16_4gpus / test_fp8_4gpus variants (Line 93, Line 98)

Based on learnings: In tests/integration/test_lists/waives.txt, the short nvbug URL format https://nvbugs/XXXX is preferred and is correctly used here.

Also applies to: 53-56, 64-66, 93-93, 98-98

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/integration/test_lists/waives.txt` around lines 45 - 48, Waiver entries
for TestDeepSeekV3Lite::test_bfloat16_4gpus (and other listed variants) in
tests/integration/test_lists/waives.txt are temporary and must be updated once
the referenced NVBugs are fixed: remove or adjust the exact entries matching
accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_bfloat16_4gpus (Lines
45-48),
accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_fp8_block_scales_4gpus
(Lines 53-56),
accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_nvfp4_4gpus (Lines
64-66), and
accuracy/test_llm_api_pytorch.py::TestLlama3_1_8BInstruct::test_bfloat16_4gpus /
test_fp8_4gpus (Lines 93,98) after the corresponding NVBugs are resolved; then
re-run the affected suites in accuracy/test_llm_api_pytorch.py to confirm tests
can be unwaived, and keep using the short nvbug URL format (https://nvbugs/XXXX)
when editing waives.txt.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tests/integration/test_lists/waives.txt`:
- Around line 45-48: Waiver entries for TestDeepSeekV3Lite::test_bfloat16_4gpus
(and other listed variants) in tests/integration/test_lists/waives.txt are
temporary and must be updated once the referenced NVBugs are fixed: remove or
adjust the exact entries matching
accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_bfloat16_4gpus (Lines
45-48),
accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_fp8_block_scales_4gpus
(Lines 53-56),
accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_nvfp4_4gpus (Lines
64-66), and
accuracy/test_llm_api_pytorch.py::TestLlama3_1_8BInstruct::test_bfloat16_4gpus /
test_fp8_4gpus (Lines 93,98) after the corresponding NVBugs are resolved; then
re-run the affected suites in accuracy/test_llm_api_pytorch.py to confirm tests
can be unwaived, and keep using the short nvbug URL format (https://nvbugs/XXXX)
when editing waives.txt.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: a3f332da-61c5-4104-a813-ea82a12b9e95

📥 Commits

Reviewing files that changed from the base of the PR and between 4437cb9 and bb20189.

📒 Files selected for processing (1)
  • tests/integration/test_lists/waives.txt

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #52002 [ run ] triggered by Bot. Commit: bb20189 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #52002 [ run ] completed with state SUCCESS. Commit: bb20189
/LLM/main/L0_MergeRequest_PR pipeline #41346 (Partly Tested) completed with status: 'SUCCESS'

CI Report

Link to invocation

@xinhe-nv

xinhe-nv commented Jun 4, 2026

Copy link
Copy Markdown
Collaborator Author

/bot reuse-pipeline

@xinhe-nv
xinhe-nv requested a review from StanleySun639 June 4, 2026 08:22
@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #52021 [ reuse-pipeline ] triggered by Bot. Commit: e93d359 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #52021 [ reuse-pipeline ] completed with state SUCCESS. Commit: e93d359
Reusing PR_Github #52002 (Partly Tested) for commit e93d359

Link to invocation

@xinhe-nv
xinhe-nv merged commit 0f5dc5a into NVIDIA:main Jun 4, 2026
7 checks passed
@xinhe-nv
xinhe-nv deleted the update-wavie branch June 4, 2026 08:39
yufeiwu-nv pushed a commit to yufeiwu-nv/TensorRT-LLM that referenced this pull request Jun 4, 2026
Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>
Signed-off-by: yufeiwu-nv <230315618+yufeiwu-nv@users.noreply.github.com>
2ez4bz pushed a commit to 2ez4bz/TensorRT-LLM that referenced this pull request Jun 8, 2026
Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants