Skip to content

[https://nvbugs/6427411][chore] Waive failed tests in Pre-merge - #16103

Merged
chienchunhung merged 1 commit into
NVIDIA:mainfrom
shuyixiong:waive_pp_test
Jul 8, 2026
Merged

[https://nvbugs/6427411][chore] Waive failed tests in Pre-merge#16103
chienchunhung merged 1 commit into
NVIDIA:mainfrom
shuyixiong:waive_pp_test

Conversation

@shuyixiong

@shuyixiong shuyixiong commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator

Summary by CodeRabbit

  • Tests
    • Updated the integration test waiver list to account for additional skipped cases across disaggregated serving, LLM API, PyTorch, and Ray scenarios.
    • Adjusted existing waivers and added new ones for broader model and configuration coverage, including inference and output-processing cases.

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

  • PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

  • PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

  • Test cases are provided for new code paths (see test instructions)

  • If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

  • Any new dependencies have been scanned for license and vulnerabilities

  • CODEOWNERS updated if ownership changes

  • Documentation updated as needed

  • Update tava architecture diagram if there is a significant design change in PR.

  • The reviewers assigned automatically/manually are appropriate for the PR.

  • Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

Signed-off-by: Shuyi Xiong <219646547+shuyixiong@users.noreply.github.com>
@shuyixiong shuyixiong changed the title [https://nvbugs/6427411] Waive failed tests in Pre-merge [https://nvbugs/6427411][chore] Waive failed tests in Pre-merge Jul 8, 2026
@shuyixiong

Copy link
Copy Markdown
Collaborator Author

/bot skip --comment "Waive tests"

@coderabbitai

coderabbitai Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

This PR updates the integration test waives list by adding new SKIP entries with nvbug references for disaggregated serving, LLM API PyTorch/Ray accuracy tests, and unittest multi-GPU cases, and replaces one existing unittest waive with three new ones.

Changes

Waives list additions

Layer / File(s) Summary
Disaggregated serving and disaggregated test waives
tests/integration/test_lists/waives.txt
New SKIP entries for DeepSeekV3Lite helix/precision configurations, Llama3_1_8BInstruct tensor-parallelism patterns in test_disaggregated_serving.py, and a new test_disaggregated_ctxpp2_genpp2 waive.
LLM API PyTorch and Ray accuracy test waives
tests/integration/test_lists/waives.txt
New SKIP entries for DeepSeekV3Lite fp8_block_scales_4gpus/nvfp4_4gpus, Llama3/Nemotron parallelism variants, test_pp2_ray, and Ray example inference tests (pp2, tp2pp2).
Unittest multi-GPU waive replacement
tests/integration/test_lists/waives.txt
Removed the phi3 lora fused modules waive and added waives for test_additional_model_outputs_integration_pp2 and two tinyllama logits processor multi-GPU cases.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Possibly related PRs

  • NVIDIA/TensorRT-LLM#16093: Both PRs add new SKIP/waive entries to the same waives.txt file for newly failing DeepSeek/disaggregated/Ray-related CI test parameterizations.

Suggested reviewers: dongxuy04, pcastonguay, yuxianq, mzweilz

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description is still the template skeleton and does not explain the issue, solution, or test coverage. Fill in the Description and Test Coverage sections with the problem, the waiver changes made, and the tests that cover them.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title follows the required ticket/type format and clearly summarizes the test-waiver change.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/integration/test_lists/waives.txt`:
- Around line 431-434: The waiver list update is adding new skips without
removing the stale phi3 waiver, so this replacement is broader than intended. In
the waives file, update the affected hunk so the existing
unittest/llmapi/test_llm_multi_gpu_pytorch.py::test_phi3_lora_fused_modules_output_on_tp2_identical_to_tp1
entry is removed if this change is meant to replace it, while keeping the
intended new tinyllama-related waives only. Verify the final list reflects the
exact current skip set and does not retain obsolete entries.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: b733868e-56a0-4c7c-b0de-33710f84a52d

📥 Commits

Reviewing files that changed from the base of the PR and between 3eb7feb and 8f5a95d.

📒 Files selected for processing (1)
  • tests/integration/test_lists/waives.txt

Comment thread tests/integration/test_lists/waives.txt
@chienchunhung
chienchunhung self-requested a review July 8, 2026 05:32
@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #58155 [ skip ] triggered by Bot. Commit: 8f5a95d Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #58155 [ skip ] completed with state SUCCESS. Commit: 8f5a95d
Skipping testing for commit 8f5a95d

Link to invocation

@chienchunhung
chienchunhung merged commit 8c1b230 into NVIDIA:main Jul 8, 2026
13 of 17 checks passed
BrianLi23 pushed a commit to BrianLi23/TensorRT-LLM that referenced this pull request Jul 9, 2026
…IA#16103)

Signed-off-by: Shuyi Xiong <219646547+shuyixiong@users.noreply.github.com>
mingyangHao added a commit to mingyangHao/TensorRT-LLM that referenced this pull request Jul 9, 2026
Remove the PP sampler waivers introduced by NVIDIA#16103, NVIDIA#16105, NVIDIA#16127, and NVIDIA#16146. Also remove overlapping PP and gpu2 waivers from NVIDIA#16109 so the restored pre-NVIDIA#15920 sampler path is exercised in CI.

Signed-off-by: Mingyang Hao <200044211+mingyangHao@users.noreply.github.com>
mingyangHao added a commit to mingyangHao/TensorRT-LLM that referenced this pull request Jul 9, 2026
Remove the PP sampler waivers introduced by NVIDIA#16103, NVIDIA#16105, NVIDIA#16127, and NVIDIA#16146. Also remove overlapping PP and gpu2 waivers from NVIDIA#16109, plus the PP2 waiver added by NVIDIA#16169, so the restored pre-NVIDIA#15920 sampler path is exercised in CI.

Signed-off-by: Mingyang Hao <200044211+mingyangHao@users.noreply.github.com>
mingyangHao added a commit to mingyangHao/TensorRT-LLM that referenced this pull request Jul 9, 2026
Remove the PP sampler waivers introduced by NVIDIA#16103, NVIDIA#16105, NVIDIA#16127, and NVIDIA#16146. Also remove overlapping PP and gpu2 waivers from NVIDIA#16109, plus the PP2 waiver added by NVIDIA#16169, so the restored pre-NVIDIA#15920 sampler path is exercised in CI.

Signed-off-by: Mingyang Hao <200044211+mingyangHao@users.noreply.github.com>
trtllm-agent added a commit to tensorrt-cicd/TensorRT-LLM that referenced this pull request Jul 20, 2026
The TorchSampler IndexError described in nvbug 6435097 was introduced by
PR NVIDIA#15920 ("Move greedy stop checks to host") and cleanly reverted by
PR NVIDIA#16163 (commit 9a8ec05). PR NVIDIA#16163 is already present at HEAD, so
the underlying code bug is fixed.

The waiver at nvbugs/6427411 for
test_ctx_pp_gen_tp_asymmetric[GSM8K-gen_tp=1-ctx_pp=2] was added by
PRs NVIDIA#16103 / NVIDIA#16105 / NVIDIA#16127 before the revert landed and was not
cleaned up afterward.

Verified locally on this HEAD (e523b43):
  pytest tests/integration/defs/accuracy/test_disaggregated_serving.py::\
    TestLlama3_1_8BInstruct::test_ctx_pp_gen_tp_asymmetric[GSM8K-gen_tp=1-ctx_pp=2]
  -> PASSED, GSM8K accuracy = 73.465 (threshold 70.997).

Signed-off-by: trtllm-agent <296075020+trtllm-agent@users.noreply.github.com>
trtllm-agent added a commit to tensorrt-cicd/TensorRT-LLM that referenced this pull request Jul 21, 2026
The TorchSampler IndexError described in nvbug 6435097 was introduced by
PR NVIDIA#15920 ("Move greedy stop checks to host") and cleanly reverted by
PR NVIDIA#16163 (commit 9a8ec05). PR NVIDIA#16163 is already present at HEAD, so
the underlying code bug is fixed.

The waiver at nvbugs/6427411 for
test_ctx_pp_gen_tp_asymmetric[GSM8K-gen_tp=1-ctx_pp=2] was added by
PRs NVIDIA#16103 / NVIDIA#16105 / NVIDIA#16127 before the revert landed and was not
cleaned up afterward.

Verified locally on this HEAD (e523b43):
  pytest tests/integration/defs/accuracy/test_disaggregated_serving.py::\
    TestLlama3_1_8BInstruct::test_ctx_pp_gen_tp_asymmetric[GSM8K-gen_tp=1-ctx_pp=2]
  -> PASSED, GSM8K accuracy = 73.465 (threshold 70.997).

Signed-off-by: trtllm-agent <296075020+trtllm-agent@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants