[https://nvbugs/6302903][infra] Unwaive deepseek-r1-fp4 disagg perf-sanity e2e test by xwang233 · Pull Request #15787 · NVIDIA/TensorRT-LLM

xwang233 · 2026-06-30T16:50:04Z

Unwaive perf/test_perf_sanity.py::test_e2e[disagg_upload-e2e-gb200_deepseek-r1-fp4_8k1k_con4096_ctx1_dep4_gen1_dep16_eplb0_mtp1_ccb-NIXL] (nvbugs/6302903).

The test passes in a local reproduction run on GB200 (0/20480 failed requests), so the waiver is removed to let CI exercise it again.

Summary by CodeRabbit

Chores
- Removed an obsolete test waiver from the integration test list.

…anity e2e test Signed-off-by: Xiao Wang <24860335+xwang233@users.noreply.github.com>

coderabbitai · 2026-06-30T16:51:18Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: cbe586bd-d472-4bbf-aba5-fb53c4339bb5

📥 Commits

Reviewing files that changed from the base of the PR and between 23c835d and afada7d.

📒 Files selected for processing (1)

tests/integration/test_lists/waives.txt

💤 Files with no reviewable changes (1)

tests/integration/test_lists/waives.txt

📝 Walkthrough

Walkthrough

Removes one SKIP waiver entry from tests/integration/test_lists/waives.txt for a specific perf/test_perf_sanity.py::test_e2e disagg_upload configuration on GB200 with DeepSeek-R1-FP4, previously waived via nvbugs/6302903.

Changes

Waiver Removal

Layer / File(s)	Summary
Remove perf test waiver entry `tests/integration/test_lists/waives.txt`	Deletes the SKIP waiver for `perf/test_perf_sanity.py::test_e2e[disagg_upload-e2e-gb200_deepseek-r1-fp4_8k1k_con4096_ctx1_dep4_gen1_dep16_eplb0_mtp1_ccb-NIXL]` (nvbugs/6302903).

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~2 minutes

Possibly related PRs

NVIDIA/TensorRT-LLM#14450: Modifies the same waives.txt file by adding/removing SKIP waiver entries.
NVIDIA/TensorRT-LLM#15774: Removes/adjusts SKIP waivers in the same file for overlapping DeepSeek parametrized test cases.
NVIDIA/TensorRT-LLM#15778: Modifies the same waiver file by removing or adding SKIP entries for integration tests.

Suggested reviewers

jieli-matrix
xinhe-nv
yingguo-trt

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description explains why the waiver was removed, but it omits the required template sections for Description, Test Coverage, and PR Checklist.	Add the required template sections, especially a brief Description, Test Coverage, and completed PR Checklist items.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title matches the change: it identifies the nvbugs ticket and clearly says the perf-sanity test is being unwaived.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands.}

xwang233 · 2026-06-30T16:52:02Z

/bot run --add-multi-gpu-test --stage-list "GB200-20_GPUs-5_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE4-GPU16-Post-Merge"

tensorrt-cicd · 2026-06-30T16:58:29Z

PR_Github #56668 [ run ] triggered by Bot. Commit: afada7d Link to invocation

tensorrt-cicd · 2026-06-30T17:14:12Z

PR_Github #56668 [ run ] completed with state FAILURE. Commit: afada7d
/LLM/main/L0_MergeRequest_PR pipeline #45492 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

xwang233 · 2026-06-30T17:54:11Z

/bot run --add-multi-gpu-test --stage-list "GB200-20_GPUs-5_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE4-GPU16-Post-Merge"

tensorrt-cicd · 2026-06-30T18:03:07Z

PR_Github #56686 [ run ] triggered by Bot. Commit: afada7d Link to invocation

tensorrt-cicd · 2026-06-30T18:54:18Z

PR_Github #56686 [ run ] completed with state FAILURE. Commit: afada7d
/LLM/main/L0_MergeRequest_PR pipeline #45509 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

…eek-r1-disagg-perfsanity Signed-off-by: Xiao Wang <xiaow@nvidia.com> Signed-off-by: Xiao Wang <24860335+xwang233@users.noreply.github.com>

xwang233 · 2026-06-30T22:58:45Z

/bot run --stage-list "GB200-20_GPUs-5_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE4-GPU16-Post-Merge-*"

tensorrt-cicd · 2026-06-30T23:04:20Z

PR_Github #56752 [ run ] triggered by Bot. Commit: ceb1189 Link to invocation

tensorrt-cicd · 2026-06-30T23:51:25Z

PR_Github #56752 [ run ] completed with state FAILURE. Commit: ceb1189
/LLM/main/L0_MergeRequest_PR pipeline #45572 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

xwang233 · 2026-07-01T00:21:26Z

/bot run --stage-list "GB200-20_GPUs-5_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE4-GPU16-Post-Merge-*"

tensorrt-cicd · 2026-07-01T00:27:13Z

PR_Github #56764 [ run ] triggered by Bot. Commit: ceb1189 Link to invocation

tensorrt-cicd · 2026-07-01T01:24:40Z

PR_Github #56764 [ run ] completed with state SUCCESS. Commit: ceb1189
/LLM/main/L0_MergeRequest_PR pipeline #45585 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

xwang233 · 2026-07-01T03:28:44Z

/bot run --stage-list "GB200-20_GPUs-5_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE4-GPU16-Post-Merge-*" --disable-fail-fast

tensorrt-cicd · 2026-07-01T03:34:11Z

PR_Github #56814 [ run ] triggered by Bot. Commit: 5cf29c8 Link to invocation

tensorrt-cicd · 2026-07-01T05:35:17Z

PR_Github #56814 [ run ] completed with state SUCCESS. Commit: 5cf29c8
/LLM/main/L0_MergeRequest_PR pipeline #45630 (Partly Tested) completed with status: 'SUCCESS'

CI Report

Link to invocation

[https://nvbugs/6302903][infra] Unwaive deepseek-r1-fp4 disagg perf-s…

afada7d

…anity e2e test Signed-off-by: Xiao Wang <24860335+xwang233@users.noreply.github.com>

github-actions Bot assigned xwang233 Jun 30, 2026

Merge remote-tracking branch 'origin/main' into unwaive-6302903-deeps…

ceb1189

…eek-r1-disagg-perfsanity Signed-off-by: Xiao Wang <xiaow@nvidia.com> Signed-off-by: Xiao Wang <24860335+xwang233@users.noreply.github.com>

Merge branch 'main' into unwaive-6302903-deepseek-r1-disagg-perfsanity

5cf29c8

xwang233 closed this Jul 6, 2026

Uh oh!

Conversation

xwang233 commented Jun 30, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Jun 30, 2026

Walkthrough

Changes

Estimated code review effort

Possibly related PRs

Suggested reviewers

❌ Failed checks (1 warning)

Uh oh!

xwang233 commented Jun 30, 2026

Uh oh!

tensorrt-cicd commented Jun 30, 2026

Uh oh!

tensorrt-cicd commented Jun 30, 2026

Uh oh!

xwang233 commented Jun 30, 2026

Uh oh!

tensorrt-cicd commented Jun 30, 2026

Uh oh!

tensorrt-cicd commented Jun 30, 2026

Uh oh!

xwang233 commented Jun 30, 2026

Uh oh!

tensorrt-cicd commented Jun 30, 2026

Uh oh!

tensorrt-cicd commented Jun 30, 2026

Uh oh!

xwang233 commented Jul 1, 2026

Uh oh!

tensorrt-cicd commented Jul 1, 2026

Uh oh!

tensorrt-cicd commented Jul 1, 2026

Uh oh!

xwang233 commented Jul 1, 2026

Uh oh!

tensorrt-cicd commented Jul 1, 2026

Uh oh!

tensorrt-cicd commented Jul 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

xwang233 commented Jun 30, 2026 •

edited by coderabbitai Bot

Loading