[TRTLLM-13262][ci] Move non-default-feature tests to post merge by QiJune · Pull Request #15038 · NVIDIA/TensorRT-LLM

QiJune · 2026-06-06T07:26:17Z

cuda_graph and overlap_scheduler are recommended default-on features, and the python_scheduler path is a non-default (P1) feature. The variants that disable these only need post-merge coverage, so move them out of pre_merge to free pre-merge GPU time without dropping any coverage.

Relocated DeepSeekV3Lite 4-GPU tests (pre_merge -> post_merge, same condition):

l0_gb200_multi_gpus.yml: 13 (nvfp4/bfloat16 cuda_graph=False/overlap_scheduler=False + python_scheduler)
l0_dgx_b200.yml: 5 (python_scheduler + bfloat16 cuda_graph=False/overlap_scheduler=False)
l0_dgx_b300.yml: 1 (nvfp4 cuda_graph=False/overlap_scheduler=False)

Pure relocation: per-file test counts unchanged, no duplicates introduced. Pre-merge retains only the cuda_graph=True + overlap_scheduler=True configs.

Summary by CodeRabbit

Tests
- Updated test coverage for various GPU configurations and model optimization variants across multiple hardware platforms to ensure consistent performance validation.

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

coderabbitai · 2026-06-06T07:30:18Z

Complex PR? Review this PR in Change Stack to move by importance, not file order.

📝 Walkthrough

Walkthrough

This PR updates test configuration matrices across three hardware-specific test-db files for DeepSeekV3Lite model testing. Test entries are reorganized between pre-merge and post-merge stages and new parameter variant combinations are added, particularly focusing on MoE backend engines, tensor/expert parallelism, and optimization settings like CUDA graphs and torch compilation.

Changes

DeepSeekV3Lite Test Matrix Reorganization

Layer / File(s)	Summary
DGX-B200 python scheduler test reorganization `tests/integration/test_lists/test-db/l0_dgx_b200.yml`	`TestDeepSeekV3Lite::test_bfloat16_4gpus_python_scheduler` entries for tp4/ep4 (mtp_nextn=0/2) and pp4 variant are removed from one test list segment and added back into a later segment, preserving timeout markers.
DGX-B300 nvfp4 test configuration swap `tests/integration/test_lists/test-db/l0_dgx_b300.yml`	A `TestDeepSeekV3Lite::test_nvfp4_4gpus` entry with TRTLLM backend and mtp_nextn=0 is added to post_merge; a different TRTLLM variant with cuda_graph=False is removed from pre_merge.
GB200 pre-merge and post-merge test matrix reorganization `tests/integration/test_lists/test-db/l0_gb200_multi_gpus.yml`	Pre-merge test list replaces bfloat16 and basic nvfp4 tests with expanded nvfp4 variants using CUTLASS and TRTLLM backends across multiple mtp_nextn, fp8kv, and optimization combinations. Post-merge adds baseline nvfp4 configurations with optimization settings disabled (attention_dp, cuda_graph, overlap_scheduler all false).

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

NVIDIA/TensorRT-LLM#14929: Updates integration test selection matrices for TestDeepSeekV3Lite across similar parameter variants including mtp_nextn and scheduler/precision configurations.

Suggested reviewers

yuanjingx87
dpitman-nvda
niukuo

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies the main change: moving non-default-feature tests from pre-merge to post-merge to optimize pre-merge GPU usage while maintaining coverage.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description check	✅ Passed	PR description clearly explains the purpose (moving non-default-feature tests to reduce pre-merge GPU time) and provides specific details about what tests were relocated and why, matching the template's Description section requirements.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

QiJune · 2026-06-06T07:55:38Z

/bot run

tensorrt-cicd · 2026-06-06T08:02:01Z

PR_Github #52490 [ run ] triggered by Bot. Commit: bd2defe Link to invocation

tensorrt-cicd · 2026-06-06T12:51:17Z

PR_Github #52490 [ run ] completed with state SUCCESS. Commit: bd2defe
/LLM/main/L0_MergeRequest_PR pipeline #41782 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

QiJune · 2026-06-06T13:32:31Z

/bot run

tensorrt-cicd · 2026-06-06T13:37:59Z

PR_Github #52512 [ run ] triggered by Bot. Commit: bd2defe Link to invocation

tensorrt-cicd · 2026-06-06T22:21:14Z

PR_Github #52512 [ run ] completed with state FAILURE. Commit: bd2defe
/LLM/main/L0_MergeRequest_PR pipeline #41802 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

QiJune · 2026-06-08T05:56:13Z

/bot skip --comment "trivial changes"

…ost-merge cuda_graph and overlap_scheduler are recommended default-on features, and the python_scheduler path is a non-default (P1) feature. The variants that disable these only need post-merge coverage, so move them out of pre_merge to free pre-merge GPU time without dropping coverage. Relocated DeepSeekV3Lite 4-GPU tests (pre_merge -> post_merge, same condition): - l0_gb200_multi_gpus.yml: 13 (nvfp4/bfloat16 cuda_graph=False/overlap_scheduler=False + python_scheduler) - l0_dgx_b200.yml: 4 python_scheduler (the pp4 cuda_graph=False test stays in pre_merge as the only pipeline-parallel smoke for that SKU) - l0_dgx_b300.yml: 1 (nvfp4 cuda_graph=False/overlap_scheduler=False) Also drop duplicate entries in l0_dgx_b200.yml that were listed in both pre_merge and post_merge (kept the appropriate stage for each): - TestNemotronUltraV3::test_accuracy[nvfp4-4] (kept pre-merge) - test_e2e[aggr_upload-super_ad_blackwell-super_ad_ws4_1k1k] (kept post-merge) Pure relocation/dedup: no coverage dropped; pre-merge retains the cuda_graph=True + overlap_scheduler=True configs plus the pp4 smoke. Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>

QiJune · 2026-06-08T06:25:34Z

/bot skip --comment "trivial changes"

tensorrt-cicd · 2026-06-08T06:31:52Z

PR_Github #52677 [ skip ] triggered by Bot. Commit: 2f2ad45 Link to invocation

tensorrt-cicd · 2026-06-08T06:38:03Z

PR_Github #52677 [ skip ] completed with state SUCCESS. Commit: 2f2ad45
Skipping testing for commit 2f2ad45

Link to invocation

sunnyqgg

LGTE

github-actions Bot assigned QiJune Jun 6, 2026

QiJune force-pushed the gb200 branch from b77e9b4 to bd2defe Compare June 6, 2026 07:54

QiJune force-pushed the gb200 branch from bd2defe to 2f2ad45 Compare June 8, 2026 06:25

QiJune requested review from YihuiLu512 and sunnyqgg June 8, 2026 06:26

sunnyqgg approved these changes Jun 8, 2026

View reviewed changes

QiJune merged commit 09c21b6 into NVIDIA:main Jun 8, 2026
7 checks passed

Uh oh!

Conversation

QiJune commented Jun 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Description

Test Coverage

PR Checklist

GitHub Bot Help

Uh oh!

coderabbitai Bot commented Jun 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Possibly related PRs

Suggested reviewers

Uh oh!

QiJune commented Jun 6, 2026

Uh oh!

tensorrt-cicd commented Jun 6, 2026

Uh oh!

tensorrt-cicd commented Jun 6, 2026

Uh oh!

QiJune commented Jun 6, 2026

Uh oh!

tensorrt-cicd commented Jun 6, 2026

Uh oh!

tensorrt-cicd commented Jun 6, 2026

Uh oh!

QiJune commented Jun 8, 2026

Uh oh!

QiJune commented Jun 8, 2026

Uh oh!

tensorrt-cicd commented Jun 8, 2026

Uh oh!

tensorrt-cicd commented Jun 8, 2026

Uh oh!

sunnyqgg left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

QiJune commented Jun 6, 2026 •

edited

Loading

coderabbitai Bot commented Jun 6, 2026 •

edited

Loading