[TRTLLM-13696][test] Part2.4: Migrate CPU only tests - others by tongyuantongyu · Pull Request #17339 · NVIDIA/TensorRT-LLM

tongyuantongyu · 2026-08-06T04:29:27Z

Dev Engineer Review

Migrates CPU-compatible attention, executor, MoE, visual-generation, weight-sharing, API stability, Dynamo, KV-cache, scaffolding, and usage tests to the CPU stage.
Adds pytest.mark.cpu_only at module or class scope.
Adds CPU-safe CUDA capability handling in test_attention_mla.py.
Removes the migrated tests from GPU test lists.
Adds the corresponding suites to l0_cpu.yml.
Test-list paths and YAML changes are consistent. No public API changes are present.
The full CI run was unstable. Investigate the reported failures and retrigger CI with the required approval.

QA Engineer Review

Modified test coverage includes:
- Attention synchronization and MLA capability tests.
- Sparse DSA indexer tests.
- KV-cache executor, scheduler, estimation, budgeting, salting, routing, and rebalancing tests.
- DWDP, MoE host-sharer, and Cute DSL backend tests.
- Visual-generation attention, mapping, multinode, sparse-attention, pipeline, transformer, cache, payload, utility, endpoint, and output tests.
- Weight-sharing, API stability, Dynamo import, scaffolding, and usage tests.
The modified tests are assigned to tests/integration/test_lists/test-db/l0_cpu.yml.
GPU test lists remove the migrated entries.
Verdict: needs follow-up. CBTS coverage data and the failing full-run tests are not available.

Description

Move the rest of test cases don't need GPU to CPU stage.

Test Coverage

The CPU stage.

PR Checklist

Please review the following before submitting your PR:

PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

tongyuantongyu · 2026-08-06T04:29:51Z

/bot run --stage-list "CPU-Generic-x86-1, CPU-Generic-arm-1" --disable-fail-fast

tensorrt-cicd · 2026-08-06T04:36:44Z

PR_Github #64218 [ run ] triggered by Bot. Commit: 4cd04e1 Link to invocation

tensorrt-cicd · 2026-08-06T06:34:34Z

PR_Github #64218 [ run ] completed with state SUCCESS. Commit: 4cd04e1
/LLM/main/L0_MergeRequest_PR pipeline #52125 (Partly Tested) completed with status: 'SUCCESS'

CI Report

Link to invocation

coderabbitai · 2026-08-06T07:24:15Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 9daccf54-bfd6-41cd-8639-af3e5f4dc457

📥 Commits

Reviewing files that changed from the base of the PR and between 9ac759c and 19e3f6b.

📒 Files selected for processing (59)

tests/integration/test_lists/test-db/l0_a10.yml
tests/integration/test_lists/test-db/l0_b200.yml
tests/integration/test_lists/test-db/l0_cpu.yml
tests/integration/test_lists/test-db/l0_dgx_b200.yml
tests/integration/test_lists/test-db/l0_dgx_b300.yml
tests/integration/test_lists/test-db/l0_gb300_multi_gpus.yml
tests/integration/test_lists/test-db/l0_h100.yml
tests/integration/test_lists/test-db/l0_rtx_pro_6000.yml
tests/unittest/_torch/attention/sparse/dsa/test_dsa_indexer.py
tests/unittest/_torch/attention/test_attention_mla.py
tests/unittest/_torch/attention/test_attention_op_sync.py
tests/unittest/_torch/executor/test_dual_pool_kv_cache.py
tests/unittest/_torch/executor/test_kv_cache_budget_split.py
tests/unittest/_torch/executor/test_kv_cache_estimation.py
tests/unittest/_torch/executor/test_kv_cache_v2_multimodal_runs.py
tests/unittest/_torch/executor/test_kv_cache_v2_scheduler.py
tests/unittest/_torch/executor/test_kv_pool_rebalance.py
tests/unittest/_torch/executor/test_kvcache_aware_router.py
tests/unittest/_torch/modules/dwdp/test_dwdp_fixup_moe_backends.py
tests/unittest/_torch/modules/moe/test_cute_dsl_b12x_moe_backend.py
tests/unittest/_torch/modules/test_moe_host_sharer.py
tests/unittest/_torch/visual_gen/multi_gpu/test_attn2d_attention.py
tests/unittest/_torch/visual_gen/multi_gpu/test_visual_gen_mapping.py
tests/unittest/_torch/visual_gen/multi_gpu/test_visual_gen_multinode.py
tests/unittest/_torch/visual_gen/sparse_attention/test_skip_softmax.py
tests/unittest/_torch/visual_gen/test_attention_integration.py
tests/unittest/_torch/visual_gen/test_cache_dit.py
tests/unittest/_torch/visual_gen/test_flux_infer.py
tests/unittest/_torch/visual_gen/test_ltx2_pipeline.py
tests/unittest/_torch/visual_gen/test_ltx2_transformer.py
tests/unittest/_torch/visual_gen/test_qwen_image_pipeline_config.py
tests/unittest/_torch/visual_gen/test_teacache.py
tests/unittest/_torch/visual_gen/test_tensor_payload.py
tests/unittest/_torch/visual_gen/test_trtllm_serve_endpoints.py
tests/unittest/_torch/visual_gen/test_utils.py
tests/unittest/_torch/visual_gen/test_vanilla_key_padding_mask.py
tests/unittest/_torch/visual_gen/test_visual_gen_args.py
tests/unittest/_torch/visual_gen/test_visual_gen_params.py
tests/unittest/_torch/visual_gen/test_visual_gen_utils.py
tests/unittest/_torch/weight_sharing/test_gms_source_identity_gate.py
tests/unittest/_torch/weight_sharing/test_mx_source_identity_gate.py
tests/unittest/_torch/weight_sharing/test_source_identity.py
tests/unittest/api_stability/test_llm_api.py
tests/unittest/dynamo/test_imports.py
tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_manager_v2.py
tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_salting.py
tests/unittest/scaffolding/test_bench.py
tests/unittest/scaffolding/test_mcp_worker.py
tests/unittest/scaffolding/test_parallel_process.py
tests/unittest/scaffolding/test_task_collection.py
tests/unittest/usage/test_collectors.py
tests/unittest/usage/test_config.py
tests/unittest/usage/test_llmapi_config_capture.py
tests/unittest/usage/test_llmapi_config_telemetry_docs.py
tests/unittest/usage/test_opt_out.py
tests/unittest/usage/test_reporter.py
tests/unittest/usage/test_schema.py
tests/unittest/usage/test_transport.py
tests/unittest/visual_gen/test_output.py

💤 Files with no reviewable changes (7)

tests/integration/test_lists/test-db/l0_dgx_b300.yml
tests/integration/test_lists/test-db/l0_rtx_pro_6000.yml
tests/integration/test_lists/test-db/l0_gb300_multi_gpus.yml
tests/integration/test_lists/test-db/l0_b200.yml
tests/integration/test_lists/test-db/l0_h100.yml
tests/integration/test_lists/test-db/l0_dgx_b200.yml
tests/integration/test_lists/test-db/l0_a10.yml

🚧 Files skipped from review as they are similar to previous changes (52)

tests/unittest/scaffolding/test_mcp_worker.py
tests/unittest/_torch/visual_gen/test_teacache.py
tests/unittest/usage/test_opt_out.py
tests/unittest/usage/test_collectors.py
tests/unittest/visual_gen/test_output.py
tests/unittest/usage/test_config.py
tests/unittest/_torch/visual_gen/test_utils.py
tests/unittest/_torch/visual_gen/test_visual_gen_args.py
tests/unittest/_torch/visual_gen/test_visual_gen_params.py
tests/unittest/_torch/visual_gen/sparse_attention/test_skip_softmax.py
tests/unittest/_torch/weight_sharing/test_gms_source_identity_gate.py
tests/unittest/scaffolding/test_task_collection.py
tests/unittest/_torch/visual_gen/multi_gpu/test_attn2d_attention.py
tests/unittest/_torch/visual_gen/test_attention_integration.py
tests/unittest/_torch/executor/test_kv_cache_v2_scheduler.py
tests/unittest/_torch/executor/test_kv_cache_v2_multimodal_runs.py
tests/unittest/api_stability/test_llm_api.py
tests/unittest/_torch/executor/test_kvcache_aware_router.py
tests/unittest/usage/test_llmapi_config_telemetry_docs.py
tests/unittest/_torch/visual_gen/test_qwen_image_pipeline_config.py
tests/unittest/_torch/executor/test_kv_cache_estimation.py
tests/unittest/usage/test_schema.py
tests/unittest/_torch/visual_gen/test_vanilla_key_padding_mask.py
tests/unittest/_torch/attention/sparse/dsa/test_dsa_indexer.py
tests/unittest/usage/test_reporter.py
tests/unittest/usage/test_transport.py
tests/unittest/scaffolding/test_parallel_process.py
tests/unittest/_torch/executor/test_dual_pool_kv_cache.py
tests/unittest/_torch/visual_gen/test_ltx2_transformer.py
tests/unittest/_torch/weight_sharing/test_mx_source_identity_gate.py
tests/unittest/_torch/visual_gen/test_visual_gen_utils.py
tests/unittest/_torch/visual_gen/multi_gpu/test_visual_gen_multinode.py
tests/unittest/_torch/attention/test_attention_mla.py
tests/unittest/usage/test_llmapi_config_capture.py
tests/integration/test_lists/test-db/l0_cpu.yml
tests/unittest/_torch/modules/test_moe_host_sharer.py
tests/unittest/dynamo/test_imports.py
tests/unittest/_torch/visual_gen/multi_gpu/test_visual_gen_mapping.py
tests/unittest/_torch/visual_gen/test_flux_infer.py
tests/unittest/_torch/executor/test_kv_cache_budget_split.py
tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_salting.py
tests/unittest/_torch/executor/test_kv_pool_rebalance.py
tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_manager_v2.py
tests/unittest/_torch/attention/test_attention_op_sync.py
tests/unittest/_torch/visual_gen/test_trtllm_serve_endpoints.py
tests/unittest/_torch/modules/moe/test_cute_dsl_b12x_moe_backend.py
tests/unittest/_torch/modules/dwdp/test_dwdp_fixup_moe_backends.py
tests/unittest/scaffolding/test_bench.py
tests/unittest/_torch/weight_sharing/test_source_identity.py
tests/unittest/_torch/visual_gen/test_tensor_payload.py
tests/unittest/_torch/visual_gen/test_cache_dit.py
tests/unittest/_torch/visual_gen/test_ltx2_pipeline.py

Walkthrough

The pull request classifies compatible tests as CPU-only, makes MLA capability checks CPU-safe, and updates hardware-specific integration test lists. CPU coverage expands, while selected GPU-specific test entries are removed or replaced.

Changes

CPU test execution

Layer / File(s)	Summary
Attention test CPU compatibility `tests/unittest/_torch/attention/...`	Attention tests receive CPU-only markers. MLA tests cache CUDA capability detection and use `(0, 0)` when no GPU is available.
CPU-only test markers `tests/unittest/_torch/executor/...`, `tests/unittest/_torch/modules/...`, `tests/unittest/_torch/visual_gen/...`, `tests/unittest/_torch/weight_sharing/...`, `tests/unittest/api_stability/...`, `tests/unittest/dynamo/...`, `tests/unittest/kv_cache_manager_v2_tests/...`, `tests/unittest/scaffolding/...`, `tests/unittest/usage/...`, `tests/unittest/visual_gen/...`	The listed test modules and classes are marked with `pytest.mark.cpu_only`.
Hardware test-list routing `tests/integration/test_lists/test-db/l0_a10.yml`, `tests/integration/test_lists/test-db/l0_b200.yml`, `tests/integration/test_lists/test-db/l0_cpu.yml`, `tests/integration/test_lists/test-db/l0_dgx_b300.yml`	CPU coverage adds multiple unittest suites. Selected A10 and B200 executor, MoE, and visual-generation entries are removed or replaced.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

NVIDIA/TensorRT-LLM#16498: Both changes configure tests for CPU-only execution and update related test lists.

Suggested labels: ci: full pre-merge approved

Suggested reviewers: qijune, mzweilz, niukuo

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 45.83% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies the ticket, test scope, and migration of remaining CPU-only tests.
Description check	✅ Passed	The description states the migration goal, identifies CPU-stage coverage, and includes the repository checklist.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands.}

yuxianq

Attention part LGTM

tongyuantongyu · 2026-08-06T09:11:22Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-08-06T09:16:49Z

PR_Github #64293 [ run ] triggered by Bot. Commit: c1f6428 Link to invocation

BowenFu

Checked the one thing that can actually go wrong here: all 31 entries removed from the GPU stage lists are covered by l0_cpu.yml at this head — unittest/_torch/executor and unittest/_torch/modules bridge the test_kv_cache_*/test_moe_host_sharer/test_cute_dsl_b12x_moe_backend removals, unittest/usage and unittest/dynamo the rest — so this is a move, not a drop, and nothing falls out of every stage.

Also confirmed the newly-marked scopes are genuinely GPU-free rather than GPU tests hidden behind a marker: the mixed files use narrow class/method markers and leave the real CUDA cases unmarked. The cuda_capability hoist in test_attention_mla.py reads the same get_device_capability() on a GPU box, so tokens_per_block and the FP8 parameterization are unchanged there.

Not blocking, just noting: that hoist caches capability at import time. Fine today since nothing in the file switches device, but it is now an assumption the file depends on.

DomBrown

Approving from API committee perspective

WeiHaocheng

LGTM for scaffolding part.

tensorrt-cicd · 2026-08-06T11:40:25Z

PR_Github #64293 [ run ] completed with state FAILURE. Commit: c1f6428
/LLM/main/L0_MergeRequest_PR pipeline #52192 completed with status: 'UNSTABLE'

CI Report

⚠️ Multi-GPU Label Required:
Multi-GPU tests require the ci: full pre-merge approved label on this PR. Ask a member of NVIDIA/trt-llm-ci-approvers to add the label, then re-trigger CI with the same bot command (no rebase needed).

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

brnguyen2

Approving — the comments below are optional touch-ups, not blockers.

Mechanically clean, and the marker/test-list bookkeeping lines up on the files I spot-checked. I verified that the fully-marked files really are GPU-free (kv-cache estimation/rebalance/dual-pool mock torch.cuda.*, test_visual_gen_multinode.py patches set_device, TestSequenceSharderDistributed uses gloo), and that no GPU list is left with a directory entry that would now collect zero tests — weight_sharing and api_stability both keep unmarked files.

One gap worth closing before merge: three files get pytestmark = pytest.mark.cpu_only but appear in no test list at all, so they still never run — and the marker now also locks them out of any future GPU-stage directory entry. Details inline.

Since GPU stages run with -m "not cpu_only", the marker alone is enough to drop a test from GPU; the list deletions are cosmetic, so the two halves of this change can't drift out of sync in a dangerous direction. Good.

Test-only, no docs or release notes owed.

Signed-off-by: Yuan Tong <13075180+tongyuantongyu@users.noreply.github.com>

coderabbitai · 2026-08-07T10:47:29Z

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

tongyuantongyu · 2026-08-10T03:06:59Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-08-10T03:12:52Z

PR_Github #64937 [ run ] triggered by Bot. Commit: 19e3f6b Link to invocation

tensorrt-cicd · 2026-08-10T05:46:16Z

PR_Github #64937 [ run ] completed with state SUCCESS. Commit: 19e3f6b
/LLM/main/L0_MergeRequest_PR pipeline #52772 completed with status: 'SUCCESS'

CI Report

Link to invocation

github-actions Bot assigned tongyuantongyu Aug 6, 2026

github-actions Bot added the VisualGen label Aug 6, 2026

tongyuantongyu force-pushed the ytong/host-only-rest branch from 4cd04e1 to c1f6428 Compare August 6, 2026 06:06

tongyuantongyu marked this pull request as ready for review August 6, 2026 07:15

tongyuantongyu requested review from a team and WeiHaocheng as code owners August 6, 2026 07:15

tongyuantongyu requested review from Barry-Delaney, BowenFu, QiJune, arysef, joyang-nv, leslie-fang25, liji-nv, nvpohanh, yibinl-nvidia and yuxianq August 6, 2026 07:15

tongyuantongyu requested review from StanleySun639, mzweilz, niukuo and xinhe-nv August 6, 2026 07:16

yuxianq approved these changes Aug 6, 2026

View reviewed changes

nvpohanh approved these changes Aug 6, 2026

View reviewed changes

BowenFu approved these changes Aug 6, 2026

View reviewed changes

DomBrown approved these changes Aug 6, 2026

View reviewed changes

zhenhuaw-me approved these changes Aug 6, 2026

View reviewed changes

WeiHaocheng approved these changes Aug 6, 2026

View reviewed changes

brnguyen2 approved these changes Aug 6, 2026

View reviewed changes

Comment thread tests/integration/test_lists/test-db/l0_cpu.yml

Comment thread tests/unittest/_torch/attention/test_attention_mla.py

coderabbitai Bot mentioned this pull request Aug 7, 2026

[TRTLLM-13696][test] Part2.3: Migrate CPU only tests - auto deploy #17314

Merged

1 task

StanleySun639 approved these changes Aug 7, 2026

View reviewed changes

tongyuantongyu added 3 commits August 7, 2026 18:45

Extract CPU only unittests - other

e3d916c

Signed-off-by: Yuan Tong <13075180+tongyuantongyu@users.noreply.github.com>

Add cpu cases

1310ce1

Signed-off-by: Yuan Tong <13075180+tongyuantongyu@users.noreply.github.com>

remove empty gpu cases

19e3f6b

Signed-off-by: Yuan Tong <13075180+tongyuantongyu@users.noreply.github.com>

tongyuantongyu force-pushed the ytong/host-only-rest branch from c1f6428 to 19e3f6b Compare August 7, 2026 10:47

leslie-fang25 approved these changes Aug 8, 2026

View reviewed changes

Shixiaowei02 added the ci: full pre-merge approved label Aug 10, 2026

QiJune approved these changes Aug 10, 2026

View reviewed changes

Uh oh!

Conversation

tongyuantongyu commented Aug 6, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Dev Engineer Review

QA Engineer Review

Description

Test Coverage

PR Checklist

GitHub Bot Help

Uh oh!

tongyuantongyu commented Aug 6, 2026

Uh oh!

tensorrt-cicd commented Aug 6, 2026

Uh oh!

tensorrt-cicd commented Aug 6, 2026

Uh oh!

coderabbitai Bot commented Aug 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

yuxianq left a comment

Choose a reason for hiding this comment

Uh oh!

tongyuantongyu commented Aug 6, 2026

Uh oh!

tensorrt-cicd commented Aug 6, 2026

Uh oh!

BowenFu left a comment

Choose a reason for hiding this comment

Uh oh!

DomBrown left a comment

Choose a reason for hiding this comment

Uh oh!

WeiHaocheng left a comment

Choose a reason for hiding this comment

Uh oh!

tensorrt-cicd commented Aug 6, 2026

Uh oh!

brnguyen2 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

coderabbitai Bot commented Aug 7, 2026

Uh oh!

tongyuantongyu commented Aug 10, 2026

Uh oh!

tensorrt-cicd commented Aug 10, 2026

Uh oh!

tensorrt-cicd commented Aug 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

13 participants

tongyuantongyu commented Aug 6, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Aug 6, 2026 •

edited

Loading