[TRTLLM-13783][test] Remove TensorRT-backend tests#15810
Conversation
e3b37a8 to
425ca67
Compare
Remove the TensorRT-only test suites as part of deprecating the legacy
TensorRT backend. Scoped to tests/ — no runtime source under tensorrt_llm/
or cpp/ is touched, so this cannot change runtime behavior.
- Delete 143 TensorRT-only test files: tests/unittest/trt/** and the TRT
Builder / runtime.Session / plugin / plugin_gen, engine-cache llmapi,
ModelRunnerCpp, medusa_utils, old runtime.kv_cache_manager, and the
TRT-engine Triton integration tests. Each targets engine build, the
tensorrt_llm.functional graph layer, TRT plugins, or the TRT runtime;
behavior that also matters for PyTorch is already covered under _torch/**.
- Relocate the tests that were living under trt/ but exercise surviving
PyTorch code, instead of deleting them (reformatted to satisfy ruff at
their new, non-excluded paths):
- test_fp8_quantization.py -> unittest/_torch/thop/serial/
- test_mode.py (QuantMode) -> unittest/quantization/
- test_nvfp4_marlin_gemm (torch.ops.trtllm.marlin_nvfp4_gemm, reference
torch.matmul, no TRT engine), rescued from trt/functional/test_fp4_gemm.py
-> unittest/_torch/thop/serial/test_nvfp4_marlin_gemm.py
- Add triton_server/test_triton_llmapi.py — PyTorch-path Triton coverage
that replaces the removed TRT-engine Triton tests.
- Clean now-dangling entries from test-db/l0_*.yml, waives.txt, and
qa/llm_triton_integration.txt, and register unittest/quantization/test_mode.py
in l0_a10.yml (previously not run in CI).
Derived from upstream PR NVIDIA#15810 (e3b37a8), extended with base-divergence
fixes, the relocations/rescue above, and the test_mode CI entry.
Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
|
/bot run --disable-fail-fast |
📝 WalkthroughWalkthroughThis PR removes legacy TensorRT-based integration and unit tests, adds a Triton llmapi integration module, updates test lists and waivers, and adds a new nvfp4 Marlin GEMM unit test with minor quantization formatting changes. ChangesTriton LLM API Integration Migration
Unit Test Cleanup and Additions
Estimated code review effort: 4 (Complex) | ~45 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
@coderabbitai review the pr, if there is missing tensorrt flow tests |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -uo pipefail
echo "=== Remaining files under tests/unittest/trt/ ==="
fd . tests/unittest/trt 2>/dev/null
echo "=== Remaining trt-only test dirs ==="
fd -t d 'trt' tests 2>/dev/null
echo "=== References to build_engines module ==="
rg -n 'build_engines' tests --type=py -g '!*.pyc'
echo "=== References to ModelRunnerCpp in tests ==="
rg -n 'ModelRunnerCpp' tests
echo "=== References to medusa_utils ==="
rg -n 'medusa_utils' tests
echo "=== References to old runtime.kv_cache_manager ==="
rg -n 'kv_cache_manager' tests | grep -v '_torch'
echo "=== dangling references to deleted test names in test-lists ==="
rg -n 'test_gpt_attention|test_bert_attention|test_llm_models|test_llm_multi_gpu|test_model_runner_cpp|test_build_cache|plugin_gen' tests/integration/test_lists
echo "=== check waives.txt for dangling refs to trt tests ==="
rg -n 'unittest/trt/|triton_server/test_triton_rcca|triton_server/test_triton_memleak|triton_server/test_triton_multi_node|test_lora_attention_pytorch_flow_vs_trt|test_lora_plugin_vs_lora_op' tests/integration/test_lists/waives.txt
echo "=== check any remaining .yml referencing triton_server/test_triton_llm.py (old removed file?) ==="
rg -n 'test_triton_llm\.py' tests/integrationLength of output: 53737 🏁 Script executed: #!/bin/bash
set -uo pipefail
echo "=== Context around ModelRunnerCpp usage in test_e2e.py ==="
sed -n '80,120p' tests/integration/defs/test_e2e.py
echo "=== function name using ModelRunnerCpp in test_e2e.py ==="
rg -n 'def test_' tests/integration/defs/test_e2e.py | awk -F: '{print $1":"$2}' | head -1
ast-grep run --pattern 'def $FUNC($$$) { $$$ }' --lang python tests/integration/defs/test_e2e.py 2>/dev/null | head -5
echo "=== Check whisper test wiring in test-lists ==="
rg -n 'test_whisper|validate_whisper' tests/integration/test_lists/test-db/*.yml
echo "=== medusa cpp session entries in l0_gh200.yml ==="
rg -n 'medusa' tests/integration/test_lists/test-db/l0_gh200.yml
echo "=== Check test_medusa.py for TRT engine building (kept file) ==="
fd test_medusa.py tests
rg -n 'ModelRunnerCpp|trtllm-build|engine_dir' tests/integration/defs/examples/test_medusa.py | head -30Length of output: 5272 🏁 Script executed: #!/bin/bash
set -uo pipefail
echo "=== check test_triton.py::test_whisper still present? ==="
rg -n 'def test_whisper' tests/integration/defs/triton_server/test_triton.py
echo "=== full test_triton.py current tests ==="
rg -n 'def test_' tests/integration/defs/triton_server/test_triton.py
echo "=== check test_e2e.py::test_gpt3_175b_1layers_build_only wired in test-lists ==="
rg -n 'test_gpt3_175b_1layers_build_only' tests/integration/test_lists -r
echo "=== check for other trtllm-build usages remaining under tests/integration/defs (examples) ==="
rg -l 'trtllm-build' tests/integration/defs/examples | sortLength of output: 5115
Fully removed ✅
Still present (TensorRT-engine-based flow tests not removed by this PR)
So while this PR removes the pure-unittest TRT coverage ( Would you like me to open a follow-up issue tracking the remaining TensorRT-engine-based integration tests ( (๑•̀ㅁ•́๑)✧ investigating mode engaged |
|
PR_Github #57192 [ run ] triggered by Bot. Commit: |
|
PR_Github #57192 [ run ] completed with state
|
Remove the TensorRT-only test suites as part of deprecating the legacy
TensorRT backend. Scoped to tests/ — no runtime source under tensorrt_llm/
or cpp/ is touched, so this cannot change runtime behavior.
- Delete 143 TensorRT-only test files: tests/unittest/trt/** and the TRT
Builder / runtime.Session / plugin / plugin_gen, engine-cache llmapi,
ModelRunnerCpp, medusa_utils, old runtime.kv_cache_manager, and the
TRT-engine Triton integration tests. Each targets engine build, the
tensorrt_llm.functional graph layer, TRT plugins, or the TRT runtime;
behavior that also matters for PyTorch is already covered under _torch/**.
- Relocate the tests that were living under trt/ but exercise surviving
PyTorch code, instead of deleting them (reformatted to satisfy ruff at
their new, non-excluded paths):
- test_fp8_quantization.py -> unittest/_torch/thop/serial/
- test_mode.py (QuantMode) -> unittest/quantization/
- test_nvfp4_marlin_gemm (torch.ops.trtllm.marlin_nvfp4_gemm, reference
torch.matmul, no TRT engine), rescued from trt/functional/test_fp4_gemm.py
-> unittest/_torch/thop/serial/test_nvfp4_marlin_gemm.py
- Add triton_server/test_triton_llmapi.py — PyTorch-path Triton coverage
that replaces the removed TRT-engine Triton tests.
- Clean now-dangling entries from test-db/l0_*.yml, waives.txt, and
qa/llm_triton_integration.txt, and register unittest/quantization/test_mode.py
in l0_a10.yml (previously not run in CI).
Derived from upstream PR NVIDIA#15810 (e3b37a8), extended with base-divergence
fixes, the relocations/rescue above, and the test_mode CI entry.
Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
425ca67 to
c955d12
Compare
Remove the TensorRT-only test suites as part of deprecating the legacy
TensorRT backend. Scoped to tests/ — no runtime source under tensorrt_llm/
or cpp/ is touched, so this cannot change runtime behavior.
- Delete 143 TensorRT-only test files: tests/unittest/trt/** and the TRT
Builder / runtime.Session / plugin / plugin_gen, engine-cache llmapi,
ModelRunnerCpp, medusa_utils, old runtime.kv_cache_manager, and the
TRT-engine Triton integration tests. Each targets engine build, the
tensorrt_llm.functional graph layer, TRT plugins, or the TRT runtime;
behavior that also matters for PyTorch is already covered under _torch/**.
- Relocate the tests that were living under trt/ but exercise surviving
PyTorch code, instead of deleting them (reformatted to satisfy ruff at
their new, non-excluded paths):
- test_fp8_quantization.py -> unittest/_torch/thop/serial/
- test_mode.py (QuantMode) -> unittest/quantization/
- test_nvfp4_marlin_gemm (torch.ops.trtllm.marlin_nvfp4_gemm, reference
torch.matmul, no TRT engine), rescued from trt/functional/test_fp4_gemm.py
-> unittest/_torch/thop/serial/test_nvfp4_marlin_gemm.py
- Add triton_server/test_triton_llmapi.py — PyTorch-path Triton coverage
that replaces the removed TRT-engine Triton tests.
- Clean now-dangling entries from test-db/l0_*.yml, waives.txt, and
qa/llm_triton_integration.txt, and register unittest/quantization/test_mode.py
in l0_a10.yml (previously not run in CI).
Derived from upstream PR NVIDIA#15810 (e3b37a8), extended with base-divergence
fixes, the relocations/rescue above, and the test_mode CI entry.
Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
c955d12 to
5954f17
Compare
|
/bot run --disable-fail-fast |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/integration/test_lists/waives.txt`:
- Around line 325-326: The waiver entries are being parsed as a single test id
per line, so this combined waiver in the waives list only records one malformed
skip instead of multiple tests. Split the combined entry into separate lines in
the waives file, with one test id or skip entry per line, so each affected test
is actually waived.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: b663f624-6317-4e56-bf62-7fefba8a3cf4
📒 Files selected for processing (162)
tests/integration/defs/llmapi/test_llm_examples.pytests/integration/defs/triton_server/build_engines.pytests/integration/defs/triton_server/test_triton.pytests/integration/defs/triton_server/test_triton_llm.pytests/integration/defs/triton_server/test_triton_llmapi.pytests/integration/defs/triton_server/test_triton_memleak.pytests/integration/defs/triton_server/test_triton_multi_node.pytests/integration/defs/triton_server/test_triton_rcca.pytests/integration/test_lists/qa/llm_triton_integration.txttests/integration/test_lists/test-db/l0_a10.ymltests/integration/test_lists/test-db/l0_a100.ymltests/integration/test_lists/test-db/l0_a30.ymltests/integration/test_lists/test-db/l0_b200.ymltests/integration/test_lists/test-db/l0_dgx_h100.ymltests/integration/test_lists/test-db/l0_dgx_h200.ymltests/integration/test_lists/test-db/l0_gb203.ymltests/integration/test_lists/test-db/l0_gh200.ymltests/integration/test_lists/test-db/l0_h100.ymltests/integration/test_lists/test-db/l0_l40s.ymltests/integration/test_lists/test-db/l0_sanity_check.ymltests/integration/test_lists/waives.txttests/unittest/_torch/modules/tests_lora_modules/test_lora_attention_pytorch_flow_vs_trt.pytests/unittest/_torch/modules/tests_lora_modules/test_lora_plugin_vs_lora_op.pytests/unittest/_torch/thop/serial/test_fp8_quantization.pytests/unittest/_torch/thop/serial/test_nvfp4_marlin_gemm.pytests/unittest/llmapi/test_build_cache.pytests/unittest/llmapi/test_llm_models.pytests/unittest/llmapi/test_llm_multi_gpu.pytests/unittest/others/test_builder.pytests/unittest/others/test_debugging_api.pytests/unittest/others/test_graph_rewriter.pytests/unittest/others/test_kv_cache_manager.pytests/unittest/others/test_layer.pytests/unittest/others/test_model_dtype.pytests/unittest/others/test_module.pytests/unittest/others/test_plugins.pytests/unittest/others/test_precision_control.pytests/unittest/others/test_session.pytests/unittest/quantization/test_mode.pytests/unittest/test_model_runner_cpp.pytests/unittest/tools/plugin_gen/test_core.pytests/unittest/tools/plugin_gen/test_plugin_gen.pytests/unittest/tools/plugin_gen/test_shape_infer.pytests/unittest/trt/__init__.pytests/unittest/trt/attention/hf_dynamic_cache_compat.pytests/unittest/trt/attention/test_bert_attention.pytests/unittest/trt/attention/test_gpt_attention.pytests/unittest/trt/attention/test_gpt_attention_IFB.pytests/unittest/trt/attention/test_gpt_attention_no_cache.pytests/unittest/trt/attention/test_sage_attention.pytests/unittest/trt/functional/__init__.pytests/unittest/trt/functional/test_alibi.pytests/unittest/trt/functional/test_allreduce_norm.pytests/unittest/trt/functional/test_allreduce_prepost_residual_norm.pytests/unittest/trt/functional/test_arange.pytests/unittest/trt/functional/test_argmax.pytests/unittest/trt/functional/test_assertion.pytests/unittest/trt/functional/test_avg_pool2d.pytests/unittest/trt/functional/test_cast.pytests/unittest/trt/functional/test_conv2d.pytests/unittest/trt/functional/test_conv3d.pytests/unittest/trt/functional/test_cos.pytests/unittest/trt/functional/test_cumsum.pytests/unittest/trt/functional/test_dora.pytests/unittest/trt/functional/test_einsum.pytests/unittest/trt/functional/test_embedding_single_gpu.pytests/unittest/trt/functional/test_exp.pytests/unittest/trt/functional/test_expand.pytests/unittest/trt/functional/test_flatten.pytests/unittest/trt/functional/test_flip.pytests/unittest/trt/functional/test_fp4_gemm.pytests/unittest/trt/functional/test_fp4_gemm_ootb.pytests/unittest/trt/functional/test_gather.pytests/unittest/trt/functional/test_gather_nd.pytests/unittest/trt/functional/test_geglu.pytests/unittest/trt/functional/test_gelu.pytests/unittest/trt/functional/test_gemm_swiglu.pytests/unittest/trt/functional/test_group_norm.pytests/unittest/trt/functional/test_identity.pytests/unittest/trt/functional/test_index_select.pytests/unittest/trt/functional/test_interpolate.pytests/unittest/trt/functional/test_logsoftmax.pytests/unittest/trt/functional/test_lora.pytests/unittest/trt/functional/test_low_latency_gemm.pytests/unittest/trt/functional/test_mamba_conv1d.pytests/unittest/trt/functional/test_masked_scatter.pytests/unittest/trt/functional/test_masked_select.pytests/unittest/trt/functional/test_matmul.pytests/unittest/trt/functional/test_meshgrid2d.pytests/unittest/trt/functional/test_moe.pytests/unittest/trt/functional/test_nccl.pytests/unittest/trt/functional/test_nonzero.pytests/unittest/trt/functional/test_outer.pytests/unittest/trt/functional/test_pad.pytests/unittest/trt/functional/test_permute.pytests/unittest/trt/functional/test_pp_reduce_scatter.pytests/unittest/trt/functional/test_quant.pytests/unittest/trt/functional/test_rearrange.pytests/unittest/trt/functional/test_repeat.pytests/unittest/trt/functional/test_repeat_interleave.pytests/unittest/trt/functional/test_rg_lru.pytests/unittest/trt/functional/test_sample.pytests/unittest/trt/functional/test_scatter.pytests/unittest/trt/functional/test_scatter_nd.pytests/unittest/trt/functional/test_select.pytests/unittest/trt/functional/test_selective_scan.pytests/unittest/trt/functional/test_sigmoid.pytests/unittest/trt/functional/test_silu.pytests/unittest/trt/functional/test_sin.pytests/unittest/trt/functional/test_slice.pytests/unittest/trt/functional/test_softplus.pytests/unittest/trt/functional/test_split.pytests/unittest/trt/functional/test_squeeze.pytests/unittest/trt/functional/test_swiglu.pytests/unittest/trt/functional/test_topk.pytests/unittest/trt/functional/test_transpose.pytests/unittest/trt/functional/test_unbind.pytests/unittest/trt/functional/test_unsqueeze.pytests/unittest/trt/functional/test_view.pytests/unittest/trt/functional/test_where.pytests/unittest/trt/model/__init__.pytests/unittest/trt/model/eagle/test_decode_draft_tokens_plugin.pytests/unittest/trt/model/eagle/test_prepare_drafter_inputs_plugin.pytests/unittest/trt/model/eagle/test_sample_accept_draft_tokens_plugin.pytests/unittest/trt/model/redrafter/test_beams2tree.pytests/unittest/trt/model/redrafter/test_draft_token.pytests/unittest/trt/model/redrafter/test_draft_token_indices.pytests/unittest/trt/model/redrafter/test_gather_beams.pytests/unittest/trt/model/redrafter/test_mask.pytests/unittest/trt/model/redrafter/test_packed_position_ids.pytests/unittest/trt/model/redrafter/test_prefix_match_indices.pytests/unittest/trt/model/redrafter/test_prepare_input.pytests/unittest/trt/model/redrafter/test_process_logits.pytests/unittest/trt/model/redrafter/test_top1.pytests/unittest/trt/model/redrafter/test_unpack_gen_data.pytests/unittest/trt/model/redrafter/test_validate.pytests/unittest/trt/model/test_gpt.pytests/unittest/trt/model/test_gpt_e2e.pytests/unittest/trt/model/test_llama.pytests/unittest/trt/model/test_mamba.pytests/unittest/trt/model/test_mistral.pytests/unittest/trt/model/test_nemotron_nas.pytests/unittest/trt/model/test_phi.pytests/unittest/trt/model/test_unet.pytests/unittest/trt/model_api/profile_utils.pytests/unittest/trt/model_api/test_model_api_multi_gpu.pytests/unittest/trt/python_plugin/plugin_wrapper_utils.pytests/unittest/trt/python_plugin/test_plugin_wrapper.pytests/unittest/trt/quantization/__init__.pytests/unittest/trt/quantization/_utils.pytests/unittest/trt/quantization/test_fp8_rowwise_gemm.pytests/unittest/trt/quantization/test_functional.pytests/unittest/trt/quantization/test_moe_weight_only_quant_matmul.pytests/unittest/trt/quantization/test_qserve_gemm.pytests/unittest/trt/quantization/test_quant.pytests/unittest/trt/quantization/test_quant_layer.pytests/unittest/trt/quantization/test_smooth_quant_gemm.pytests/unittest/trt/quantization/test_smooth_quant_layer_norm.pytests/unittest/trt/quantization/test_smooth_quant_rms_norm.pytests/unittest/trt/quantization/test_weight_only_groupwise_quant_matmul.pytests/unittest/trt/quantization/test_weight_only_quant_matmul.pytests/unittest/utils/test_medusa_utils.py
💤 Files with no reviewable changes (85)
- tests/unittest/trt/functional/test_gelu.py
- tests/unittest/others/test_plugins.py
- tests/unittest/trt/functional/test_embedding_single_gpu.py
- tests/unittest/others/test_kv_cache_manager.py
- tests/unittest/trt/functional/test_outer.py
- tests/unittest/trt/functional/test_logsoftmax.py
- tests/unittest/trt/functional/test_cast.py
- tests/unittest/trt/functional/test_dora.py
- tests/unittest/trt/functional/test_matmul.py
- tests/unittest/trt/functional/test_pad.py
- tests/unittest/trt/functional/test_cos.py
- tests/unittest/others/test_builder.py
- tests/unittest/trt/functional/test_geglu.py
- tests/integration/test_lists/test-db/l0_gb203.yml
- tests/unittest/trt/functional/test_flip.py
- tests/unittest/trt/functional/test_index_select.py
- tests/integration/defs/triton_server/test_triton_memleak.py
- tests/unittest/trt/functional/test_expand.py
- tests/unittest/others/test_model_dtype.py
- tests/unittest/_torch/modules/tests_lora_modules/test_lora_plugin_vs_lora_op.py
- tests/unittest/others/test_debugging_api.py
- tests/unittest/trt/attention/hf_dynamic_cache_compat.py
- tests/integration/defs/llmapi/test_llm_examples.py
- tests/unittest/trt/functional/test_gather_nd.py
- tests/unittest/others/test_module.py
- tests/unittest/tools/plugin_gen/test_shape_infer.py
- tests/unittest/trt/functional/test_gather.py
- tests/unittest/trt/functional/test_masked_scatter.py
- tests/unittest/trt/functional/test_nonzero.py
- tests/unittest/trt/functional/test_interpolate.py
- tests/unittest/tools/plugin_gen/test_core.py
- tests/unittest/trt/functional/test_group_norm.py
- tests/unittest/trt/functional/test_identity.py
- tests/unittest/trt/functional/test_gemm_swiglu.py
- tests/integration/test_lists/test-db/l0_b200.yml
- tests/unittest/others/test_session.py
- tests/unittest/trt/functional/test_argmax.py
- tests/unittest/others/test_precision_control.py
- tests/unittest/trt/functional/test_cumsum.py
- tests/unittest/trt/functional/test_flatten.py
- tests/unittest/trt/attention/test_bert_attention.py
- tests/unittest/trt/functional/test_lora.py
- tests/unittest/trt/functional/test_avg_pool2d.py
- tests/unittest/trt/functional/test_meshgrid2d.py
- tests/unittest/trt/functional/test_exp.py
- tests/unittest/trt/functional/test_low_latency_gemm.py
- tests/unittest/trt/functional/test_moe.py
- tests/unittest/trt/functional/test_conv2d.py
- tests/unittest/trt/functional/test_arange.py
- tests/unittest/_torch/modules/tests_lora_modules/test_lora_attention_pytorch_flow_vs_trt.py
- tests/unittest/trt/functional/test_assertion.py
- tests/unittest/tools/plugin_gen/test_plugin_gen.py
- tests/unittest/trt/functional/test_fp4_gemm_ootb.py
- tests/unittest/trt/functional/test_allreduce_prepost_residual_norm.py
- tests/unittest/test_model_runner_cpp.py
- tests/unittest/trt/attention/test_gpt_attention_no_cache.py
- tests/unittest/trt/functional/test_conv3d.py
- tests/unittest/trt/functional/test_mamba_conv1d.py
- tests/integration/defs/triton_server/test_triton_multi_node.py
- tests/integration/test_lists/test-db/l0_gh200.yml
- tests/unittest/llmapi/test_build_cache.py
- tests/integration/test_lists/test-db/l0_sanity_check.yml
- tests/unittest/trt/functional/test_fp4_gemm.py
- tests/unittest/trt/functional/test_permute.py
- tests/unittest/trt/functional/test_alibi.py
- tests/integration/test_lists/test-db/l0_l40s.yml
- tests/unittest/llmapi/test_llm_models.py
- tests/integration/defs/triton_server/test_triton.py
- tests/integration/test_lists/test-db/l0_a30.yml
- tests/unittest/trt/functional/test_nccl.py
- tests/unittest/llmapi/test_llm_multi_gpu.py
- tests/unittest/trt/functional/test_einsum.py
- tests/unittest/trt/functional/test_allreduce_norm.py
- tests/unittest/trt/attention/test_gpt_attention_IFB.py
- tests/unittest/others/test_layer.py
- tests/integration/test_lists/test-db/l0_h100.yml
- tests/integration/defs/triton_server/test_triton_rcca.py
- tests/unittest/trt/attention/test_sage_attention.py
- tests/integration/test_lists/test-db/l0_dgx_h100.yml
- tests/integration/defs/triton_server/build_engines.py
- tests/unittest/trt/attention/test_gpt_attention.py
- tests/integration/test_lists/test-db/l0_a100.yml
- tests/integration/test_lists/test-db/l0_dgx_h200.yml
- tests/unittest/trt/functional/test_masked_select.py
- tests/unittest/others/test_graph_rewriter.py
|
PR_Github #57336 [ run ] triggered by Bot. Commit: |
5954f17 to
ba4e584
Compare
Remove the TensorRT-only test suites as part of deprecating the legacy
TensorRT backend. Scoped to tests/ — no runtime source under tensorrt_llm/
or cpp/ is touched, so this cannot change runtime behavior.
- Delete 143 TensorRT-only test files: tests/unittest/trt/** and the TRT
Builder / runtime.Session / plugin / plugin_gen, engine-cache llmapi,
ModelRunnerCpp, medusa_utils, old runtime.kv_cache_manager, and the
TRT-engine Triton integration tests. Each targets engine build, the
tensorrt_llm.functional graph layer, TRT plugins, or the TRT runtime;
behavior that also matters for PyTorch is already covered under _torch/**.
- Relocate the tests that were living under trt/ but exercise surviving
PyTorch code, instead of deleting them (reformatted to satisfy ruff at
their new, non-excluded paths):
- test_fp8_quantization.py -> unittest/_torch/thop/serial/
- test_mode.py (QuantMode) -> unittest/quantization/
- test_nvfp4_marlin_gemm (torch.ops.trtllm.marlin_nvfp4_gemm, reference
torch.matmul, no TRT engine), rescued from trt/functional/test_fp4_gemm.py
-> unittest/_torch/thop/serial/test_nvfp4_marlin_gemm.py
- Add triton_server/test_triton_llmapi.py — PyTorch-path Triton coverage
that replaces the removed TRT-engine Triton tests.
- Clean now-dangling entries from test-db/l0_*.yml, waives.txt, and
qa/llm_triton_integration.txt, and register unittest/quantization/test_mode.py
in l0_a10.yml (previously not run in CI).
Derived from upstream PR NVIDIA#15810 (e3b37a8), extended with base-divergence
fixes, the relocations/rescue above, and the test_mode CI entry.
Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
ba4e584 to
fd7467e
Compare
|
/bot run --add-multi-gpu-test --disable-fail-fast |
|
PR_Github #57360 [ run ] triggered by Bot. Commit: |
|
PR_Github #57336 [ run ] completed with state |
Remove the TensorRT-only test suites as part of deprecating the legacy
TensorRT backend. Scoped to tests/ — no runtime source under tensorrt_llm/
or cpp/ is touched, so this cannot change runtime behavior.
- Delete 143 TensorRT-only test files: tests/unittest/trt/** and the TRT
Builder / runtime.Session / plugin / plugin_gen, engine-cache llmapi,
ModelRunnerCpp, medusa_utils, old runtime.kv_cache_manager, and the
TRT-engine Triton integration tests. Each targets engine build, the
tensorrt_llm.functional graph layer, TRT plugins, or the TRT runtime;
behavior that also matters for PyTorch is already covered under _torch/**.
- Relocate the tests that were living under trt/ but exercise surviving
PyTorch code, instead of deleting them (reformatted to satisfy ruff at
their new, non-excluded paths):
- test_fp8_quantization.py -> unittest/_torch/thop/serial/
- test_mode.py (QuantMode) -> unittest/quantization/
- test_nvfp4_marlin_gemm (torch.ops.trtllm.marlin_nvfp4_gemm, reference
torch.matmul, no TRT engine), rescued from trt/functional/test_fp4_gemm.py
-> unittest/_torch/thop/serial/test_nvfp4_marlin_gemm.py
- Add triton_server/test_triton_llmapi.py — PyTorch-path Triton coverage
that replaces the removed TRT-engine Triton tests.
- Clean now-dangling entries from test-db/l0_*.yml, waives.txt, and
qa/llm_triton_integration.txt, and register unittest/quantization/test_mode.py
in l0_a10.yml (previously not run in CI).
Derived from upstream PR NVIDIA#15810 (e3b37a8), extended with base-divergence
fixes, the relocations/rescue above, and the test_mode CI entry.
Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
07fb0bb to
970f453
Compare
|
/bot run --add-multi-gpu-test --disable-fail-fast |
|
PR_Github #57665 [ run ] triggered by Bot. Commit: |
|
PR_Github #57593 [ run ] completed with state |
|
PR_Github #57665 [ run ] completed with state
|
Remove the TensorRT-only test suites as part of deprecating the legacy
TensorRT backend. Scoped to tests/ — no runtime source under tensorrt_llm/
or cpp/ is touched, so this cannot change runtime behavior.
- Delete 143 TensorRT-only test files: tests/unittest/trt/** and the TRT
Builder / runtime.Session / plugin / plugin_gen, engine-cache llmapi,
ModelRunnerCpp, medusa_utils, old runtime.kv_cache_manager, and the
TRT-engine Triton integration tests. Each targets engine build, the
tensorrt_llm.functional graph layer, TRT plugins, or the TRT runtime;
behavior that also matters for PyTorch is already covered under _torch/**.
- Relocate the tests that were living under trt/ but exercise surviving
PyTorch code, instead of deleting them (reformatted to satisfy ruff at
their new, non-excluded paths):
- test_fp8_quantization.py -> unittest/_torch/thop/serial/
- test_mode.py (QuantMode) -> unittest/quantization/
- test_nvfp4_marlin_gemm (torch.ops.trtllm.marlin_nvfp4_gemm, reference
torch.matmul, no TRT engine), rescued from trt/functional/test_fp4_gemm.py
-> unittest/_torch/thop/serial/test_nvfp4_marlin_gemm.py
- Add triton_server/test_triton_llmapi.py — PyTorch-path Triton coverage
that replaces the removed TRT-engine Triton tests.
- Clean now-dangling entries from test-db/l0_*.yml, waives.txt, and
qa/llm_triton_integration.txt, and register unittest/quantization/test_mode.py
in l0_a10.yml (previously not run in CI).
Derived from upstream PR NVIDIA#15810 (e3b37a8), extended with base-divergence
fixes, the relocations/rescue above, and the test_mode CI entry.
Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
Follow-up to NVIDIA#15763 ([TRTLLM-13781] Remove legacy TensorRT examples), which deleted the TRT example scripts and waived their now-unrunnable tests with the note "tests to be removed in follow-up PR". This is that follow-up. Remove the waived test entries from the test-db/*.yml schedule lists and drop their matching SKIP lines from waives.txt. Covers examples/ (gpt, enc_dec, multimodal, whisper, redrafter, medusa, draft_target_model, eagle, commandr, llama, qwen, phi, mamba, granite, bert), accuracy/ test_cli_flow.py, and the TRT-engine cpp/ tests (test_e2e.py, test_multi_gpu.py executor/disagg cases). Also drop the now-empty 1-GPU tensorrt condition block in l0_a100.yml so trt-test-db does not choke on a null tests list, and remove orphaned waives whose only test entry is now gone: the GH200/sm100 video-neva platform waives, and the sm100 "Disable for Blackwell" combined-job waive (its multi-target l0_a100 tensorrt job was removed with the TRT tests). Also unschedule the legacy TensorRT-engine Triton tests in triton_server/test_triton.py (test_gpt, test_gptj, test_mistral*, test_whisper, test_medusa, test_eagle, test_llava*, test_mllama, test_qwen2_vl, test_t5_ib, test_gpt_ib*, etc.). Each builds a TRT engine via triton_server/build_model.sh, which pushd's into the examples/models/** dirs removed above and fails once they are gone ("pushd: examples/models/contrib/gptj: No such file or directory"). Drop their entries from l0_a30.yml, l0_a100.yml and l0_b200.yml (removing the now-empty a100/b200 triton post_merge blocks so trt-test-db does not choke on a null tests list) and their orphaned waives.txt lines. The self-contained Triton unit tests that build no engine are kept (test_cpp_unit_tests, test_python_bls_unit_tests, test_python_preproc_unit_tests, test_python_multimodal_encoders_unit_tests, test_fill_template, test_llmapi_unit_tests); PyTorch-path Triton coverage lives in test_triton_llmapi.py. Restore six shared multi-GPU C++ unit tests that NVIDIA#15763 over-waived by bulk-waiving all of cpp/test_multi_gpu.py: test_mpi_utils, test_fused_gemm_allreduce, three test_cache_transceiver params, and test_user_buffer. These run pure gtest binaries from tests/unit_tests/multi_gpu (mpiUtilsTest, gemmAllReduceTest, cacheTransceiverTest, userBufferTest) with synthetic inputs and need no example model or TRT engine; they were running before NVIDIA#15763. They are re-added as active in l0_dgx_h100.yml. test_cache_transceiver mooncake keeps its real nvbugs/5838199 waive. The test .py files are left in place — they still hold surviving PyTorch-path cases (e.g. test_gpt_oss_20b_lora_torch). The one whole-file waive (unittest/bindings/test_executor_bindings.py) is kept: every test there builds a TRT engine via trtllm.Executor, and its removal belongs with the C++ Executor-bindings removal, not this test-list cleanup. Scoped to tests/ — no runtime source touched. Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
The removal of legacy TensorRT-backend Triton tests emptied the triton test blocks in l0_a100, l0_b200, and l0_dgx_h200, so the corresponding post-merge stages fail with 'no tests to run'. Remove those stage definitions. The A30 Triton stages keep their remaining tests and stay. Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
970f453 to
76a74b7
Compare
|
/bot skip --comment "the last CI passed with two time-out runs, and these two time-out runs passed previously and not related with my changes" |
|
PR_Github #57769 [ skip ] triggered by Bot. Commit: |
|
PR_Github #57769 [ skip ] completed with state |
With the TensorRT-engine Triton backend and its test_triton.py gone, the triton_server/ test infra had accumulated dead code that only the removed tests used (orphaned by NVIDIA#15810 and the original vendoring): - Delete local_venv.py and runner_interface.py: dead duplicate copies of the parent defs/local_venv.py + defs/runner_interface.py (the active path, conftest.py -> defs.local_venv, uses the parent versions). Zero importers. - Trim common.py to the three helpers the surviving test_triton_llmapi.py actually imports (check_server_ready, prepare_llmapi_model_repo, set_llmapi_decoupled_mode). Removed ~680 lines of unreachable code: the module-level cuda-python install + getSMVersion + unused skip_pre_ada marker, and the ~22 helpers that drove the now-deleted templates / C++ backend (prepare_ib/multimodal/disaggregated_*, run_cpp_backend_tests, check_server_metrics, modify_*_config_pbtxt, etc.). common.py is imported only by test_triton_llmapi.py, and the three kept functions are fully self-contained (os/time + check_call/check_output/print_info). check_test_list.py --validate stays green. Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
With the TensorRT-engine Triton backend and its test_triton.py gone, the triton_server/ test infra had accumulated dead code that only the removed tests used (orphaned by NVIDIA#15810 and the original vendoring): - Delete local_venv.py and runner_interface.py: dead duplicate copies of the parent defs/local_venv.py + defs/runner_interface.py (the active path, conftest.py -> defs.local_venv, uses the parent versions). Zero importers. - Trim common.py to the three helpers the surviving test_triton_llmapi.py actually imports (check_server_ready, prepare_llmapi_model_repo, set_llmapi_decoupled_mode). Removed ~680 lines of unreachable code: the module-level cuda-python install + getSMVersion + unused skip_pre_ada marker, and the ~22 helpers that drove the now-deleted templates / C++ backend (prepare_ib/multimodal/disaggregated_*, run_cpp_backend_tests, check_server_metrics, modify_*_config_pbtxt, etc.). common.py is imported only by test_triton_llmapi.py, and the three kept functions are fully self-contained (os/time + check_call/check_output/print_info). check_test_list.py --validate stays green. Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
With the TensorRT-engine Triton backend and its test_triton.py gone, the triton_server/ test infra had accumulated dead code that only the removed tests used (orphaned by NVIDIA#15810 and the original vendoring): - Delete local_venv.py and runner_interface.py: dead duplicate copies of the parent defs/local_venv.py + defs/runner_interface.py (the active path, conftest.py -> defs.local_venv, uses the parent versions). Zero importers. - Trim common.py to the three helpers the surviving test_triton_llmapi.py actually imports (check_server_ready, prepare_llmapi_model_repo, set_llmapi_decoupled_mode). Removed ~680 lines of unreachable code: the module-level cuda-python install + getSMVersion + unused skip_pre_ada marker, and the ~22 helpers that drove the now-deleted templates / C++ backend (prepare_ib/multimodal/disaggregated_*, run_cpp_backend_tests, check_server_metrics, modify_*_config_pbtxt, etc.). common.py is imported only by test_triton_llmapi.py, and the three kept functions are fully self-contained (os/time + check_call/check_output/print_info). check_test_list.py --validate stays green. Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
With the TensorRT-engine Triton backend and its test_triton.py gone, the triton_server/ test infra had accumulated dead code that only the removed tests used (orphaned by NVIDIA#15810 and the original vendoring): - Delete local_venv.py and runner_interface.py: dead duplicate copies of the parent defs/local_venv.py + defs/runner_interface.py (the active path, conftest.py -> defs.local_venv, uses the parent versions). Zero importers. - Trim common.py to the three helpers the surviving test_triton_llmapi.py actually imports (check_server_ready, prepare_llmapi_model_repo, set_llmapi_decoupled_mode). Removed ~680 lines of unreachable code: the module-level cuda-python install + getSMVersion + unused skip_pre_ada marker, and the ~22 helpers that drove the now-deleted templates / C++ backend (prepare_ib/multimodal/disaggregated_*, run_cpp_backend_tests, check_server_metrics, modify_*_config_pbtxt, etc.). common.py is imported only by test_triton_llmapi.py, and the three kept functions are fully self-contained (os/time + check_call/check_output/print_info). check_test_list.py --validate stays green. Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
PR3.1 test changes — per-file record
Commit
425ca67b0a·[TRTLLM-13783][test] Remove TensorRT-backend testsBase:
main@e5a05b25be·161 files, +559 / −40,887Every change below is under
tests/; no runtime source (tensorrt_llm/**,cpp/**)is touched. Totals: 143 removed · 2 relocated · 2 added · 14 modified.
Legend: Removed = deleted; Relocated = git-renamed to a surviving location (kept
because it tests PyTorch code); Added = new file; Modified = edited in place.
Added (2)
tests/unittest/_torch/thop/serial/test_nvfp4_marlin_gemm.pytrt/functional/test_fp4_gemm.py::test_nvfp4_marlin_gemm. Tests the live PyTorch optorch.ops.trtllm.marlin_nvfp4_gemm(+gptq_marlin_repack,block_scale_interleave_reverse,marlin_utils) against atorch.matmulreference — no TRT engine. Had no surviving coverage elsewhere, so it is re-homed rather than lost. TRT-only imports dropped; reference kept astorch.matmul.tests/integration/defs/triton_server/test_triton_llmapi.pytest_llmapi_backendTP1/TP4,test_llmapi_lora,test_llmapi_backend_multi_instance) that replaces the removed TRT-engine Triton harness (test_triton_llm.pyetc.).Relocated (2) — PyTorch tests kept out of the cull
Both lived under
trt/quantization/but exercise surviving PyTorch code, so they aregit-renamed (not deleted). Reformatted on move because their new paths are not in
pyproject's ruff legacy-exclude (
tests/unittest/trt/*was), soruff-formatnow applies.trt/quantization/test_fp8_quantization.py→_torch/thop/serial/test_fp8_quantization.py(R077)torch.ops.{tensorrt_llm,trtllm}.quantize_e4m3_*+_torch.autotunertrt/quantization/test_mode.py→quantization/test_mode.py(R095)QuantModeconfig enumModified (14)
tests/integration/test_lists/test-db/l0_a10.ymlunittest/quantization/test_mode.pytest_mode(it had no CI entry before or after the move — closes a pre-existing gap)tests/integration/test_lists/test-db/l0_a100.ymltests/integration/test_lists/test-db/l0_a30.ymltests/integration/test_lists/test-db/l0_b200.ymltests/integration/test_lists/test-db/l0_dgx_h200.ymltests/integration/test_lists/test-db/l0_gb203.ymltests/integration/test_lists/test-db/l0_gh200.ymltests/integration/test_lists/test-db/l0_h100.ymltests/integration/test_lists/test-db/l0_l40s.ymltests/integration/test_lists/test-db/l0_sanity_check.ymltests/integration/test_lists/waives.txttests/integration/test_lists/qa/llm_triton_integration.txttest_triton_llmapi.pycasestests/integration/defs/llmapi/test_llm_examples.pytest_llmapi_tensorrt_enginetests/integration/defs/triton_server/test_triton.pyRemoved (143)
Shared rationale: each targets TensorRT-backend code only — engine build, the
tensorrt_llm.functionalgraph layer, TRT plugins, or the TRT runtime. None test thePyTorch backend. Where the behavior also matters for PyTorch, it is already covered by a
surviving
_torch/**suite (noted per group).trt/functional/— 70 (69 op tests +__init__.py)Reason: each builds a TRT
INetworkDefinition(net_guard/create_session) around atensorrt_llm.functionalop and checks the engine output. The op exists only for TRTgraph construction — there is no PyTorch equivalent to re-cover. (The one live-op method,
test_nvfp4_marlin_gemm, was rescued to_torch/thop/serial/— see Added.)trt/model/+trt/model_api/— 26Reason: build a
tensorrt_llm.modelsPretrainedModel → TRT engine → run. Covered bytests/unittest/_torch/modeling/. Theeagle/redrafterfiles test TRT plugins forthose speculative paths (
_torch/speculative/covers the PyTorch flow).trt/model/__init__.pytrt/model/eagle/test_{decode_draft_tokens,prepare_drafter_inputs,sample_accept_draft_tokens}_plugin.py(3)trt/model/redrafter/test_{beams2tree,draft_token,draft_token_indices,gather_beams,mask,packed_position_ids,prefix_match_indices,prepare_input,process_logits,top1,unpack_gen_data,validate}.py(12)trt/model/test_{gpt,gpt_e2e,llama,mamba,mistral,nemotron_nas,phi,unet}.py(8)trt/model_api/profile_utils.py,trt/model_api/test_model_api_multi_gpu.py(2)trt/quantization/— 13Reason: TRT quant GEMM / plugin engine tests + helpers. PyTorch quant is covered by
_torch/thop/and_torch/modules/. (The two non-TRT files here —test_fp8_quantizationand
test_mode— were relocated, not deleted.)trt/attention/— 6Reason: TRT gpt/bert attention plugin engine tests. PyTorch attention is covered by
_torch/attention/.trt/python_plugin/— 2 ·trt/__init__.py— 1Reason: TRT Python-plugin wrapper tests + helper (
plugin_wrapper_utils.py,test_plugin_wrapper.py) and thetrt/package marker. No PyTorch equivalent.unittest/others/— 10Reason: exercise TRT-only machinery. No PyTorch equivalent except KV cache (covered by
kv_cache_manager_v2_tests/).test_builder.pyBuildertest_debugging_api.pytest_graph_rewriter.pytest_kv_cache_manager.pyruntime.kv_cache_manager(TRTgeneration.py) — PyTorch KV cache iskv_cache_manager_v2_tests/test_layer.pytensorrt_llm.layersgraph constructiontest_model_dtype.pytest_module.pyModulebase classtest_plugins.pytest_precision_control.pytest_session.pyruntime.Sessionunittest/llmapi/— 3test_build_cache.pytest_llm_models.py_tensorrt_engine.LLMengine builds per modeltest_llm_multi_gpu.pyBuildConfig-driven multi-GPU engine buildsunittest/tools/plugin_gen/— 3Reason: the TRT plugin-generator tool (
test_core.py,test_plugin_gen.py,test_shape_infer.py). No PyTorch equivalent.unittest/_torch/modules/tests_lora_modules/— 2Reason: these validate surviving PyTorch LoRA code but use a built TRT engine as the
numeric reference, so they are structurally TRT-coupled and cannot run once the backend is
gone. Kept deleted; the numeric coverage should be re-added against a pure-
torchreferencein a follow-up.
test_lora_plugin_vs_lora_op.pytorch.ops.trtllm.lora_grouped_gemmnumericslora_plugintest_lora_attention_pytorch_flow_vs_trt.py_torchLlamaAttention+ LoRA +KVCacheManagerflowintegration/defs/triton_server/— 5Reason: TRT-engine Triton harness (
trtllm-build+ C++ Triton backend). Replaced by theadded
test_triton_llmapi.py.Top-level — 2
unittest/test_model_runner_cpp.pyModelRunnerCppover a TRT engine — PyTorch path is_torch/pyexecutorunittest/utils/test_medusa_utils.pyruntime.medusa_utils(TRT) — PyTorch Medusa is_torch/speculative/Verification
test_nvfp4_marlin_gemm,test_fp8_quantization,test_mode);test_triton_llmapi.pyis collection-clean(executes in the Triton integration CI stage).
scripts/check_test_list.py --validatepasses — no test-list entry references a deletedtest; the new
test_modeentry resolves.unittest/_torch/thop/serialdir entry(H100),
test_triton_llmapiviaqa/llm_triton_integration.txt,test_modevial0_a10.yml.PR3.2 waived-examples cleanup — per-file record
Commit
bdb9795a55·[TRTLLM-13987][test] Remove waived legacy TensorRT examples testsBase:
f33293814e([TRTLLM-13783][test] Remove TensorRT-backend tests)Scope: 10 files, −303 lines, 0 additions. Every change is under
tests/integration/test_lists/; no runtime source and no test.pyfile is touched.Why this commit exists
PR #15763 (
[TRTLLM-13781] Remove legacy TensorRT examples) deleted the TensorRTexample scripts but could not delete the tests that drove them in the same PR, so it
waived every one of those tests with the marker:
That produced 140 dead (skipped) test entries. This commit is that follow-up: it removes
the waived entries from the CI schedule lists and their waive lines, so the dead
entries stop being carried. It was prompted by review feedback (xinhe-nv + an automated
reviewer) asking whether the remaining TRT-flow tests were tracked.
The core invariant
A test runs iff it appears in a
test-db/*.yml(orqa/) schedule list; awaives.txtline only forces
SKIP. So fully removing a waived test requires editing both:validate-test-listspre-commit hook fails ("WAIVE NOT IN ACTIVE TEST LISTS") and, if still scheduled, it runs and fails.test_executor_bindings.pyis the one case where "both" is impossible (see below), so it isintentionally left waived.
Counts at a glance
test_executor_bindings.py)waives.txtl0_a100.yml)Modified files (10)
Schedule lists —
test-db/*.yml(9 files, −162 lines total)Each removed line is a schedule entry whose test is in the TRTLLM-13781 waived batch. The
matching waive line is removed from
waives.txtin the same commit. Grouped by the testfile each removed entry belongs to:
l0_a10.ymlcpp/test_e2e.py×3,examples/test_bert.py×1,examples/test_whisper.py×1l0_a100.ymlexamples/test_whisper.py×4 + 13-line emptytensorrtcondition block (pruned)l0_a30.ymlaccuracy/test_cli_flow.py×7,cpp/test_e2e.py×2,examples/test_commandr.py×2,examples/test_draft_target_model.py×4,examples/test_enc_dec.py×12,examples/test_gpt.py×8,examples/test_medusa.py×1,examples/test_multimodal.py×2,examples/test_redrafter.py×1l0_b200.ymlaccuracy/test_cli_flow.py×7l0_dgx_h100.ymlcpp/test_multi_gpu.py×35 removed − 6 shared unit tests restored as active (net 29)l0_dgx_h200.ymlaccuracy/test_cli_flow.py×7,examples/test_enc_dec.py×2,examples/test_gpt.py×1,examples/test_llama.py×1,examples/test_mamba.py×1l0_gh200.ymlaccuracy/test_cli_flow.py×1l0_h100.ymlaccuracy/test_cli_flow.py×4,cpp/test_e2e.py×4,examples/test_eagle.py×2,examples/test_enc_dec.py×5,examples/test_granite.py×1,examples/test_medusa.py×2,examples/test_multimodal.py×8,examples/test_phi.py×1,examples/test_qwen.py×1,examples/test_redrafter.py×4l0_l40s.ymlaccuracy/test_cli_flow.py×3,examples/test_gpt.py×2,examples/test_granite.py×1,examples/test_llama.py×1,examples/test_medusa.py×2,examples/test_qwen.py×1,examples/test_redrafter.py×4Special case —
l0_a100.ymlempty-block prune. Removing the 1-GPUtensorrtblock'sonly surviving tests would have left a
tests:key with no list items. YAML parses that asnull, andtrt-test-dbthen crashes inspread()withTypeError: 'NoneType' object is not iterable(the exact failure that motivated thel0_dgx_h200.ymlfix in the prior commit). So the whole now-empty- condition:block wasdeleted, not just its test lines. This is why
l0_a100.ymlshows 17 deletions for only 4test entries.
waives.txt(−141 lines)... SKIP (TRTLLM-13781: legacy TensorRT examples removed; ...)cpp/test_multi_gpu.py×35,accuracy/test_cli_flow.py×28,examples/test_enc_dec.py×19,examples/test_gpt.py×11,examples/test_multimodal.py×10,cpp/test_e2e.py×9,examples/test_whisper.py×4,examples/test_redrafter.py×4,examples/test_draft_target_model.py×4,examples/test_medusa.py×3,examples/test_qwen.py×2,examples/test_llama.py×2,examples/test_eagle.py×2,examples/test_commandr.py×2,examples/test_phi.py×1,examples/test_mamba.py×1,examples/test_granite.py×1,examples/test_bert.py×1full:GH200/...video-neva... SKIP (https://nvbugs/4731514)video-nevamultimodal test carried three waives — the TRTLLM-13781 one plus twofull:<platform>/-scoped ones. Once its only schedule entry (l0_h100.yml) was removed, this GH200-scoped waive pointed at a test in no active list, sovalidate-test-listsfailed. Removed.full:sm100/...video-neva... SKIP (megatron-core 0.8 is not supported in python 3.12)Correction — 6 over-waived shared C++ unit tests restored as active
While reviewing
tests/integration/defs/cpp/, six entries in the TRTLLM-13781 batch turnedout not to depend on examples at all — #15763 over-waived them by bulk-waiving the whole
cpp/test_multi_gpu.pyfile. They are restored as active inl0_dgx_h100.yml(un-waived),not deleted:
Evidence they are example-independent:
build_google_tests+build_dir(+nprocs/kvcache_type) —not
multi_gpu_model/prepare_models_disagg, the fixtures that build a TRT engine fromexample checkpoints (via
cpp_common.prepare_model_tests→examples/…/build_*_engines.py).build_google_testsonly compiles the C++google-teststarget; no engine, no example.build/tests/unit_tests/multi_gpu/:mpiUtilsTest,kernels/gemmAllReduceTest --m=2032 --n=8200 --k=1024,cacheTransceiverTest,userBufferTest. No model / engine / checkpoint reference.keep_log_files) just backs up XML results.l0_dgx_h100.yml; [TRTLLM-13781][chore] Remove legacy TensorRT examples #15763waived them under "legacy TensorRT examples removed", and this commit's first pass then removed
the dead entries. Net path was running → waived → removed — an unintended coverage loss for
non-deprecated multi-GPU C++ (MoE GEMM-allreduce, disagg KV-cache transceiver, user buffers,
MPI utils). Restoring them reverts that regression.
test_cache_transceiver[8proc-mooncake_kvcache-90]keeps its real waive (nvbugs/5838199) —that one is unrelated to examples. The engine-based
test_multi_gpu.pycases(
test_llama_executor*,test_enc_dec,test_trt_gpt_real_decoder,TestDisagg::*) stayremoved/waived; they genuinely need the removed example engines and belong with the later C++
TRT-engine-path removal.
Intentionally NOT changed
Kept waived:
unittest/bindings/test_executor_bindings.pyThe one member of the TRTLLM-13781 batch left in
waives.txt. Reasons:unittest/bindingsdirectory entry (in
l0_a10.yml,l0_gh200.yml,l0_h100.yml) — there is noper-test line to delete, so it can't be descheduled without either a waive or deleting the file.
trtllm.Executorfrom a builtrank0.engine(itsmodel_filesfixture generates engines via the removed example build path), so removingthe waive would un-skip a test that cannot build its engine → CI failure.
test-list cleanup. Keeping the waive is the lowest-risk holding state.
Test
.pyfiles untouchedNo test source file was deleted. Files like
examples/test_gpt.py,test_medusa.py,test_multimodal.py, etc. still hold surviving PyTorch-path cases (e.g.examples/test_gpt.py::test_gpt_oss_20b_lora_torch). Only the specific TRT-flowparametrizations were unscheduled/unwaived. Deleting now-orphaned example test files (those
whose every case was TRT-only) is left as a possible later cleanup.
Verification
test-db/*.ymlparse; no condition block has anull/emptytests:.list (
test-db/*.yml,qa/*).waives.txtline (after stripping anyfull:<platform>/prefix) references a removed test id.Validate test list entries exist in source files (AST)(validate-test-lists) andChecks for duplicated test items in waives.txt.test_executor_bindings.pyTRTLLM-13781 waive remains.Summary by CodeRabbit
Description
Test Coverage
PR Checklist
Please review the following before submitting your PR:
PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either
api-compatibleorapi-breaking. Forapi-breaking, includeBREAKINGin the PR title.Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.
GitHub Bot Help
To see a list of available CI bot commands, please comment
/bot help.