[None][fix] Clean up Triton related tests by Tabrizian · Pull Request #13163 · NVIDIA/TensorRT-LLM

Tabrizian · 2026-04-17T21:17:43Z

Description

Cleans up the Triton-server integration tests and migrates QA coverage to L0 post-merge:

Remove broken/deprecated tests: drops the multimodal (LLaVA, LLaVA-OneVision, MLLaMA, BLIP2, Qwen2-VL, Pixtral), speculative-decoding-BLS, disaggregated-serving-BLS, Medusa, and unit-test-wrapper Triton tests from test_triton.py / test_triton_llm.py, along with their entries in the test manifests and waives.
Remove dead helper code: following the test removals, deletes the now-unused multimodal engine builders (prepare_blip2_opt_engine, prepare_llava_engine, prepare_llava_onevision_engine, prepare_mllama_engine, prepare_mistral3_pixtral_engine), the prepare_multimodal_model_repo / prepare_disaggregated_serving_model_repo / modify_disaggregated_serving_config_pbtxt helpers, the multimodal & disaggregated_serving_bls branches inside modify_ib_config_pbtxt, and the multimodal example-root / model-root fixtures. All removed symbols were verified to have no remaining references.
Restore test_benchmark_core_model dependencies: the cleanup had accidentally removed MODEL_CONFIGS, the model_setup fixture, and LATENCY_THRESHOLDS, which test_benchmark_core_model still references — this caused a NameError at collection time. These are restored.
Migrate QA coverage to L0 post-merge: promotes all still-valid QA Triton cases from tests/integration/test_lists/qa/llm_triton_integration.txt into the l0_a30 post_merge / triton section (505 cases not already present), since the QA list is being deprecated. Hardware-gated cases (8-GPU 175b/70b, 80 GB return-logits) are gated by their own skip_less_device / skip_less_device_memory markers, consistent with existing cases in that section. The multi-node case is excluded (needs dedicated multi-node orchestration).

Test Coverage

This PR only touches the Triton integration tests and their manifests; it adds no product code. Coverage is validated by the standard CI test-list validation (collection + manifest consistency) and by the migrated l0_a30 post_merge / triton cases.

PR Checklist

Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

Tabrizian · 2026-04-17T21:20:18Z

/bot run --disable-fail-fast

coderabbitai · 2026-04-17T21:24:23Z

📝 Walkthrough

Walkthrough

This pull request removes a significant number of Triton integration tests from both test definitions and test manifests, while also replacing a few tests with new variants. Changes include deletion of 23 test functions and restructuring of test configurations across multiple test files and test-list registries.

Changes

Cohort / File(s)	Summary
Test Definitions `tests/integration/defs/triton_server/test_triton.py`	Removed 23 test functions including Mistral IB variants, GPT in-flight batching/streaming/speculative-decoding tests, Medusa/Eagle tests, multimodal model tests (Blip2, LLaVA, Qwen2-VL), disaggregated serving tests, and unit test wrappers (Python BLS, C++, extensive tests).
Test Definitions - LLM `tests/integration/defs/triton_server/test_triton_llm.py`	Removed unused `re` import. Deleted `test_medusa_vicuna_7b_ifb` test. Renamed/replaced `test_gpt_speculative_decoding_bls` with `test_gpt_175b_dummyWeights_ifb` (changed from speculative decoding to standard inflight batching with 8-GPU world size). Renamed/replaced `test_llama_v3_speculative_decoding_bls` with `test_gpt_next_ptuning_ifb` (changed from speculative decoding to pTuning mode with 1-GPU world size). Deleted disaggregated serving and Pixtral multimodal tests.
Test Manifests - QA `tests/integration/test_lists/qa/llm_triton_integration.txt`	Removed 67 lines of test entries including `test_medusa_vicuna_7b_ifb`, `test_gpt_speculative_decoding_bls`, `test_llama_v3_speculative_decoding_bls`, `test_llava`, `test_llava_onevision`, `test_mllama`, `test_gpt_disaggregated_serving_bls`, and `test_mistral_small_3_1_24b_pixtral` variants.
Test Manifests - Device Lists `tests/integration/test_lists/test-db/l0_a100.yml`, `tests/integration/test_lists/test-db/l0_b200.yml`	Removed post-merge Triton backend test condition blocks targeting A100/B200 GPUs, including associated test entries for disaggregated serving, LLaVA, GPT IB variants, speculative decoding, Medusa, and other LLM tests.
Test Manifests - A30 Device List `tests/integration/test_lists/test-db/l0_a30.yml`	Removed `t5_ib`, `mistral_ib`, `mistral_ib_mm`, `gpt_ib_streaming`, unit test entries, and LLM BLS/speculative/generation variants while keeping remaining LLM token-count and preprocessing tests.
Test Waives/Skips `tests/integration/test_lists/waives.txt`	Removed 21 skip waiver entries for Triton server tests including speculative decoding, gather logits, disaggregated serving, LLaVA/LLaMA/Eagle variants, and unit test entries.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 16.67% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title '[None][fix] Clean up Triton related tests' directly reflects the main objective of the PR, which is to remove and clean up Triton-related test configurations and test code across multiple files.
Description check	✅ Passed	The PR description matches the required template and includes a clear description, test coverage, and checklist.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands.}

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@tests/integration/defs/triton_server/test_triton_llm.py`:
- Around line 2034-2037: The test contains an in-test workaround that invokes
pip ("pip3 install -U nvidia-cublas-cu12") based on query_gpu_name(), which
mutates the shared Python environment and requires network access; remove that
conditional pip install from the test body (the code block referencing
query_gpu_name() and the pip call) and instead apply the workaround outside the
test runtime — either bake nvidia-cublas-cu12 into the runner/CI image or move
installation into an isolated setup step that launches Triton (e.g., a dedicated
session/module pytest fixture or the Triton start script that runs once per
job), ensuring no runtime pip installs occur inside tests like
test_triton_llm.py and that query_gpu_name() is only used for skips/assumptions,
not environment mutation.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: d4825b79-3f87-40c5-96b1-7469ff56beff

📥 Commits

Reviewing files that changed from the base of the PR and between 813d877 and 937eca9.

📒 Files selected for processing (7)

tests/integration/defs/triton_server/test_triton.py
tests/integration/defs/triton_server/test_triton_llm.py
tests/integration/test_lists/qa/llm_triton_integration.txt
tests/integration/test_lists/test-db/l0_a100.yml
tests/integration/test_lists/test-db/l0_a30.yml
tests/integration/test_lists/test-db/l0_b200.yml
tests/integration/test_lists/waives.txt

💤 Files with no reviewable changes (5)

tests/integration/test_lists/test-db/l0_b200.yml
tests/integration/test_lists/test-db/l0_a100.yml
tests/integration/test_lists/waives.txt
tests/integration/test_lists/qa/llm_triton_integration.txt
tests/integration/defs/triton_server/test_triton.py

tensorrt-cicd · 2026-04-17T21:26:21Z

PR_Github #44072 [ run ] triggered by Bot. Commit: 937eca9 Link to invocation

tensorrt-cicd · 2026-04-18T08:59:47Z

PR_Github #44072 [ run ] completed with state FAILURE. Commit: 937eca9
/LLM/main/L0_MergeRequest_PR pipeline #34502 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

StanleySun639 · 2026-04-20T08:44:29Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-04-20T08:50:55Z

PR_Github #44409 [ run ] triggered by Bot. Commit: 937eca9 Link to invocation

tensorrt-cicd · 2026-04-20T11:09:06Z

PR_Github #44409 [ run ] completed with state FAILURE. Commit: 937eca9
/LLM/main/L0_MergeRequest_PR pipeline #34822 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

Tabrizian · 2026-04-21T23:57:09Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-04-22T00:03:24Z

PR_Github #44825 [ run ] triggered by Bot. Commit: 26f8481 Link to invocation

tensorrt-cicd · 2026-04-22T01:17:10Z

PR_Github #44825 [ run ] completed with state SUCCESS. Commit: 26f8481
/LLM/main/L0_MergeRequest_PR pipeline #35171 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

…usa coverage Drops waived triton_server test entries from QA and l0 lists, then removes the corresponding multimodal, speculative-decoding-BLS, disaggregated-BLS, and medusa test functions that are no longer referenced by any active list. Keeps t5 and llmapi coverage. Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com>

Remove triton_server waives whose test functions were deleted in the prior commit, plus specific parametrizations that no longer appear in any active test list. Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com>

After removing the remaining triton_server entries, the post_merge triton-backend blocks in l0_a100.yml and l0_b200.yml were empty. Remove them entirely. Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com>

StanleySun639 · 2026-04-22T05:32:32Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-04-22T05:39:02Z

PR_Github #44900 [ run ] triggered by Bot. Commit: ec7da0b Link to invocation

tensorrt-cicd · 2026-04-22T10:46:39Z

PR_Github #44900 [ run ] completed with state SUCCESS. Commit: ec7da0b
/LLM/main/L0_MergeRequest_PR pipeline #35233 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

pcastonguay · 2026-04-27T20:11:20Z

 @pytest.mark.parametrize("MAX_BEAM_WIDTH", ["1"])
 @pytest.mark.parametrize("EXCLUDE_INPUT_IN_OUTPUT", ["False"])
 @pytest.mark.parametrize("model_setup",
                         list(MODEL_CONFIGS.keys()),


Looks like tests are failing because MODEL_CONFIGS no longer exist.

StanleySun639 · 2026-05-05T06:17:46Z

@Tabrizian TRTLLM is migrating QA role to L0 post-merge, could you please pick necessary test cases from QA test list into L0 post-merge coverage? Eventually QA test list will be depricated. cc @schetlur-nv for awareness.

The triton test cleanup accidentally removed the MODEL_CONFIGS dict, the model_setup fixture, and the LATENCY_THRESHOLDS dict while test_benchmark_core_model (still present and still listed in l0_a30.yml) references all three. This caused a NameError at collection time, failing the whole module. Restore the three definitions. Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com>

Following the removal of the multimodal, speculative-decoding-BLS, and disaggregated-serving-BLS triton tests, delete the now-unused helper code: - build_engines.py: prepare_blip2_opt_engine, prepare_llava_engine, prepare_llava_onevision_engine, prepare_mllama_engine, prepare_mistral3_pixtral_engine - common.py: prepare_multimodal_model_repo, prepare_disaggregated_serving_model_repo, modify_disaggregated_serving_config_pbtxt, and the multimodal / disaggregated_serving_bls branches inside modify_ib_config_pbtxt - conftest.py: multimodal/mllama example-root and llava/llava_onevision/ blip2/mllama/test_video model-root fixtures All removed symbols were verified to have no remaining references. Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com>

QA coverage is being migrated to L0 post-merge and the QA test list (tests/integration/test_lists/qa/llm_triton_integration.txt) will be deprecated. Promote all still-valid QA triton cases into the a30 post_merge triton section (505 cases not already present). Hardware-gated cases (8-GPU 175b/70b, the 80GB return-logits case) are gated at runtime by their own skip_less_device/skip_less_device_memory markers, consistent with the cases already in this section. The multi-node case (test_triton_multi_node.py) is excluded as it needs dedicated multi-node orchestration that a single-node condition cannot express. Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com>

Tabrizian · 2026-06-30T22:39:14Z

/bot run --disable-fail-fast

…iton-server-tests Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com> # Conflicts: # tests/integration/defs/triton_server/conftest.py # tests/integration/test_lists/qa/llm_triton_integration.txt # tests/integration/test_lists/test-db/l0_a30.yml # tests/integration/test_lists/waives.txt

Tabrizian · 2026-06-30T22:58:00Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-06-30T23:04:13Z

PR_Github #56751 [ run ] triggered by Bot. Commit: be111c8 Link to invocation

tensorrt-cicd · 2026-07-01T01:14:14Z

PR_Github #56751 [ run ] completed with state SUCCESS. Commit: be111c8
/LLM/main/L0_MergeRequest_PR pipeline #45570 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

main removed prepare_llama_v2_7b_engine and trimmed MODEL_CONFIGS to gptj_6b only, so test_benchmark_core_model no longer has a llama_v2_7b parametrization. Remove the now-invalid l0_a30 entry that the CI "Check Test List" stage flagged as an invalid test name filter. Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com>

Tabrizian · 2026-07-01T01:26:06Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-01T01:32:23Z

PR_Github #56780 [ run ] triggered by Bot. Commit: 8e1c293 Link to invocation

This PR removes all triton post-merge tests from l0_a100.yml and l0_b200.yml (they only covered multimodal / disaggregated-serving-BLS tests, which are deleted). The DGX_A100-Triton-Post-Merge-1/2 and DGX_B200-Triton-Post-Merge-1 stages in L0_Test.groovy still point at those lists, so they collect zero tests and fail with pytest exit code 5. Remove the now-empty stage definitions. Single-GPU triton coverage remains on the A30-Triton-Post-Merge stages (l0_a30) and the DGX-H200 stage keeps its llmapi_backend test. Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com>

The A30-Triton-Post-Merge stages surfaced runtime failures in four functions migrated from the QA list, deterministic across both shards: test_gpt_350m_speculative_decoding, test_gpt_next_ptuning_ifb, test_mistral_v1_multi_models, and test_rcca_bug_4934893 (the latter is tracked by nvbugs/5619369). These legacy tests do not pass in the L0 a30 environment, so exclude them from the migration; the remaining migrated triton tests pass. They can be re-added once fixed/triaged separately. Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com>

Tabrizian · 2026-07-01T05:43:19Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-01T05:49:04Z

PR_Github #56849 [ run ] triggered by Bot. Commit: 67ff028 Link to invocation

tensorrt-cicd · 2026-07-01T05:53:49Z

PR_Github #56780 [ run ] completed with state ABORTED. Commit: 8e1c293

Link to invocation

tensorrt-cicd · 2026-07-01T06:53:42Z

PR_Github #56849 [ run ] completed with state FAILURE. Commit: 67ff028
/LLM/main/L0_MergeRequest_PR pipeline #45662 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Tabrizian · 2026-07-01T16:01:03Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-01T16:07:24Z

PR_Github #56963 [ run ] triggered by Bot. Commit: 67ff028 Link to invocation

tensorrt-cicd · 2026-07-01T22:20:40Z

PR_Github #56963 [ run ] completed with state SUCCESS. Commit: 67ff028
/LLM/main/L0_MergeRequest_PR pipeline #45768 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Tabrizian · 2026-07-01T22:26:29Z

/bot run --disable-fail-fast

Per the QA->L0 migration rule, move all QA triton tests to l0_a30 post-merge except multi-model (test_mistral_v1_multi_models) and speculative-decoding (test_gpt_350m_speculative_decoding[_return_logits]) tests. test_gpt_next_ptuning_ifb is dropped (fails on a30). Re-add the test_rcca_bug_4934893 cases and waive all 8 param combos under nvbugs/5619369 (known-failing RCCA), and drop the spec-dec return_logits entries that remained. Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com>

Tabrizian · 2026-07-01T22:56:25Z

/bot run --disable-fail-fast

…iton-server-tests Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com> # Conflicts: # jenkins/L0_Test.groovy

Tabrizian · 2026-07-01T23:20:29Z

/bot run --disable-fail-fast

Tabrizian · 2026-07-01T23:42:50Z

/bot run --disable-fail-fast

Tabrizian · 2026-07-02T00:02:20Z

/bot run --disable-fail-fast

Tabrizian · 2026-07-06T16:38:09Z

Closing, since this was already completed here: #15814 (comment)

Tabrizian requested review from a team as code owners April 17, 2026 21:17

github-actions Bot assigned Tabrizian Apr 17, 2026

Tabrizian requested a review from pcastonguay April 17, 2026 21:20

coderabbitai Bot reviewed Apr 17, 2026

View reviewed changes

Comment thread tests/integration/defs/triton_server/test_triton_llm.py

StanleySun639 approved these changes Apr 20, 2026

View reviewed changes

Tabrizian force-pushed the user/imant/cleanup-triton-server-tests branch from 937eca9 to 26f8481 Compare April 21, 2026 23:57

Tabrizian added 3 commits April 22, 2026 13:32

[None][test] Drop stale triton_server waives

75cdae3

Remove triton_server waives whose test functions were deleted in the prior commit, plus specific parametrizations that no longer appear in any active test list. Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com>

StanleySun639 force-pushed the user/imant/cleanup-triton-server-tests branch from 26f8481 to ec7da0b Compare April 22, 2026 05:32

pcastonguay approved these changes Apr 22, 2026

View reviewed changes

pcastonguay reviewed Apr 27, 2026

View reviewed changes

Tabrizian added 3 commits June 30, 2026 15:20

Tabrizian requested a review from a team July 1, 2026 04:32

Tabrizian requested a review from a team as a code owner July 1, 2026 04:32

Tabrizian requested review from ZhanruiSunCh and dpitman-nvda July 1, 2026 04:32

Merge remote-tracking branch 'origin/main' into user/imant/cleanup-tr…

86550be

…iton-server-tests Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com> # Conflicts: # jenkins/L0_Test.groovy

Tabrizian closed this Jul 6, 2026

Uh oh!

Conversation

Tabrizian commented Apr 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Test Coverage

PR Checklist

GitHub Bot Help

Uh oh!

Tabrizian commented Apr 17, 2026

Uh oh!

coderabbitai Bot commented Apr 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

❌ Failed checks (1 warning)

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

tensorrt-cicd commented Apr 17, 2026

Uh oh!

tensorrt-cicd commented Apr 18, 2026

Uh oh!

StanleySun639 commented Apr 20, 2026

Uh oh!

tensorrt-cicd commented Apr 20, 2026

Uh oh!

tensorrt-cicd commented Apr 20, 2026

Uh oh!

Tabrizian commented Apr 21, 2026

Uh oh!

tensorrt-cicd commented Apr 22, 2026

Uh oh!

tensorrt-cicd commented Apr 22, 2026

Uh oh!

StanleySun639 commented Apr 22, 2026

Uh oh!

tensorrt-cicd commented Apr 22, 2026

Uh oh!

tensorrt-cicd commented Apr 22, 2026

Uh oh!

pcastonguay Apr 27, 2026

Choose a reason for hiding this comment

Uh oh!

StanleySun639 commented May 5, 2026

Uh oh!

Tabrizian commented Jun 30, 2026

Uh oh!

Tabrizian commented Jun 30, 2026

Uh oh!

tensorrt-cicd commented Jun 30, 2026

Uh oh!

tensorrt-cicd commented Jul 1, 2026

Uh oh!

Tabrizian commented Jul 1, 2026

Uh oh!

tensorrt-cicd commented Jul 1, 2026

Uh oh!

Tabrizian commented Jul 1, 2026

Uh oh!

tensorrt-cicd commented Jul 1, 2026

Uh oh!

tensorrt-cicd commented Jul 1, 2026

Uh oh!

tensorrt-cicd commented Jul 1, 2026

Uh oh!

Tabrizian commented Jul 1, 2026

Uh oh!

tensorrt-cicd commented Jul 1, 2026

Uh oh!

tensorrt-cicd commented Jul 1, 2026

Uh oh!

Tabrizian commented Jul 1, 2026

Uh oh!

Tabrizian commented Jul 1, 2026

Uh oh!

Tabrizian commented Jul 1, 2026

Tabrizian commented Apr 17, 2026 •

edited

Loading

coderabbitai Bot commented Apr 17, 2026 •

edited

Loading