[None][Test] Consolidate dis-agg E2E Tests by Shixiaowei02 · Pull Request #16614 · NVIDIA/TensorRT-LLM

Shixiaowei02 · 2026-07-20T09:32:16Z

Description

This pull request removes support for the UCX backend in disaggregated serving test configurations and related test cases. The main changes include deleting all UCX-specific test YAML files, removing UCX test entries from the test configuration loader, and updating test logic to use the DEFAULT backend instead of UCX. Additionally, some test parameterizations were simplified to avoid redundant or unnecessary cases.

Removal of UCX backend support:

Deleted UCX-specific test configuration files: disagg_config_ctxtp2_gentp2_deepseek_v3_lite_ucx.yaml and disagg_config_ctxtp2_gentp2_llama31_8b_ucx.yaml. [1] [2]
Removed all references to UCX-based tests from get_test_config in test_disaggregated.py, including "deepseek_v3_lite_fp8_ucx" and "llama31_8b_ucx". [1] [2]
Removed UCX-specific test functions: test_disaggregated_deepseek_v3_lite_fp8_ucx and test_disaggregated_deepseek_v3_lite_fp8_ucx_tp1_single_gpu. [1] [2]

Test configuration and parameterization updates:

Updated all test cases in test_disaggregated_serving.py to use "backend": "DEFAULT" instead of "UCX" for cache_transceiver_config. [1] [2] [3] [4] [5]
Simplified test parameterizations by removing redundant or unnecessary combinations, e.g., only testing block reuse on/off and limiting grammar backend parameterization to "xgrammar". [1] [2] [3]

These changes streamline the test suite by removing support for an unused backend and reducing test complexity.

Test Coverage

PR Checklist

Please review the following before submitting your PR:

PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

Summary by CodeRabbit

Tests
- Streamlined disaggregated serving coverage for automatic data types, focusing on KV cache block reuse scenarios.
- Standardized guided-decoding coverage on the xgrammar backend.
- Updated test manifests across H100 and RTX environments to reflect the revised test scenarios.

coderabbitai · 2026-07-20T09:36:27Z

📝 Walkthrough

Walkthrough

The change narrows disaggregated-serving accuracy coverage by limiting auto-dtype tests to block-reuse modes and guided-decoding tests to the xgrammar backend, with matching updates to QA and test-database manifests.

Changes

Disaggregated serving test coverage

Layer / File(s)	Summary
Narrow auto-dtype parameter coverage `tests/integration/defs/accuracy/test_disaggregated_serving.py`, `tests/integration/test_lists/qa/llm_function_core.txt`, `tests/integration/test_lists/test-db/l0_dgx_h100.yml`	Auto-dtype tests now cover `block_reuse` and `no_block_reuse`, use a fixed overlap-scheduler setting, and update the corresponding test selections.
Restrict guided-decoding backend coverage `tests/integration/defs/accuracy/test_disaggregated_serving.py`, `tests/integration/test_lists/qa/*`, `tests/integration/test_lists/test-db/l0_rtx_pro_6000.yml`	Guided-decoding coverage retains `xgrammar` cases and removes `llguidance` entries for standard and helix/MTP variants.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: bo-nv, chienchunhung, brnguyen2

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check	⚠️ Warning	Title is relevant, but it uses the invalid type tag "[Test]" instead of a required lowercase type.	Use the required format, e.g. "[None][chore] Consolidate disagg E2E tests".

✅ Passed checks (3 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description check	✅ Passed	The description explains the change and checklist, but the Test Coverage section is empty.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands.}

Shixiaowei02 · 2026-07-20T09:38:11Z

/bot run --stage-list "DGX_H100-2_GPUs-PyTorch-Others-*, DGX_H100-4_GPUs-PyTorch-DeepSeek-1"

tensorrt-cicd · 2026-07-20T09:43:41Z

PR_Github #60342 [ run ] triggered by Bot. Commit: 6fb85e4 Link to invocation

tensorrt-cicd · 2026-07-20T09:58:42Z

PR_Github #60342 [ run ] completed with state FAILURE. Commit: 6fb85e4
/LLM/main/L0_MergeRequest_PR pipeline #48684 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Shixiaowei02 · 2026-07-20T10:52:18Z

/bot run --stage-list "DGX_H100-2_GPUs-PyTorch-Others-*, DGX_H100-4_GPUs-PyTorch-DeepSeek-1"

tensorrt-cicd · 2026-07-20T10:58:17Z

PR_Github #60359 [ run ] triggered by Bot. Commit: f430de6 Link to invocation

tensorrt-cicd · 2026-07-20T11:46:01Z

PR_Github #60359 [ run ] completed with state FAILURE. Commit: f430de6
/LLM/main/L0_MergeRequest_PR pipeline #48701 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Tabrizian · 2026-07-20T15:58:57Z

-@pytest.mark.skip_less_device(4)
-@pytest.mark.parametrize("llama_model_root", ['llama-3.1-8b-instruct'],
-                         indirect=True)
-def test_disaggregated_logprobs_serving(disaggregated_test_root,


Do we test log probs in any other place?

The API-level coverage here is unique: streaming vs. non-streaming logprobs consistency and top_logprobs across the ctx/gen boundary (RCCA for https://nvbugs/5926823). Only the core propagation is covered elsewhere (test_disaggregated_single_gpu.py::test_disaggregated_logprobs).

The test isn't UCX-specific — it just reused the UCX yaml. It also has an open bug (https://nvbugs/6275959, fix in flight: #15221). Suggest migrating it to a DEFAULT/NIXL config and keeping the waive, instead of deleting.

fredricz-20070104 · 2026-07-21T02:15:53Z

One thing worth calling out that isn't mentioned in the PR description: besides the UCX consolidation and parameter trimming, this PR also removes test_disaggregated_logprobs_serving (and its corresponding waive entry). That test is the resident regression guard for a customer bug (NVBug 5926823, disagg + streaming + logprobs).

Per our RCCA / test-escape policy, a customer-bug regression test shouldn't be silently dropped — it should be confirmed by the RCCA owner and the removal should be documented in the description. Could you please either (a) confirm that scenario is now covered elsewhere and note it in the PR description, or (b) keep (fix) the test instead of deleting it? Thanks!

fredricz-20070104

Please resolve my comments before merging.

nv-xtf

LGTM overall — just one question about the backend default behavior (DEFAULT still resolves to UCX in this harness), see inline comment.

nv-xtf · 2026-07-21T02:56:25Z

            "cuda_graph_config": None,
            "cache_transceiver_config": {
-                "backend": "UCX",
+                "backend": "DEFAULT",


Quick question: DEFAULT still resolves to UCX here because launch_disaggregated_llm sets TRTLLM_USE_UCX_KVCACHE=1 for any backend other than NIXL. Is the intent only to remove explicit per-test UCX pinning while retaining UCX as the harness default, or should these tests eventually exercise NIXL?

…nd config Review follow-up on NVIDIA#16614 (yingguo-trt, fredricz-20070104): the consolidation dropped test_disaggregated_logprobs_serving, which is the resident regression guard for NVBug 5926823 (disagg + streaming + logprobs). Its API-level coverage is unique -- streaming vs. non-streaming logprobs consistency across both the completions and chat APIs, plus chat top_logprobs across the ctx/gen boundary. test_disaggregated_single_gpu.py::test_disaggregated_logprobs only covers the core prefill->decode propagation, so deleting this would be a test escape. The test was never UCX-specific; it only reused the UCX yaml. Restore it on a new DEFAULT-backend config and drop the explicit TRTLLM_USE_UCX_KVCACHE pin, which keeps the de-UCX-specialization goal of this PR intact. UCX_TLS is kept, matching every other non-pinned test in this file. The waive for the open bug (NVBugs 6275959, fix in flight in NVIDIA#15221) is restored with it. Restores the numpy import: np.isclose in this test is its only user. Signed-off-by: Shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com>

The 2**4 cross-product of ctx/gen overlap-scheduler x ctx/gen block-reuse ran a full MMLU+GSM8K eval per combo. Overlap scheduler is token-invariant (covered by unit tests test_benchmark_disagg.py and test_disagg_gen_transfer_gate.py), so only the block-reuse axis changes which KV blocks are transferred. Keep block-reuse on/off with overlap enabled (production default) and drop the other 14 combos from CI (l0_dgx_h100, llm_function_core). Signed-off-by: Shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>

…d_decoding Guided decoding runs entirely on the generation worker and the KV-cache handoff is grammar-backend-agnostic, so exercising both xgrammar and llguidance under disaggregated serving only re-validates grammar-engine correctness that the aggregated guided-decoding tests already cover (test_llm_api_pytorch.py has 7 backend-parametrized sites). Keep xgrammar (the default) for the disagg plumbing guard and the DeepSeek guided+MTP interaction; drop the llguidance combos from TestLlama3_1_8BInstruct/TestDeepSeekV3Lite::test_guided_decoding (7 CI entries across l0_rtx_pro_6000, llm_function_core, llm_function_rtx6k). Signed-off-by: Shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>

…kv_cache_v2 UCX cache-transceiver coverage is model-agnostic plumbing already exercised by the cheap unit/C++ tests (unittest/disaggregated UCX params, cpp/test_multi_gpu test_cache_transceiver ucx_kvcache), so the expensive e2e UCX-specialized integration tests are redundant. Remove test_disaggregated_deepseek_v3_lite_fp8_ucx and test_disaggregated_deepseek_v3_lite_fp8_ucx_tp1_single_gpu, plus their now orphaned config disagg_config_ctxtp2_gentp2_deepseek_v3_lite_ucx.yaml and its config-map entry. Also remove TestLlama3_1_8BInstruct::test_kv_cache_v2_nixl_python: the KVCacheManager-v2 + NIXL + PYTHON transceiver path is model-agnostic and byte-covered by test_cache_transceiver_single_process.py; TestDeepSeekV3Lite keeps the canonical MLA representative. Signed-off-by: Shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>

…ogprobs test Switch the hardcoded backend=UCX in TestDeepSeekV3Lite/TestQwen3_8B test_auto_dtype_with_helix and the TestQwen3_8B chunked-prefill helper to DEFAULT, matching the rest of the accuracy suite (these tests cover helix and chunked prefill, not UCX specifically). Remove test_disaggregated_logprobs_serving and the llama31_8b_ucx config-map entry plus the orphaned yaml it exclusively used. Signed-off-by: Shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>

…nd config Review follow-up on NVIDIA#16614 (yingguo-trt, fredricz-20070104): the consolidation dropped test_disaggregated_logprobs_serving, which is the resident regression guard for NVBug 5926823 (disagg + streaming + logprobs). Its API-level coverage is unique -- streaming vs. non-streaming logprobs consistency across both the completions and chat APIs, plus chat top_logprobs across the ctx/gen boundary. test_disaggregated_single_gpu.py::test_disaggregated_logprobs only covers the core prefill->decode propagation, so deleting this would be a test escape. The test was never UCX-specific; it only reused the UCX yaml. Restore it on a new DEFAULT-backend config and drop the explicit TRTLLM_USE_UCX_KVCACHE pin, which keeps the de-UCX-specialization goal of this PR intact. UCX_TLS is kept, matching every other non-pinned test in this file. The waive for the open bug (NVBugs 6275959, fix in flight in NVIDIA#15221) is restored with it. Restores the numpy import: np.isclose in this test is its only user. Signed-off-by: Shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com>

github-actions Bot assigned Shixiaowei02 Jul 20, 2026

Shixiaowei02 marked this pull request as ready for review July 20, 2026 09:33

Shixiaowei02 requested review from a team as code owners July 20, 2026 09:33

Shixiaowei02 requested review from Tabrizian, brnguyen2, chuangz0, fredricz-20070104, joyang-nv, longlee0622, mlefeb01, mzweilz, nv-xtf, reasonsolo, schetlur-nv and yingguo-trt July 20, 2026 09:33

Shixiaowei02 force-pushed the user/xiaoweis/trim-disagg-e2e-accuracy branch from 9ce9bbb to f430de6 Compare July 20, 2026 10:49

Tabrizian approved these changes Jul 20, 2026

View reviewed changes

fredricz-20070104 approved these changes Jul 21, 2026

View reviewed changes

nv-xtf approved these changes Jul 21, 2026

View reviewed changes

yingguo-trt approved these changes Jul 28, 2026

View reviewed changes

Shixiaowei02 force-pushed the user/xiaoweis/trim-disagg-e2e-accuracy branch from f430de6 to 973bfa0 Compare August 3, 2026 05:54

Shixiaowei02 requested review from a team as code owners August 3, 2026 05:54

Shixiaowei02 requested review from StanleySun639 and ruodil August 3, 2026 05:54

Shixiaowei02 and others added 5 commits August 3, 2026 04:21

Shixiaowei02 force-pushed the user/xiaoweis/trim-disagg-e2e-accuracy branch from 973bfa0 to 7a1e5eb Compare August 3, 2026 11:23

StanleySun639 approved these changes Aug 4, 2026

View reviewed changes

Uh oh!

Conversation

Shixiaowei02 commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Test Coverage

PR Checklist

GitHub Bot Help

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

❌ Failed checks (2 warnings)

Uh oh!

Shixiaowei02 commented Jul 20, 2026

Uh oh!

tensorrt-cicd commented Jul 20, 2026

Uh oh!

tensorrt-cicd commented Jul 20, 2026

Uh oh!

Shixiaowei02 commented Jul 20, 2026

Uh oh!

tensorrt-cicd commented Jul 20, 2026

Uh oh!

tensorrt-cicd commented Jul 20, 2026

Uh oh!

Tabrizian Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

yingguo-trt Jul 21, 2026

Choose a reason for hiding this comment

Uh oh!

fredricz-20070104 commented Jul 21, 2026

Uh oh!

fredricz-20070104 left a comment

Choose a reason for hiding this comment

Uh oh!

nv-xtf left a comment

Choose a reason for hiding this comment

Uh oh!

nv-xtf Jul 21, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Shixiaowei02 commented Jul 20, 2026 •

edited

Loading

coderabbitai Bot commented Jul 20, 2026 •

edited

Loading