Skip to content

[TRTLLM-13639][test] Migrate Kimi dis-agg tests to Transceiver v2 and trim tests - #16482

Merged
Shixiaowei02 merged 1 commit into
NVIDIA:mainfrom
Shixiaowei02:dev/xiaoweis/kimi-disagg-py-transceiver
Jul 21, 2026
Merged

[TRTLLM-13639][test] Migrate Kimi dis-agg tests to Transceiver v2 and trim tests#16482
Shixiaowei02 merged 1 commit into
NVIDIA:mainfrom
Shixiaowei02:dev/xiaoweis/kimi-disagg-py-transceiver

Conversation

@Shixiaowei02

@Shixiaowei02 Shixiaowei02 commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator

Description

This pull request removes the TestKimiK2 disaggregated serving test and updates the TestKimiK25 test to use the NIXL Python (v2) cache transceiver instead of the default backend. The test list is also updated to reflect the removal of the Kimi-K2 test. These changes streamline the test suite and ensure the Kimi-K2.5 test uses the latest cache transceiver configuration.

Test removals:

  • Removed the TestKimiK2 class and its test_nvfp4 method from test_disaggregated_serving.py, which tested the Kimi-K2 model in a disaggregated serving setup.
  • Removed the corresponding TestKimiK2::test_nvfp4 entry from the test list in qa/llm_function_core.txt.

Test configuration updates:

  • Updated the TestKimiK25.test_nvfp4 method to use the NIXL backend with PYTHON transceiver runtime for both context and generation servers, replacing the previous DEFAULT backend. [1] [2]
  • Clarified the test docstring to indicate the use of the NIXL Python (v2) cache transceiver.

Test Coverage

PR Checklist

Please review the following before submitting your PR:

  • PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

  • PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

  • Test cases are provided for new code paths (see test instructions)

  • If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

  • Any new dependencies have been scanned for license and vulnerabilities

  • CODEOWNERS updated if ownership changes

  • Documentation updated as needed

  • Update tava architecture diagram if there is a significant design change in PR.

  • The reviewers assigned automatically/manually are appropriate for the PR.

  • Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

Summary by CodeRabbit

  • Tests
    • Updated NVFP4 accuracy coverage for the Kimi K2.5 model.
    • Refined GB200 and GB300 performance scenarios to emphasize key large-context configurations.
    • Improved disaggregated serving test coverage with explicit runtime and resource limits.
    • Removed outdated or redundant benchmark scenarios, including legacy UCX-based configurations.
    • Expanded performance validation for selected Kimi K2.5 and DeepSeek model variants.

@Shixiaowei02 Shixiaowei02 changed the title [None][test] Switch Kimi dis-agg tests to the transceiver v2 [None][test] Migrate Kimi dis-agg tests to Transceiver v2 and trim tests Jul 16, 2026
@Shixiaowei02 Shixiaowei02 changed the title [None][test] Migrate Kimi dis-agg tests to Transceiver v2 and trim tests [TRTLLM-13639][test] Migrate Kimi dis-agg tests to Transceiver v2 and trim tests Jul 16, 2026
@Shixiaowei02
Shixiaowei02 force-pushed the dev/xiaoweis/kimi-disagg-py-transceiver branch from 8d16a99 to 853df7a Compare July 16, 2026 07:22
@Shixiaowei02
Shixiaowei02 marked this pull request as ready for review July 16, 2026 07:23
@Shixiaowei02
Shixiaowei02 requested review from a team as code owners July 16, 2026 07:23
@Shixiaowei02

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

Replaces the Kimi-K2 NVFP4 accuracy test with Kimi-K2.5 coverage, updates disaggregated performance selections, removes obsolete benchmark variants, and explicitly configures the Python runtime for selected NIXL transceivers.

Changes

Kimi-K2.5 disaggregated serving

Layer / File(s) Summary
Kimi-K2.5 accuracy test
tests/integration/defs/accuracy/test_disaggregated_serving.py, tests/integration/test_lists/qa/llm_function_core.txt
Replaces TestKimiK2 with TestKimiK25, updates model and server limits, and configures both context and generation servers with NIXL and the Python transceiver runtime.
Performance test selection
tests/integration/test_lists/qa/*, tests/integration/test_lists/test-db/*
Removes obsolete Kimi context, UCX, and small-concurrency variants while retaining or adding selected Kimi-K2.5 NIXL and DeepSeek cases across GB200 and GB300 matrices.
Benchmark transceiver runtime
tests/scripts/perf/disaggregated/gb200_kimi-k25-thinking-fp4_*.yaml, tests/scripts/perf/disaggregated/gb300_kimi-k25-thinking-fp4_*.yaml
Adds transceiver_runtime: PYTHON to generation and context NIXL cache-transceiver configurations.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

  • NVIDIA/TensorRT-LLM#16355: Adds Python transceiver runtime settings to related disaggregated performance configurations.
  • NVIDIA/TensorRT-LLM#16390: Switches disaggregated serving context and generation servers from DEFAULT to NIXL with the Python runtime.

Suggested reviewers: fredricz-20070104

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the main change: migrating Kimi disaggregated tests to NIXL v2 and trimming redundant tests.
Description check ✅ Passed The description follows the template with a clear summary and checklist, but the Test Coverage section is left empty.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/integration/defs/accuracy/test_disaggregated_serving.py`:
- Around line 2233-2234: Update the test description/docstring associated with
the configurations near the NIXL backend and PYTHON transceiver_runtime entries
to state that both roles explicitly use NIXL with the Python runtime, rather
than the default transceiver. Apply the same wording update to the additional
location noted in the review.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: aeb230f1-3914-445e-a075-d9339380429a

📥 Commits

Reviewing files that changed from the base of the PR and between 44f0521 and 853df7a.

📒 Files selected for processing (26)
  • tests/integration/defs/accuracy/test_disaggregated_serving.py
  • tests/integration/test_lists/qa/llm_function_core.txt
  • tests/integration/test_lists/qa/llm_perf_disagg.yml
  • tests/integration/test_lists/qa/llm_perf_multinode.txt
  • tests/integration/test_lists/test-db/l0_gb200_multi_gpus_perf_sanity.yml
  • tests/integration/test_lists/test-db/l0_gb200_multi_nodes_perf_sanity_ctx1_node1_gpu4_gen1_node1_gpu4.yml
  • tests/integration/test_lists/test-db/l0_gb200_multi_nodes_perf_sanity_ctx1_node1_gpu4_gen1_node2_gpu8.yml
  • tests/integration/test_lists/test-db/l0_gb300_multi_gpus_perf_sanity.yml
  • tests/integration/test_lists/test-db/l0_gb300_multi_nodes_perf_sanity_ctx1_node1_gpu4_gen1_node1_gpu4.yml
  • tests/integration/test_lists/test-db/l0_gb300_multi_nodes_perf_sanity_ctx1_node1_gpu4_gen1_node2_gpu8.yml
  • tests/scripts/perf/disaggregated/gb200_kimi-k25-thinking-fp4_1k1k_con2048_ctx1_dep4_gen1_dep32_eplb0_mtp0_ccb-NIXL.yaml
  • tests/scripts/perf/disaggregated/gb200_kimi-k25-thinking-fp4_1k1k_con4096_ctx1_dep4_gen1_dep8_eplb0_mtp0_ccb-NIXL.yaml
  • tests/scripts/perf/disaggregated/gb200_kimi-k25-thinking-fp4_1k1k_con4096_ctx1_dep4_gen1_dep8_eplb0_mtp0_ccb-UCX.yaml
  • tests/scripts/perf/disaggregated/gb200_kimi-k25-thinking-fp4_1k1k_con4_ctx1_dep4_gen1_tep4_eplb0_mtp0_ccb-NIXL.yaml
  • tests/scripts/perf/disaggregated/gb200_kimi-k25-thinking-fp4_8k1k_con1024_ctx1_dep4_gen1_dep32_eplb416_mtp3_ccb-NIXL.yaml
  • tests/scripts/perf/disaggregated/gb200_kimi-k25-thinking-fp4_8k1k_con4096_ctx1_dep4_gen1_dep16_eplb0_mtp0_ccb-NIXL.yaml
  • tests/scripts/perf/disaggregated/gb200_kimi-k25-thinking-fp4_8k1k_con4096_ctx1_dep4_gen1_dep16_eplb0_mtp0_ccb-UCX.yaml
  • tests/scripts/perf/disaggregated/gb200_kimi-k25-thinking-fp4_8k1k_con4_ctx1_dep4_gen1_tep8_eplb0_mtp3_ccb-NIXL.yaml
  • tests/scripts/perf/disaggregated/gb300_kimi-k25-thinking-fp4_1k1k_con2048_ctx1_dep4_gen1_dep32_eplb0_mtp0_ccb-NIXL.yaml
  • tests/scripts/perf/disaggregated/gb300_kimi-k25-thinking-fp4_1k1k_con4096_ctx1_dep4_gen1_dep8_eplb0_mtp0_ccb-NIXL.yaml
  • tests/scripts/perf/disaggregated/gb300_kimi-k25-thinking-fp4_1k1k_con4096_ctx1_dep4_gen1_dep8_eplb0_mtp0_ccb-UCX.yaml
  • tests/scripts/perf/disaggregated/gb300_kimi-k25-thinking-fp4_1k1k_con4_ctx1_dep4_gen1_tep4_eplb0_mtp0_ccb-NIXL.yaml
  • tests/scripts/perf/disaggregated/gb300_kimi-k25-thinking-fp4_8k1k_con1024_ctx1_dep4_gen1_dep32_eplb416_mtp3_ccb-NIXL.yaml
  • tests/scripts/perf/disaggregated/gb300_kimi-k25-thinking-fp4_8k1k_con4096_ctx1_dep4_gen1_dep16_eplb0_mtp0_ccb-NIXL.yaml
  • tests/scripts/perf/disaggregated/gb300_kimi-k25-thinking-fp4_8k1k_con4096_ctx1_dep4_gen1_dep16_eplb0_mtp0_ccb-UCX.yaml
  • tests/scripts/perf/disaggregated/gb300_kimi-k25-thinking-fp4_8k1k_con4_ctx1_dep4_gen1_tep8_eplb0_mtp3_ccb-NIXL.yaml
💤 Files with no reviewable changes (16)
  • tests/integration/test_lists/test-db/l0_gb300_multi_gpus_perf_sanity.yml
  • tests/scripts/perf/disaggregated/gb200_kimi-k25-thinking-fp4_1k1k_con4_ctx1_dep4_gen1_tep4_eplb0_mtp0_ccb-NIXL.yaml
  • tests/scripts/perf/disaggregated/gb200_kimi-k25-thinking-fp4_8k1k_con4_ctx1_dep4_gen1_tep8_eplb0_mtp3_ccb-NIXL.yaml
  • tests/scripts/perf/disaggregated/gb300_kimi-k25-thinking-fp4_1k1k_con4_ctx1_dep4_gen1_tep4_eplb0_mtp0_ccb-NIXL.yaml
  • tests/scripts/perf/disaggregated/gb300_kimi-k25-thinking-fp4_1k1k_con4096_ctx1_dep4_gen1_dep8_eplb0_mtp0_ccb-UCX.yaml
  • tests/scripts/perf/disaggregated/gb300_kimi-k25-thinking-fp4_8k1k_con4096_ctx1_dep4_gen1_dep16_eplb0_mtp0_ccb-UCX.yaml
  • tests/scripts/perf/disaggregated/gb200_kimi-k25-thinking-fp4_1k1k_con4096_ctx1_dep4_gen1_dep8_eplb0_mtp0_ccb-UCX.yaml
  • tests/scripts/perf/disaggregated/gb200_kimi-k25-thinking-fp4_8k1k_con4096_ctx1_dep4_gen1_dep16_eplb0_mtp0_ccb-UCX.yaml
  • tests/integration/test_lists/qa/llm_function_core.txt
  • tests/scripts/perf/disaggregated/gb300_kimi-k25-thinking-fp4_8k1k_con4_ctx1_dep4_gen1_tep8_eplb0_mtp3_ccb-NIXL.yaml
  • tests/integration/test_lists/test-db/l0_gb200_multi_nodes_perf_sanity_ctx1_node1_gpu4_gen1_node2_gpu8.yml
  • tests/integration/test_lists/test-db/l0_gb200_multi_gpus_perf_sanity.yml
  • tests/integration/test_lists/test-db/l0_gb300_multi_nodes_perf_sanity_ctx1_node1_gpu4_gen1_node2_gpu8.yml
  • tests/integration/test_lists/test-db/l0_gb300_multi_nodes_perf_sanity_ctx1_node1_gpu4_gen1_node1_gpu4.yml
  • tests/integration/test_lists/test-db/l0_gb200_multi_nodes_perf_sanity_ctx1_node1_gpu4_gen1_node1_gpu4.yml
  • tests/integration/test_lists/qa/llm_perf_multinode.txt

Comment thread tests/integration/defs/accuracy/test_disaggregated_serving.py
@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #59657 [ run ] triggered by Bot. Commit: 853df7a Link to invocation

@Shixiaowei02
Shixiaowei02 force-pushed the dev/xiaoweis/kimi-disagg-py-transceiver branch from 853df7a to ef95e3a Compare July 16, 2026 07:31
@Shixiaowei02

Copy link
Copy Markdown
Collaborator Author

/bot run --stage-list "DGX_B200-8_GPUs-PyTorch-1,DGX_B200-8_GPUs-PyTorch-2,DGX_B200-8_GPUs-PyTorch-3,DGX_B200-8_GPUs-PyTorch-4"

@Shixiaowei02

Copy link
Copy Markdown
Collaborator Author

/bot kill

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #59662 [ run ] triggered by Bot. Commit: ef95e3a Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #59664 [ kill ] triggered by Bot. Commit: ef95e3a Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #59657 [ run ] completed with state ABORTED. Commit: 853df7a

Link to invocation

@Shixiaowei02
Shixiaowei02 force-pushed the dev/xiaoweis/kimi-disagg-py-transceiver branch from ef95e3a to d1be051 Compare July 16, 2026 07:56
@Shixiaowei02
Shixiaowei02 requested a review from a team as a code owner July 16, 2026 07:56
@Shixiaowei02
Shixiaowei02 requested a review from nv-guomingz July 16, 2026 07:56
@Shixiaowei02
Shixiaowei02 force-pushed the dev/xiaoweis/kimi-disagg-py-transceiver branch from da7dcf0 to bc28173 Compare July 20, 2026 02:28
@Shixiaowei02

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #60221 [ run ] triggered by Bot. Commit: bc28173 Link to invocation

Mirror the GPT-OSS approach (NVIDIA#16479): KimiK25ForConditionalGeneration now
declares "PYTHON" as its preferred KV-cache transceiver runtime, so
disaggregated Kimi-K2.5 adopts the Python (v2) transceiver by default when the
user leaves transceiver_runtime at 'auto' and the effective backend is NIXL.
Explicit user settings and non-NIXL backends are unaffected.

- modeling_kimi_k25.py: add get_preferred_transceiver_runtime() -> "PYTHON".
- test_modeling_kimi_k25.py: unit test asserting the preference.
- TestKimiK25::test_nvfp4: set cache_transceiver_config explicitly to
  backend=NIXL + transceiver_runtime=PYTHON (ctx and gen). The disagg test
  harness injects TRTLLM_USE_UCX_KVCACHE=1 for any non-NIXL backend, so a
  "DEFAULT" backend would resolve to UCX and force the C++ transceiver; the
  test sets NIXL explicitly so it actually exercises the Python transceiver
  rather than relying on the model-default resolution.
- Remove the redundant TestKimiK2::test_nvfp4 disaggregated accuracy test;
  Kimi disaggregated accuracy is covered by TestKimiK25, and Kimi-K2-Thinking
  accuracy stays covered by the aggregated tests in test_llm_api_pytorch.

Signed-off-by: Shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>
@Shixiaowei02
Shixiaowei02 force-pushed the dev/xiaoweis/kimi-disagg-py-transceiver branch from bc28173 to 0089eeb Compare July 20, 2026 02:58
@Shixiaowei02

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #60221 [ run ] completed with state FAILURE. Commit: bc28173
/LLM/main/L0_MergeRequest_PR pipeline #48589 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #60233 [ run ] triggered by Bot. Commit: 0089eeb Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #60233 [ run ] completed with state SUCCESS. Commit: 0089eeb
/LLM/main/L0_MergeRequest_PR pipeline #48599 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@Shixiaowei02

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast --stage-list "GH200-PackageSanityCheck-PY312-UB2404, A10-PyTorch-1, B300-PyTorch-1, H100_PCIe-AutoDeploy-1"

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #60354 [ run ] triggered by Bot. Commit: 0089eeb Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #60354 [ run ] completed with state SUCCESS. Commit: 0089eeb
/LLM/main/L0_MergeRequest_PR pipeline #48697 (Partly Tested) completed with status: 'SUCCESS'

CI Report

Link to invocation

@Shixiaowei02

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast --only-multi-gpu-test

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #60404 [ run ] triggered by Bot. Commit: 0089eeb Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #60404 [ run ] completed with state FAILURE. Commit: 0089eeb
/LLM/main/L0_MergeRequest_PR pipeline #48742 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@Shixiaowei02

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast --stage-list "DGX_B300-4_GPUs-PyTorch-1, GB200-4_GPUs-PyTorch-2, GB200-4_GPUs-PyTorch-3, GB200-4_GPUs-PyTorch-5, GB200-4_GPUs-PyTorch-PerfSanity-1, GB200-4_GPUs-PyTorch-PerfSanity-2, GB200-8_GPUs-2_Nodes-PyTorch-1"

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #60536 [ run ] triggered by Bot. Commit: 0089eeb Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #60536 [ run ] completed with state SUCCESS. Commit: 0089eeb
/LLM/main/L0_MergeRequest_PR pipeline #48855 (Partly Tested) completed with status: 'SUCCESS'

CI Report

Link to invocation

@Shixiaowei02

Copy link
Copy Markdown
Collaborator Author

/bot skip --comment "Related tests have passed."

@Shixiaowei02
Shixiaowei02 enabled auto-merge (squash) July 21, 2026 07:25
@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #60631 [ skip ] triggered by Bot. Commit: 0089eeb Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #60631 [ skip ] completed with state SUCCESS. Commit: 0089eeb
Skipping testing for commit 0089eeb

Link to invocation

@Shixiaowei02
Shixiaowei02 merged commit 9390924 into NVIDIA:main Jul 21, 2026
8 checks passed
@Shixiaowei02
Shixiaowei02 deleted the dev/xiaoweis/kimi-disagg-py-transceiver branch July 21, 2026 08:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants