[None][feat] Default GLM-5 to the Python KV-cache transceiver (duplicate of #16524 for parallel CI) by chuangz0 · Pull Request #16656 · NVIDIA/TensorRT-LLM

chuangz0 · 2026-07-21T03:46:33Z

…in disagg

GLM-5 and GLM-5.2 (GlmMoeDsaForCausalLM / glm_moe_dsa) share the DeepseekV3ForCausalLM implementation class; override get_preferred_transceiver_runtime() to opt GLM checkpoints into the Python (v2) transceiver, differentiated per checkpoint so DeepSeek-V3/ V3.2 keep the C++ default. The preference is adopted only when the user leaves cache_transceiver_config.transceiver_runtime at 'auto' and the effective backend is NIXL.

Set transceiver_runtime: PYTHON explicitly in the GLM-5 perf-sanity disaggregated NIXL configs, and switch the GLM-5 disagg stress-test config from backend DEFAULT to NIXL + PYTHON so the TRTLLM_USE_UCX_KVCACHE=1 fallback cannot silently revert it to the C++ transceiver.

Add unit tests covering the per-architecture preference and the end-to-end 'auto' resolution for GLM-5 on NIXL.

Summary by CodeRabbit

New Features
- GLM-5-family models now automatically prefer the Python cache-transceiver runtime when using NIXL.
- Added explicit Python runtime settings across GLM-5 disaggregated serving and performance configurations.
Tests
- Added coverage for GLM-5 runtime selection, DeepSeek V3 behavior, missing configuration handling, and end-to-end runtime resolution.

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

…in disagg GLM-5 and GLM-5.2 (GlmMoeDsaForCausalLM / glm_moe_dsa) share the DeepseekV3ForCausalLM implementation class; override get_preferred_transceiver_runtime() to opt GLM checkpoints into the Python (v2) transceiver, differentiated per checkpoint so DeepSeek-V3/ V3.2 keep the C++ default. The preference is adopted only when the user leaves cache_transceiver_config.transceiver_runtime at 'auto' and the effective backend is NIXL. Set transceiver_runtime: PYTHON explicitly in the GLM-5 perf-sanity disaggregated NIXL configs, and switch the GLM-5 disagg stress-test config from backend DEFAULT to NIXL + PYTHON so the TRTLLM_USE_UCX_KVCACHE=1 fallback cannot silently revert it to the C++ transceiver. Add unit tests covering the per-architecture preference and the end-to-end 'auto' resolution for GLM-5 on NIXL. Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>

…ansceiver test Address review: the GLM-positive case satisfied both predicates at once, so deleting either production check still passed. Add one case per predicate in isolation (GLM architecture with a non-GLM model_type, and a non-GLM architecture with model_type=glm_moe_dsa). Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>

chuangz0 · 2026-07-21T03:53:34Z

/bot run --stage-list "A10-PyTorch-1, A10-PyTorch-2"

coderabbitai · 2026-07-21T03:56:23Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: d0801be8-c4f8-4b8c-a22e-d11f62db2f62

📥 Commits

Reviewing files that changed from the base of the PR and between 002f099 and 2eb5072.

📒 Files selected for processing (15)

tensorrt_llm/_torch/models/modeling_deepseekv3.py
tests/integration/defs/disaggregated/test_configs/disagg_config_ctxtp4ep4_gentp4ep4_glm5_nvfp4_dp_tllm.yaml
tests/scripts/perf-sanity/disaggregated/gb200_glm-5-fp4_1k1k_con1_ctx1_dep4_gen1_tep4_eplb0_mtp3_ccb-NIXL.yaml
tests/scripts/perf-sanity/disaggregated/gb200_glm-5-fp4_1k1k_con4096_ctx1_dep4_gen1_dep8_eplb256_mtp1_ccb-NIXL.yaml
tests/scripts/perf-sanity/disaggregated/gb200_glm-5-fp4_1k1k_con512_ctx1_dep4_gen1_dep32_eplb0_mtp3_ccb-NIXL.yaml
tests/scripts/perf-sanity/disaggregated/gb200_glm-5-fp4_8k1k_con1024_ctx1_dep4_gen1_dep8_eplb256_mtp1_ccb-NIXL.yaml
tests/scripts/perf-sanity/disaggregated/gb200_glm-5-fp4_8k1k_con1_ctx1_dep4_gen1_tep8_eplb0_mtp3_ccb-NIXL.yaml
tests/scripts/perf-sanity/disaggregated/gb200_glm-5-fp4_8k1k_con512_ctx1_dep4_gen1_dep32_eplb0_mtp3_ccb-NIXL.yaml
tests/scripts/perf-sanity/disaggregated/gb300_glm-5-fp4_1k1k_con1_ctx1_dep2_gen1_tep4_eplb0_mtp3_ccb-NIXL.yaml
tests/scripts/perf-sanity/disaggregated/gb300_glm-5-fp4_1k1k_con4096_ctx1_dep2_gen1_dep8_eplb256_mtp1_ccb-NIXL.yaml
tests/scripts/perf-sanity/disaggregated/gb300_glm-5-fp4_1k1k_con512_ctx1_dep2_gen1_dep32_eplb0_mtp3_ccb-NIXL.yaml
tests/scripts/perf-sanity/disaggregated/gb300_glm-5-fp4_8k1k_con1024_ctx1_dep2_gen1_dep8_eplb256_mtp1_ccb-NIXL.yaml
tests/scripts/perf-sanity/disaggregated/gb300_glm-5-fp4_8k1k_con1_ctx1_dep2_gen1_tep8_eplb0_mtp3_ccb-NIXL.yaml
tests/scripts/perf-sanity/disaggregated/gb300_glm-5-fp4_8k1k_con512_ctx1_dep2_gen1_dep32_eplb0_mtp3_ccb-NIXL.yaml
tests/unittest/llmapi/test_llm_args.py

📝 Walkthrough

Walkthrough

Adds GLM-5 detection that prefers the PYTHON KV-cache transceiver runtime, validates automatic resolution, and updates disaggregated integration and performance configurations to use NIXL with the Python runtime.

Changes

GLM-5 transceiver runtime selection

Layer / File(s)	Summary
Model preference and resolution tests `tensorrt_llm/_torch/models/modeling_deepseekv3.py`, `tests/unittest/llmapi/test_llm_args.py`	Detects GLM-5 architecture or model type and returns `PYTHON`; tests cover matching, non-matching, absent configurations, and automatic resolution.
Integration configuration wiring `tests/integration/defs/disaggregated/test_configs/...`	Sets context and generation server transceivers to `NIXL` with `transceiver_runtime: PYTHON`.
Performance configuration wiring `tests/scripts/perf-sanity/disaggregated/gb200_.yaml`, `tests/scripts/perf-sanity/disaggregated/gb300_.yaml`	Adds `transceiver_runtime: PYTHON` to generation and context cache transceiver configurations.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

NVIDIA/TensorRT-LLM#16524: Adds the same GLM-5 transceiver preference and related NIXL configuration updates.

Suggested labels: api-compatible

Suggested reviewers: chenfeiz0326, nv-xtf

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 40.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is concise and accurately describes the main change to default GLM-5 to the Python KV-cache transceiver.
Description check	✅ Passed	The description covers the issue, solution, and test coverage, though it does not use the template headings explicitly.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands.}

tensorrt-cicd · 2026-07-21T03:58:46Z

PR_Github #60566 [ run ] triggered by Bot. Commit: 49fe117 Link to invocation

tensorrt-cicd · 2026-07-21T05:59:01Z

PR_Github #60566 [ run ] completed with state SUCCESS. Commit: 49fe117
/LLM/main/L0_MergeRequest_PR pipeline #48878 (Partly Tested) completed with status: 'SUCCESS'

CI Report

Link to invocation

chuangz0 requested review from a team as code owners July 21, 2026 03:46

chuangz0 requested review from JunyiXu-nv, LarryXFly, asfiyab-nvidia, aswinvisva, chienchunhung, dc3671, nv-ananjappa, reasonsolo and xinhe-nv July 21, 2026 03:46

github-actions Bot assigned chuangz0 Jul 21, 2026

chuangz0 added 2 commits July 21, 2026 11:52

chuangz0 force-pushed the glm_model_default_python_transceiver_dup branch from 2eb5072 to 49fe117 Compare July 21, 2026 03:52

xinhe-nv approved these changes Jul 23, 2026

View reviewed changes

chuangz0 closed this Jul 23, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[None][feat] Default GLM-5 to the Python KV-cache transceiver (duplicate of #16524 for parallel CI) - #16656

[None][feat] Default GLM-5 to the Python KV-cache transceiver (duplicate of #16524 for parallel CI)#16656
chuangz0 wants to merge 2 commits into
NVIDIA:mainfrom
chuangz0:glm_model_default_python_transceiver_dup

chuangz0 commented Jul 21, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

chuangz0 commented Jul 21, 2026

Uh oh!

coderabbitai Bot commented Jul 21, 2026

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

tensorrt-cicd commented Jul 21, 2026

Uh oh!

tensorrt-cicd commented Jul 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

chuangz0 commented Jul 21, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Description

Test Coverage

PR Checklist

GitHub Bot Help

Uh oh!

chuangz0 commented Jul 21, 2026

Uh oh!

coderabbitai Bot commented Jul 21, 2026

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

tensorrt-cicd commented Jul 21, 2026

Uh oh!

tensorrt-cicd commented Jul 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

chuangz0 commented Jul 21, 2026 •

edited by coderabbitai Bot

Loading