[TRTLLM-12720][feat] Support nvfp4 w4a16 on sm120 - #16511
[TRTLLM-12720][feat] Support nvfp4 w4a16 on sm120#16511pamelap-nvidia wants to merge 17 commits into
Conversation
Signed-off-by: Pamela <179191831+pamelap-nvidia@users.noreply.github.com>
Signed-off-by: Pamela <179191831+pamelap-nvidia@users.noreply.github.com>
Signed-off-by: Pamela <179191831+pamelap-nvidia@users.noreply.github.com>
Signed-off-by: Pamela <179191831+pamelap-nvidia@users.noreply.github.com>
Signed-off-by: Pamela <179191831+pamelap-nvidia@users.noreply.github.com>
Signed-off-by: Pamela <179191831+pamelap-nvidia@users.noreply.github.com>
Signed-off-by: Pamela <179191831+pamelap-nvidia@users.noreply.github.com>
Signed-off-by: Pamela <179191831+pamelap-nvidia@users.noreply.github.com>
Signed-off-by: Pamela <179191831+pamelap-nvidia@users.noreply.github.com>
Signed-off-by: Pamela <179191831+pamelap-nvidia@users.noreply.github.com>
Signed-off-by: Pamela <179191831+pamelap-nvidia@users.noreply.github.com>
Signed-off-by: Pamela <179191831+pamelap-nvidia@users.noreply.github.com>
pamelap-nvidia
changed the title
Signed-off-by: Pamela <179191831+pamelap-nvidia@users.noreply.github.com>
Signed-off-by: Pamela <179191831+pamelap-nvidia@users.noreply.github.com>
📝 WalkthroughWalkthroughThe change adds W4A16 NVFP4 CUDA-core and MoE execution support for SM12x, extends Marlin and XQA architecture handling, propagates per-layer quantization configuration, updates checkpoint mapping, and adds unit and integration coverage. ChangesNVFP4 kernel and XQA support
Quantization and model execution
Validation
Estimated code review effort: 5 (Critical) | ~120 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (3)
cpp/tensorrt_llm/kernels/weightOnlyBatchedGemv/nvfp4ScaleLayout.h (1)
43-44: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueName the sub-tile swizzle literals. The
32,16, and4in the return expression are unexplained magic numbers describing the 128×4 SF sub-tile layout. Extractingk-prefixed named constants (e.g.kScaleRowsPerSubBlock = 32) would document intent and satisfy the magic-literal rule.As per coding guidelines: "Avoid magic literals except
0,nullptr,true, andfalse; initialize named constants instead, usingk-prefixed camelCase names."🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@cpp/tensorrt_llm/kernels/weightOnlyBatchedGemv/nvfp4ScaleLayout.h` around lines 43 - 44, Replace the unexplained 32, 16, and 4 literals in the scale-offset return expression with k-prefixed named constants documenting the 128×4 SF sub-tile layout, such as the sub-block row count and corresponding row/column strides. Define the constants in the surrounding scope and preserve the existing arithmetic and behavior.Source: Coding guidelines
tensorrt_llm/_torch/modules/linear.py (1)
2054-2066: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick winGate the CUDA-core fast path when NCCL-window output is active
W4A16NVFP4LinearMethodstill returns a freshly allocated tensor here, so this branch drops out of the zero-copy window path. If that optimization matters, skip the fast path whenuses_nccl_symmetric_memory_window()is true.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tensorrt_llm/_torch/modules/linear.py` around lines 2054 - 2066, Update W4A16NVFP4LinearMethod.apply so the _can_use_cuda_core fast path is selected only when uses_nccl_symmetric_memory_window() is false; otherwise bypass this branch and preserve the existing NCCL-window output path.tensorrt_llm/_torch/pyexecutor/mamba_cache_manager.py (1)
1861-1865: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winAdd boundary coverage for recurrent-state admission. Existing tests cover recurrent-pool sizing, but not the new
get_num_available_tokens()clamp. Add a direct case forrs_free = 0, 1, 2and assert the cap is0,interval, and2 * interval, including the parentmin()path.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tensorrt_llm/_torch/pyexecutor/mamba_cache_manager.py` around lines 1861 - 1865, Add direct boundary tests for get_num_available_tokens() covering rs_free values 0, 1, and 2; assert the recurrent-state cap is respectively 0, interval, and 2 * interval, and verify the final result also respects the parent min() path.Source: Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@cpp/tests/unit_tests/kernels/decoderXQAImplCommonTest.cpp`:
- Around line 37-43: Update skipIfXqaBlocksPerSequenceIsForced so it no longer
invokes GTEST_SKIP from the helper; instead, make it return whether skipping is
required and have each affected TEST body call GTEST_SKIP() directly when that
result is true, ensuring the test body exits immediately.
In `@tests/unittest/_torch/modules/moe/test_cute_dsl_b12x_moe_backend.py`:
- Around line 170-194: Guard both b12x selection tests,
test_get_moe_cls_cutedsl_selects_b12x_for_w4a16_nvfp4_on_supported_sm and
test_get_moe_cls_cutedsl_selects_b12x_for_layer_w4a16_nvfp4_on_supported_sm,
with pytest.importorskip("flashinfer") before invoking get_moe_cls so they are
skipped when the dependency is unavailable.
---
Nitpick comments:
In `@cpp/tensorrt_llm/kernels/weightOnlyBatchedGemv/nvfp4ScaleLayout.h`:
- Around line 43-44: Replace the unexplained 32, 16, and 4 literals in the
scale-offset return expression with k-prefixed named constants documenting the
128×4 SF sub-tile layout, such as the sub-block row count and corresponding
row/column strides. Define the constants in the surrounding scope and preserve
the existing arithmetic and behavior.
In `@tensorrt_llm/_torch/modules/linear.py`:
- Around line 2054-2066: Update W4A16NVFP4LinearMethod.apply so the
_can_use_cuda_core fast path is selected only when
uses_nccl_symmetric_memory_window() is false; otherwise bypass this branch and
preserve the existing NCCL-window output path.
In `@tensorrt_llm/_torch/pyexecutor/mamba_cache_manager.py`:
- Around line 1861-1865: Add direct boundary tests for
get_num_available_tokens() covering rs_free values 0, 1, and 2; assert the
recurrent-state cap is respectively 0, interval, and 2 * interval, and verify
the final result also respects the parent min() path.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: cb34e8a3-98a1-407c-898e-4db581d04de6
📒 Files selected for processing (45)
cpp/kernels/fmha_v2/setup.pycpp/kernels/xqa/mha.cucpp/tensorrt_llm/common/attentionOp.cppcpp/tensorrt_llm/kernels/CMakeLists.txtcpp/tensorrt_llm/kernels/decoderMaskedMultiheadAttention/decoderXQARunner.cppcpp/tensorrt_llm/kernels/decoderMaskedMultiheadAttention/decoderXQARunnerUtils.hcpp/tensorrt_llm/kernels/marlin/marlin_nvfp4_gemm.cucpp/tensorrt_llm/kernels/marlin/marlin_nvfp4_template.hcpp/tensorrt_llm/kernels/marlin/marlin_repack.cucpp/tensorrt_llm/kernels/weightOnlyBatchedGemv/cudaCoreGemmW4A16NVFP4.cucpp/tensorrt_llm/kernels/weightOnlyBatchedGemv/cudaCoreGemmW4A16NVFP4.hcpp/tensorrt_llm/kernels/weightOnlyBatchedGemv/nvfp4ScaleLayout.hcpp/tensorrt_llm/nanobind/bindings.cppcpp/tensorrt_llm/thop/CMakeLists.txtcpp/tensorrt_llm/thop/cublasScaledMMLut.hcpp/tensorrt_llm/thop/w4a16Nvfp4Gemm.cppcpp/tests/unit_tests/kernels/CMakeLists.txtcpp/tests/unit_tests/kernels/decoderXQAImplCommonTest.cpptensorrt_llm/_torch/attention_backend/trtllm.pytensorrt_llm/_torch/custom_ops/cpp_custom_ops.pytensorrt_llm/_torch/model_config.pytensorrt_llm/_torch/models/checkpoints/hf/nemotron_h_weight_mapper.pytensorrt_llm/_torch/models/modeling_nemotron_h.pytensorrt_llm/_torch/models/modeling_qwen3_5.pytensorrt_llm/_torch/modules/fused_moe/configurable_moe.pytensorrt_llm/_torch/modules/fused_moe/create_moe.pytensorrt_llm/_torch/modules/fused_moe/fused_moe_cute_dsl_b12x.pytensorrt_llm/_torch/modules/fused_moe/fused_moe_cutlass.pytensorrt_llm/_torch/modules/fused_moe/quantization.pytensorrt_llm/_torch/modules/linear.pytensorrt_llm/_torch/modules/mlp.pytensorrt_llm/_torch/pyexecutor/mamba_cache_manager.pytensorrt_llm/models/quant_config_utils.pytests/integration/defs/accuracy/test_llm_api_pytorch.pytests/integration/test_lists/test-db/l0_gb10.ymltests/integration/test_lists/test-db/l0_rtx_pro_6000.ymltests/unittest/_torch/modeling/test_modeling_nemotron_h_moe_quant.pytests/unittest/_torch/modeling/test_modeling_qwen3_5_quant.pytests/unittest/_torch/models/checkpoints/hf/test_nemotron_h_weight_mapper.pytests/unittest/_torch/models/checkpoints/hf/test_qwen3_5_weight_mapper.pytests/unittest/_torch/modules/moe/test_cute_dsl_b12x_moe_backend.pytests/unittest/_torch/modules/test_w4a16_nvfp4_linear.pytests/unittest/_torch/test_model_config.pytests/unittest/_torch/thop/serial/test_nvfp4_marlin_gemm.pytests/unittest/models/test_quant_config_utils.py
| void skipIfXqaBlocksPerSequenceIsForced() | ||
| { | ||
| if (tc::getEnvXqaBlocksPerSequence().has_value()) | ||
| { | ||
| GTEST_SKIP() << "TRTLLM_XQA_BLOCKS_PER_SEQUENCE overrides multi-block tuning."; | ||
| } | ||
| } |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
file='cpp/tests/unit_tests/kernels/decoderXQAImplCommonTest.cpp'
echo '--- file outline ---'
ast-grep outline "$file" --view expanded || true
echo
echo '--- relevant lines ---'
nl -ba "$file" | sed -n '1,220p'
echo
echo '--- helper usages ---'
rg -n "skipIfXqaBlocksPerSequenceIsForced|GTEST_SKIP\\(" "$file"Repository: NVIDIA/TensorRT-LLM
Length of output: 855
🏁 Script executed:
#!/bin/bash
set -euo pipefail
file='cpp/tests/unit_tests/kernels/decoderXQAImplCommonTest.cpp'
echo '--- relevant slice ---'
sed -n '1,140p' "$file"Repository: NVIDIA/TensorRT-LLM
Length of output: 3353
🌐 Web query:
GoogleTest GTEST_SKIP helper function returns current test skip semantics
💡 Result:
The GTEST_SKIP construct in GoogleTest is a macro, not a function [1]. It is specifically implemented as a macro because it must terminate the execution of the current test body immediately by returning from the scope where it is called [1]. If it were a standard function, it would be unable to return from the calling test body without throwing an exception or using other non-standard mechanisms, which would fail to prevent subsequent assertions from executing [1]. When invoked, GTEST_SKIP prevents further execution of the current test at runtime [2][3]. Its behavior and usage include: Usage Locations: It can be used within individual test cases (TEST or TEST_F) or in the SetUp methods of test fixtures (classes derived from::testing::Test) or global test environments (classes derived from::testing::Environment) [2][3]. Impact: If used in a global environment's SetUp method, it skips all tests in the test program [2]. If used in a test fixture's SetUp method, it skips all tests within that specific test suite [2]. Streaming Messages: Similar to GoogleTest assertion macros, you can stream a custom message into GTEST_SKIP using the << operator to provide context for why the test was skipped [2][3]. Lifecycle: Even when a test is skipped via GTEST_SKIP, the TearDown method for that test fixture or environment will still be executed [3].
Citations:
- 1: Skipping tests at runtime with GTEST_SKIP() google/googletest#1544
- 2: https://github.com/google/googletest/blob/main/docs/reference/testing.md
- 3: https://github.com/google/googletest/blob/main/docs/advanced.md
🌐 Web query:
GoogleTest GTEST_SKIP macro expansion returns from current function and marks test skipped
💡 Result:
The GTEST_SKIP macro in GoogleTest is designed to prevent further test execution at runtime [1][2]. When invoked, it internally throws an exception to interrupt the execution flow, effectively marking the current test, test fixture, or global test environment as skipped [3][4]. Key behaviors include: - Execution Flow: When GTEST_SKIP is called, it stops the execution of the current function (e.g., a test case or SetUp method) [1][2]. However, it is important to note that if you call GTEST_SKIP within a SetUp method, subsequent code in that same SetUp method—or in derived classes that call the base SetUp—may still execute unless you explicitly return or structure your code to prevent it [3]. - Scope: - Individual Tests: Skips the specific test case where it is called [2]. - Test Fixtures: If called in a fixture's SetUp method, it skips all tests within that test suite [1][5]. - Global Environments: If called in a global test environment's SetUp method, it skips all tests in the test program [1][5]. - Teardown: TearDown methods will still be executed even if a test is skipped via GTEST_SKIP [2][6]. - Integration: Similar to other GoogleTest assertions, you can stream a custom message into the macro (e.g., GTEST_SKIP << "Reason for skipping";) to provide diagnostic information [1][2]. To avoid issues where logic continues to run after a skip, it is recommended to explicitly return after calling GTEST_SKIP or to use architectural patterns (such as delegating logic to a secondary method only after preconditions are met) to ensure that code execution truly halts [3].
Citations:
- 1: https://github.com/google/googletest/blob/main/docs/reference/testing.md
- 2: https://github.com/google/googletest/blob/HEAD/docs/advanced.md
- 3: Recommended pattern for GTEST_SKIP called from SetUp in a base class and dealing with that in derived classes google/googletest#4879
- 4: GTEST_SKIP doesn't work in test environments google/googletest#2189
- 5: http://google.github.io/googletest/reference/testing.html
- 6: https://github.com/google/googletest/blob/main/docs/advanced.md
🌐 Web query:
GoogleTest GTEST_SKIP macro definition source
💡 Result:
The GTEST_SKIP macro in GoogleTest is defined in the internal header file gtest-internal.h [1][2]. It is defined as follows: #define GTEST_SKIP_(message) \ return GTEST_MESSAGE_(message,::testing::TestPartResult::kSkip) The macro expands to a return statement that utilizes GTEST_MESSAGE_ to report a test result of type kSkip [1][2]. This mechanism allows the macro to effectively exit the current function or test body scope when invoked, preventing any subsequent code within that scope from executing [3][4]. It is designed to be used in test cases, as well as in the SetUp methods of test fixtures and global environments, to skip execution based on runtime conditions [5][4].
Citations:
- 1: https://raw.githubusercontent.com/google/googletest/master/googletest/include/gtest/internal/gtest-internal.h
- 2: https://chromium.googlesource.com/external/github.com/google/googletest/+/HEAD/googletest/include/gtest/internal/gtest-internal.h
- 3: Skipping tests at runtime with GTEST_SKIP() google/googletest#1544
- 4: https://github.com/google/googletest/blob/6ec14dfd/docs/advanced.md
- 5: http://google.github.io/googletest/reference/testing.html
Move the skip into each test body. GTEST_SKIP() only returns from skipIfXqaBlocksPerSequenceIsForced(), so the TEST body still continues after the helper returns. Inline the guard or have the helper return a bool and call GTEST_SKIP() directly in the caller.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@cpp/tests/unit_tests/kernels/decoderXQAImplCommonTest.cpp` around lines 37 -
43, Update skipIfXqaBlocksPerSequenceIsForced so it no longer invokes GTEST_SKIP
from the helper; instead, make it return whether skipping is required and have
each affected TEST body call GTEST_SKIP() directly when that result is true,
ensuring the test body exits immediately.
| @pytest.mark.parametrize("sm_version", sorted(CuteDslB12xFusedMoE._SUPPORTED_SM_VERSIONS)) | ||
| def test_get_moe_cls_cutedsl_selects_b12x_for_w4a16_nvfp4_on_supported_sm(sm_version): | ||
| """CUTEDSL + W4A16_NVFP4 + SM120/121 + flashinfer importable → CuteDslB12xFusedMoE.""" | ||
| cfg = ModelConfig() | ||
| cfg.moe_backend = "CUTEDSL" | ||
| cfg.quant_config = QuantConfig(quant_algo=QuantAlgo.W4A16_NVFP4) | ||
| with patch("tensorrt_llm._utils.get_sm_version", return_value=sm_version): | ||
| cls = get_moe_cls(cfg) | ||
| assert cls is CuteDslB12xFusedMoE | ||
|
|
||
|
|
||
| @pytest.mark.parametrize("sm_version", sorted(CuteDslB12xFusedMoE._SUPPORTED_SM_VERSIONS)) | ||
| def test_get_moe_cls_cutedsl_selects_b12x_for_layer_w4a16_nvfp4_on_supported_sm(sm_version): | ||
| """MIXED_PRECISION per-layer W4A16 must select the same backend that the | ||
| layer will use after apply_layerwise_quant_config(). | ||
| """ | ||
| cfg = ModelConfig() | ||
| cfg.moe_backend = "CUTEDSL" | ||
| cfg.quant_config = QuantConfig() | ||
| cfg.quant_config_dict = { | ||
| "model.layers.0.mlp.experts": QuantConfig(quant_algo=QuantAlgo.W4A16_NVFP4, group_size=16), | ||
| } | ||
| with patch("tensorrt_llm._utils.get_sm_version", return_value=sm_version): | ||
| cls = get_moe_cls(cfg, layer_idx=0) | ||
| assert cls is CuteDslB12xFusedMoE |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Is flashinfer a hard dependency, or optional? Check requirements/deps.
fd -t f -e txt -e toml -e cfg | xargs rg -nI 'flashinfer' 2>/dev/null
# How do other unit tests treat flashinfer availability?
rg -nP 'importorskip\(["'\'']flashinfer' -g '!**/site-packages/**'Repository: NVIDIA/TensorRT-LLM
Length of output: 1827
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect the test file and nearby flashinfer-related tests.
file=tests/unittest/_torch/modules/moe/test_cute_dsl_b12x_moe_backend.py
sed -n '1,260p' "$file"
printf '\n--- SEARCH importorskip / sys.modules flashinfer usage ---\n'
rg -n 'importorskip\(["'\'']flashinfer|sys\.modules.*flashinfer|flashinfer' tests/unittest -g '!**/site-packages/**'
printf '\n--- Inspect get_moe_cls implementation ---\n'
rg -n 'def get_moe_cls|import flashinfer|flashinfer' tensorrt_llm/_torch -g '!**/site-packages/**'Repository: NVIDIA/TensorRT-LLM
Length of output: 50376
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Check dependency declarations for flashinfer.
rg -n 'flashinfer-python|flashinfer' pyproject.toml requirements* setup* README* docs* -g '!**/site-packages/**'Repository: NVIDIA/TensorRT-LLM
Length of output: 4838
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- create_moe.get_moe_cls ---'
file=tensorrt_llm/_torch/modules/fused_moe/create_moe.py
sed -n '1,260p' "$file"
printf '\n%s\n' '--- relevant section of the test file ---'
file=tests/unittest/_torch/modules/moe/test_cute_dsl_b12x_moe_backend.py
sed -n '150,250p' "$file"Repository: NVIDIA/TensorRT-LLM
Length of output: 16817
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
from pathlib import Path
p = Path('tests/unittest/_torch/modules/moe/test_cute_dsl_b12x_moe_backend.py')
for i, line in enumerate(p.read_text().splitlines(), 1):
if 155 <= i <= 245:
print(f"{i}:{line}")
PYRepository: NVIDIA/TensorRT-LLM
Length of output: 4019
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- create_moe.get_moe_cls ---'
file=tensorrt_llm/_torch/modules/fused_moe/create_moe.py
sed -n '1,260p' "$file"
printf '\n%s\n' '--- relevant section of the test file ---'
file=tests/unittest/_torch/modules/moe/test_cute_dsl_b12x_moe_backend.py
sed -n '150,245p' "$file"Repository: NVIDIA/TensorRT-LLM
Length of output: 16581
Guard the b12x selection tests on flashinfer availability
get_moe_cls() imports flashinfer on this path and falls back to CuteDslFusedMoE/CutlassFusedMoE when it is missing, so these assertions are brittle in environments without the dependency. Add pytest.importorskip("flashinfer") or mock the import here.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/unittest/_torch/modules/moe/test_cute_dsl_b12x_moe_backend.py` around
lines 170 - 194, Guard both b12x selection tests,
test_get_moe_cls_cutedsl_selects_b12x_for_w4a16_nvfp4_on_supported_sm and
test_get_moe_cls_cutedsl_selects_b12x_for_layer_w4a16_nvfp4_on_supported_sm,
with pytest.importorskip("flashinfer") before invoking get_moe_cls so they are
skipped when the dependency is unavailable.
Signed-off-by: Pamela <179191831+pamelap-nvidia@users.noreply.github.com>
|
/bot run |
|
PR_Github #60584 [ run ] triggered by Bot. Commit: |
|
PR_Github #60584 [ run ] completed with state
|
Signed-off-by: Pamela <179191831+pamelap-nvidia@users.noreply.github.com>
|
/bot run |
|
PR_Github #60716 [ run ] triggered by Bot. Commit: |
|
PR_Github #60716 [ run ] completed with state
|
|
/bot run |
|
PR_Github #60774 [ run ] triggered by Bot. Commit: |
|
PR_Github #60774 [ run ] completed with state
|
2 participants
Summary by CodeRabbit
New Features
Bug Fixes
Description
Add PyTorch backend support for W4A16 NVFP4 inference on SM120/121, including dense linear, LM-head, and MoE execution paths.
Key changes:
w4a16_nvfp4_gemmCUDA-core kernel and Torch operator for small-M workloads on SM120/121.W4A16NVFP4LinearMethodas the common W4A16 implementation:Test Coverage
Added or extended coverage for:
sf_scaleregression.Local validation:
PR Checklist
Please review the following before submitting your PR:
PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either
api-compatibleorapi-breaking. Forapi-breaking, includeBREAKINGin the PR title.Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.
GitHub Bot Help
To see a list of available CI bot commands, please comment
/bot help.