[#10966][feat] AutoDeploy: kv cache manager integration [2/2] - #11149
Conversation
KV Cache Architecture in
|
| Handler Type | Managed By | Buffer Source | Use Case |
|---|---|---|---|
KVPagedResourceHandler |
KVCacheManager |
get_buffers(idx) |
Paged KV caches for attention |
SSMResourceHandler |
MambaHybridCacheManager |
get_ssm_states(layer) |
Mamba SSM state |
CausalConvResourceHandler |
MambaHybridCacheManager |
get_conv_states(layer) |
Mamba causal conv state |
StateResourceHandler |
Local allocation | handler.allocate() |
Generic per-sequence state |
UnpagedResourceHandler |
Local allocation | handler.allocate() |
Unpaged per-token resources |
Key Files and Their Responsibilities
1. custom_ops/attention_interface.py
ResourceHandler(abstract base): Interface for allocating resourcesKVPagedResourceHandler: Describes paged KV cache withnum_kv_heads,head_dim,dtype,kv_layoutSSMResourceHandler: Describes Mamba SSM state withnum_heads,head_dim,d_stateCausalConvResourceHandler: Describes causal conv state withconv_dim,d_convAttentionDescriptor.get_cache_initializers(): ReturnsResourceHandlerDictmapping names to handlers
2. transform/library/kvcache.py
InsertCachedAttention: Iterates over attention nodes, callsget_cache_initializers(), and registers handlers viacm.add_resource()InitializeCache: Triggerscm.initialize_resources()to allocate all cachesResizeKVCache: Runs forward pass, measures memory, and callscm.resize_kv_cache_manager()
3. shim/interface.py
CachedSequenceInterface: Central class managing all cachesadd_resource(): Stores handlers in_resource_lookupinitialize_resources(): Sorts handlers by type, creates cache managers, assigns views_identify_managed_kv_resources(): Groups compatible KV handlers_identify_managed_state_resources(): Groups compatible SSM/Conv handlers with constraint checking_create_kv_cache_manager(): CreatesKVCacheManagerorMambaHybridCacheManager_allocate_unmanaged_resources(): Fallback for incompatible handlers
Example Flow: FlashInfer Attention
# In flashinfer_attention.py
class FlashinferCachedAttention(AttentionDescriptor):
@classmethod
def get_cache_initializers(cls, source_attn_node, cache_config):
k_fake = source_attn_node.args[1].meta["val"]
return {
"kv_cache": KVPagedResourceHandler(
num_kv_heads=k_fake.shape[2],
head_dim=k_fake.shape[3],
dtype=cls.resolve_cache_dtype(cache_config.dtype, k_fake.dtype),
kv_factor=2,
kv_layout="HND",
)
}This handler gets:
- Collected by
InsertCachedAttention→cm.add_resource("kv_cache_0", handler) - Sorted into
_paged_cache_orderduringinitialize_resources() - Managed by
KVCacheManagerif compatible with other KV handlers - View assigned via
self._caches["kv_cache_0"] = manager.get_buffers(0, kv_layout="HND")
Compatibility Rules
KV Cache Compatibility (for KVCacheManager)
Handlers are compatible if they match on:
head_dimdtypekv_factorkv_layout
Note: num_kv_heads can differ (supports GQA/MQA with varying head counts per layer).
State Resource Compatibility (for MambaHybridCacheManager)
SSM Resources: Compatible if state_shape and dtype match.
Conv Resources: Compatible if state_shape and dtype match, AND the n_groups constraint holds:
conv_dim = head_dim * num_heads + 2 * n_groups * d_state
If this constraint cannot be satisfied with integer n_groups >= 0, Conv resources fall back to local allocation.
eea59c4 to
9654482
Compare
|
@CodeRabbit summary |
✅ Actions performedSummary regeneration triggered. |
|
/bot run --extra-stage "DGX_B200-4_GPUs-AutoDeploy-1, DGX_H100-4_GPUs-AutoDeploy-1" --disable-fail-fast |
📝 WalkthroughWalkthroughMigrates KV cache to unified HND-style KV resources, adds typed resource handlers (KVPaged, SSM, CausalConv), refactors cache orchestration for hybrid KV+state managers, renames cache indexing from Changes
Sequence Diagram(s)mermaid Descriptor->>Shim: register resource handlers (kv/state) Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
🧪 Generate unit tests (beta)
Important Action Needed: IP Allowlist UpdateIf your organization protects your Git platform with IP whitelisting, please add the new CodeRabbit IP address to your allowlist:
Reviews will stop working after February 8, 2026 if the new IP is not added to your allowlist. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
PR_Github #34258 [ run ] triggered by Bot. Commit: |
📝 WalkthroughWalkthroughThis PR restructures AutoDeploy's cache resource handling system, introducing explicit typed resource handler classes (KVPagedResourceHandler, SSMResourceHandler, CausalConvResourceHandler), unifying KV cache initialization from separate k/v caches into single layouts, systematically renaming cache indexing parameters (cache_loc → slot_idx), removing legacy Triton paged cache implementations, and reorganizing documentation. Changes
Estimated code review effort🎯 4 (Complex) | ⏱️ ~75 minutes Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 1 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (1 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (8)
tensorrt_llm/_torch/auto_deploy/custom_ops/triton_kernels/attention_with_kv_cache.py (2)
1-2:⚠️ Potential issue | 🟠 MajorAdd the required NVIDIA copyright header.
This file is missing the NVIDIA copyright header with the year of latest meaningful modification (2026). Please add the standard header used elsewhere in the repo above the module docstring.
As per coding guidelines: All TensorRT-LLM source files (.cpp, .h, .cu, .py, and other source files) should contain an NVIDIA copyright header with the year of latest meaningful modification.🧩 Suggested placement
+ # Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved. + # (Use the standard NVIDIA license header from this repository.) + """Multi-head attention kernel that can operate with kv-caches."""
3-4: 🛠️ Refactor suggestion | 🟠 MajorUse
import triton.language as tlto preserve the namespace.The coding guidelines require maintaining the module namespace when importing. The codebase consistently uses
import triton.language as tlacross 47+ files. This import pattern violates the mandatory guideline and should be corrected for consistency.♻️ Suggested change
import triton -from triton import language as tl +import triton.language as tltensorrt_llm/_torch/auto_deploy/transform/interface.py (1)
1-4:⚠️ Potential issue | 🟡 MinorAdd the NVIDIA copyright header.
This core source file is missing the required NVIDIA copyright notice with the latest modification year.As per coding guidelines, all TensorRT-LLM source files (.cpp, .h, .cu, .py, and other source files) should contain an NVIDIA copyright header with the year of latest meaningful modification.
tensorrt_llm/_torch/auto_deploy/custom_ops/mamba/flashinfer_backend_mamba.py (1)
1-1:⚠️ Potential issue | 🟡 MinorAdd the standard NVIDIA copyright header.
This file is missing the required NVIDIA copyright header before the imports; please add the project-standard header with the latest modification year. As per coding guidelines: All TensorRT-LLM source files (.cpp, .h, .cu, .py, and other source files) should contain an NVIDIA copyright header with the year of latest meaningful modification.
tests/unittest/_torch/auto_deploy/unit/singlegpu/custom_ops/test_update_kv_cache.py (2)
1-1:⚠️ Potential issue | 🟡 MinorAdd the standard NVIDIA copyright header.
This test module is missing the required NVIDIA copyright header; please add the project-standard header with the latest modification year. As per coding guidelines: All TensorRT-LLM source files (.cpp, .h, .cu, .py, and other source files) should contain an NVIDIA copyright header with the year of latest meaningful modification.
3-3:⚠️ Potential issue | 🟡 MinorPreserve module namespace on torch_attention import.
Switch to a module import and qualify the call site.
♻️ Proposed refactor
-from tensorrt_llm._torch.auto_deploy.custom_ops.torch_attention import update_kv_cache +import tensorrt_llm._torch.auto_deploy.custom_ops.torch_attention as torch_attention @@ - update_kv_cache( + torch_attention.update_kv_cache(As per coding guidelines: Always maintain the namespace when importing Python modules, even if only one class or function from a module is used.
tensorrt_llm/_torch/auto_deploy/custom_ops/torch_attention.py (1)
239-253:⚠️ Potential issue | 🟡 MinorRename
cache_loctoslot_idxinfused_mla_refandfused_mla_ref_fakefor consistency.The rename from
cache_loctoslot_idxinupdate_kv_cacheis correct, butfused_mla_ref(lines 264, 330, 337-338) andfused_mla_ref_fake(line 397) still usecache_loc. Since these functions callupdate_kv_cacheand use the same semantic parameter, they should use the same name for consistency.tests/integration/defs/accuracy/test_llm_api_autodeploy.py (1)
46-78:⚠️ Potential issue | 🟠 MajorEnsure chunked-prefill
max_num_tokensexceeds backendmax_batch_size.
Forattn_backend="flashinfer",max_batch_sizeis 512, but chunked-prefill setsmax_num_tokensto 512, which violates the “must be > max(tokens_per_block, max_batch_size)” requirement and risks cache sizing failures.💡 Proposed fix
- # NOTE: must be > max(tokens_per_block, max_batch_size) - config["max_num_tokens"] = 512 + # NOTE: must be > max(tokens_per_block, max_batch_size) + config["max_num_tokens"] = max(backend_cfg["max_batch_size"] + 1, 512)
🤖 Fix all issues with AI agents
In `@tensorrt_llm/_torch/auto_deploy/custom_ops/attention_interface.py`:
- Around line 1081-1114: The docstring for allocate() misstates the NHD layout
shape; update the docstring to match the implementation in allocate (when
kv_layout == "NHD") which returns a tensor shaped [num_blocks, tokens_per_block,
kv_factor, num_kv_heads, head_dim], or if the intended shape is different change
the tensor construction to match the documented [num_blocks, kv_factor,
tokens_per_block, num_kv_heads, head_dim]; locate the allocate function and
either correct the NHD docstring to the implementation shape or reorder the
torch.empty dimensions to match the docstring so both agree.
In
`@tensorrt_llm/_torch/auto_deploy/custom_ops/mamba/flashinfer_backend_mamba.py`:
- Line 7: The import uses a from-import for AttentionRegistry, MHACallable,
ResourceHandlerDict; change it to import the module namespace instead (import
the attention_interface module from the parent package) and update downstream
references to use attention_interface.AttentionRegistry,
attention_interface.MHACallable, and attention_interface.ResourceHandlerDict so
the module namespace is preserved across the file.
In `@tensorrt_llm/_torch/auto_deploy/custom_ops/mla.py`:
- Around line 1-2: Add the required NVIDIA copyright header at the very top of
the file (above the existing module docstring) and include the latest
modification year and appropriate copyright language; modify
tensorrt_llm._torch.auto_deploy.custom_ops.mla.py by inserting the standard
NVIDIA header block before the triple-quoted docstring so the file starts with
the copyright notice followed by the existing module docstring.
🧹 Nitpick comments (10)
tests/unittest/_torch/auto_deploy/unit/singlegpu/custom_ops/triton_kernels/test_attention_with_kv_cache.py (1)
55-109: Use snake_case for local slot index lists.
SLOT_IDXis a local list; per guidelines it should be snake_case. Consider renaming toslot_idx_listand updating its uses in this test (and similarly elsewhere).
As per coding guidelines: Python local variables should use snake_case.♻️ Suggested rename in this test
- SLOT_IDX = list(range(0, len(SEQ_LENS))) - random.shuffle(SLOT_IDX) + slot_idx_list = list(range(0, len(SEQ_LENS))) + random.shuffle(slot_idx_list) ... - torch.tensor(SLOT_IDX, device=DEVICE, dtype=torch.int32), + torch.tensor(slot_idx_list, device=DEVICE, dtype=torch.int32), ... - for i, slot_idx in enumerate(SLOT_IDX): + for i, slot_idx in enumerate(slot_idx_list):tests/unittest/_torch/auto_deploy/unit/singlegpu/transformations/library/test_kv_cache.py (3)
10-11: Prefer module import to preserve namespace.
Please keep the module namespace and update usages accordingly.As per coding guidelines, always maintain the namespace when importing Python modules, even if only one class or function from a module is used.♻️ Suggested import style
-from tensorrt_llm._torch.auto_deploy.custom_ops.attention_interface import KVPagedResourceHandler +import tensorrt_llm._torch.auto_deploy.custom_ops.attention_interface as attention_interface
295-301: Remove the inline from-import and reuse the module alias.
This avoids redundant imports and keeps the namespace rule consistent with the module-level import.As per coding guidelines, always maintain the namespace when importing Python modules, even if only one class or function from a module is used.♻️ Suggested update
- from tensorrt_llm._torch.auto_deploy.custom_ops.attention_interface import ( - KVPagedResourceHandler, - ) - - dummy_cached_interface.add_resource( - "kv_cache_0", KVPagedResourceHandler(8, 64, dtype=torch.float16) - ) + dummy_cached_interface.add_resource( + "kv_cache_0", + attention_interface.KVPagedResourceHandler(8, 64, dtype=torch.float16), + )
528-529: Avoid asserting on private_paged_cache_order.
Prefer a public accessor or a small test helper onCachedSequenceInterfaceto keep tests resilient to internal refactors.tensorrt_llm/_torch/auto_deploy/custom_ops/mla.py (1)
10-20: Keeptriton_attentionimported as a module to preserve namespace.
Update call sites to use the module-qualified functions.As per coding guidelines, always maintain the namespace when importing Python modules, even if only one class or function from a module is used.♻️ Suggested update
-from .triton_attention import _decode_attention, _prefill_attention +from . import triton_attention @@ - _decode_attention( + triton_attention._decode_attention( @@ - _prefill_attention( + triton_attention._prefill_attention(tests/unittest/_torch/auto_deploy/unit/singlegpu/custom_ops/test_resource_handlers.py (1)
275-304: Consider moving imports to the top of the file for consistency.The
SSMResourceHandlerandCausalConvResourceHandlerimports are done locally inside test functions (lines 277, 287, 309, 321). While this works, it's inconsistent with howKVPagedResourceHandleris imported at the module level (line 15).♻️ Suggested refactor: consolidate imports at module level
from tensorrt_llm._torch.auto_deploy.custom_ops.attention_interface import ( AttentionDescriptor, KVPagedResourceHandler, ResourceHandler, SequenceInfo, StateResourceHandler, UnpagedResourceHandler, + SSMResourceHandler, + CausalConvResourceHandler, )Then remove the local imports from test functions.
docs/source/features/auto_deploy/advanced/kv_cache_architecture.md (1)
126-133: Minor: Table formatting could be improved for consistency.The static analysis flagged table formatting style. While functional, consider adding consistent spacing around pipes for better readability.
📝 Suggested formatting fix
-| Handler Type | Managed By | Buffer Source | Use Case | -|--------------|------------|---------------|----------| +| Handler Type | Managed By | Buffer Source | Use Case | +|----------------------------|--------------------------|----------------------------|---------------------------------|This is a minor style preference and doesn't affect functionality.
tests/unittest/_torch/auto_deploy/unit/singlegpu/shim/test_cached_sequence_interface.py (1)
12-19: Prefer module-level imports to keep namespace.
Consider importing the module and referencing symbols via the module namespace to align with repository import conventions.As per coding guidelines, Always maintain the namespace when importing Python modules, even if only one class or function from a module is used.
tensorrt_llm/_torch/auto_deploy/custom_ops/flashinfer_attention.py (1)
15-25: Keep attention_interface imports namespaced.
Consider importing the module and referencingKVPagedResourceHandler/others via the module namespace.As per coding guidelines, Always maintain the namespace when importing Python modules, even if only one class or function from a module is used.
tensorrt_llm/_torch/auto_deploy/shim/interface.py (1)
13-24: Use module import for_utilshelper to keep namespace.As per coding guidelines, Always maintain the namespace when importing Python modules, even if only one class or function from a module is used.♻️ Suggested refactor
-from ...._utils import torch_dtype_to_binding +from .... import _utils as trt_utils @@ - "dtype": torch_dtype_to_binding(kv_ref.dtype), + "dtype": trt_utils.torch_dtype_to_binding(kv_ref.dtype),Also applies to: 371-376
9654482 to
a348171
Compare
|
/bot run --extra-stage "DGX_B200-4_GPUs-AutoDeploy-1, DGX_H100-4_GPUs-AutoDeploy-1" --disable-fail-fast |
|
PR_Github #34263 [ run ] triggered by Bot. Commit: |
|
PR_Github #34263 [ run ] completed with state
|
Adapt TRT-LLM attention to work with Lucas's KV cache changes (PR NVIDIA#11149): - TrtllmKVResourceHandler now extends KVPagedResourceHandler for proper cache interface recognition, with __eq__ returning False to use PTCacheBackend's own cache management instead of shared KVCacheManager - Calculate optimal num_blocks based on available GPU memory (80%) instead of using undersized value from dummy KVCacheManager - Fix GPU work tensor sizing in PTCacheBackend to use actual allocated block count (config.num_pages) rather than sequence_info.num_blocks - Handle 5D->4D tensor squeeze for kv_factor=1 dimension from KVCacheManager - Remove torch.cuda.synchronize() that broke CUDA graph capture These changes resolve memory corruption issues during warmup and shape mismatches during inference that occurred after rebasing on the new KV cache manager API. Signed-off-by: Eran Geva <egeva@nvidia.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Lucas Liebenwein <11156568+lucaslie@users.noreply.github.com>
a348171 to
3a387ce
Compare
|
/bot run --extra-stage "DGX_B200-4_GPUs-AutoDeploy-1, DGX_H100-4_GPUs-AutoDeploy-1" --disable-fail-fast |
|
PR_Github #34697 [ run ] triggered by Bot. Commit: |
There was a problem hiding this comment.
Actionable comments posted: 5
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (24)
docs/source/conf.py (1)
1-5:⚠️ Potential issue | 🟠 MajorAdd the required NVIDIA copyright header.
This Python source is missing the NVIDIA copyright header with the year of latest meaningful modification (2026).As per coding guidelines, all TensorRT-LLM source files (.cpp, .h, .cu, .py, and other source files) should contain an NVIDIA copyright header with the year of latest meaningful modification.🔧 Proposed header
+# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + # Configuration file for the Sphinx documentation builder. # # For the full list of built-in configuration values, see the documentation: # https://www.sphinx-doc.org/en/master/usage/configuration.htmltensorrt_llm/_torch/auto_deploy/transform/interface.py (1)
1-4:⚠️ Potential issue | 🟠 MajorAdd the required NVIDIA copyright header.
This Python source is missing the NVIDIA copyright header with the year of latest meaningful modification (2026).As per coding guidelines, all TensorRT-LLM source files (.cpp, .h, .cu, .py, and other source files) should contain an NVIDIA copyright header with the year of latest meaningful modification.🔧 Proposed header
+# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + """The interface for all transforms. This module defines the base classes and interfaces for all transforms. """tensorrt_llm/_torch/auto_deploy/transform/library/kvcache.py (1)
1-2:⚠️ Potential issue | 🟠 MajorUpdate SPDX header year to reflect the 2026 modification.
The file was modified in 2026, so the header year should be updated.As per coding guidelines, all TensorRT-LLM source files (.cpp, .h, .cu, .py, and other source files) should contain an NVIDIA copyright header with the year of latest meaningful modification.🔧 Proposed fix
-# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.tensorrt_llm/_torch/auto_deploy/custom_ops/attention_interface.py (1)
1-10:⚠️ Potential issue | 🟠 MajorAdd the NVIDIA copyright header.
This source file is missing the required NVIDIA copyright notice with the latest modification year.
As per coding guidelines, all TensorRT-LLM source files (.cpp, .h, .cu, .py, and other source files) should contain an NVIDIA copyright header with the year of latest meaningful modification.📝 Suggested header (match existing SPDX style)
+# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License.tensorrt_llm/_torch/auto_deploy/custom_ops/triton_kernels/attention_with_kv_cache.py (1)
1-5:⚠️ Potential issue | 🟠 MajorAdd the NVIDIA copyright header.
This file is missing the required NVIDIA header with the latest modification year.
As per coding guidelines, all TensorRT-LLM source files (.cpp, .h, .cu, .py, and other source files) should contain an NVIDIA copyright header with the year of latest meaningful modification.📝 Suggested header (match SPDX style)
+# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License.tensorrt_llm/_torch/auto_deploy/custom_ops/triton_attention.py (3)
1-5:⚠️ Potential issue | 🟠 MajorAdd the NVIDIA copyright header.
This file is missing the required NVIDIA header with the latest modification year.
As per coding guidelines, all TensorRT-LLM source files (.cpp, .h, .cu, .py, and other source files) should contain an NVIDIA copyright header with the year of latest meaningful modification.📝 Suggested header (match SPDX style)
+# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License.
278-303:⚠️ Potential issue | 🟡 MinorSilence unused
slot_idxin the fake kernel.This triggers ARG001; add a local no-op to satisfy lint.
🧹 Suggested fix
def flattened_mha_fake( # Q, K, V q: torch.Tensor, k: torch.Tensor, v: torch.Tensor, # STANDARD METADATA batch_info_host: torch.Tensor, seq_len: torch.Tensor, input_pos: torch.Tensor, slot_idx: torch.Tensor, cu_seqlen: torch.Tensor, @@ ): + _ = slot_idx return q.new_empty(*q.shape[:-1], v.shape[-1]).contiguous()
143-161:⚠️ Potential issue | 🟠 MajorFix grid sizing:
max(seq_len)returns a Tensor and cannot be used in Triton grid tuples.
seq_lenis a torch.Tensor. Python'smax(seq_len)yields a 0-d Tensor, which cannot be used directly in Triton grid dimension tuples. Extract the maximum sequence length as a Python int once and reuse it in both the kernel call at line 143 and the grid assignment at line 160.🛠️ Suggested fix
- update_kv_cache[(num_prefill, n_kv_heads, (max(seq_len) + SEQ_BLOCK - 1) // SEQ_BLOCK)]( + max_seq_len = int(seq_len.max().item()) + num_blocks = (max_seq_len + SEQ_BLOCK - 1) // SEQ_BLOCK + update_kv_cache[(num_prefill, n_kv_heads, num_blocks)]( k, v, seq_len, seq_start, k_cache, v_cache, input_pos, slot_idx, max_cache_seq_len, n_kv_heads, q_d_head, v_d_head, 32, GENERATE_ONLY=False, ) - grid = (num_prefill, n_heads, (max(seq_len) + SEQ_BLOCK - 1) // SEQ_BLOCK) + grid = (num_prefill, n_heads, num_blocks)tensorrt_llm/_torch/auto_deploy/transform/library/kvcache_transformers.py (1)
1-10:⚠️ Potential issue | 🟠 MajorAdd the NVIDIA copyright header.
This file is missing the required NVIDIA header with the latest modification year.
As per coding guidelines, all TensorRT-LLM source files (.cpp, .h, .cu, .py, and other source files) should contain an NVIDIA copyright header with the year of latest meaningful modification.📝 Suggested header (match SPDX style)
+# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License.tensorrt_llm/_torch/auto_deploy/transform/library/hidden_states.py (1)
1-2:⚠️ Potential issue | 🟡 MinorUpdate the SPDX header year to the latest modification year.
The header still reflects 2025 even though this file has 2026 changes.
As per coding guidelines, all TensorRT-LLM source files (.cpp, .h, .cu, .py, and other source files) should contain an NVIDIA copyright header with the year of latest meaningful modification.📝 Suggested update
-# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.tensorrt_llm/_torch/auto_deploy/custom_ops/torch_attention.py (1)
264-264:⚠️ Potential issue | 🟡 MinorStandardize parameter naming: use
slot_idxinstead ofcache_locinfused_mla_refand related functions.The inconsistency is confirmed. Both
update_kv_cache(line 232) andfused_mla_ref(line 264) perform identical cache indexing operations—retrieving and storing key-value cache entries—but use different parameter names:slot_idxvscache_loc. Sincefused_mla_refcallsupdate_kv_cacheand passescache_locas the argument for whatupdate_kv_cacheexpects asslot_idx(line 330), and both functions use the parameter identically for indexing (lines 248, 251 vs 337–338), the naming should be unified. The same pattern appears infused_mla_ref_fake(line 397).tests/unittest/_torch/auto_deploy/unit/singlegpu/custom_ops/triton_kernels/test_attention_with_kv_cache.py (2)
1-6:⚠️ Potential issue | 🟡 MinorMissing SPDX copyright header.
This file is missing the required NVIDIA copyright header. Per coding guidelines, all TensorRT-LLM source files should contain an SPDX copyright header with the year of latest meaningful modification.
📝 Proposed fix
+# SPDX-FileCopyrightText: Copyright (c) 2022-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + import math import random
194-199:⚠️ Potential issue | 🟠 MajorMissing
assertontorch.allclose— test will always pass.The
torch.allclosecall on line 194 is not wrapped in anassertstatement, so this test will pass regardless of whether the values match.🐛 Proposed fix
- torch.allclose( + assert torch.allclose( ref.squeeze().cpu().to(torch.float32), output.squeeze().cpu().to(torch.float32), atol=1e-2, rtol=1e-2, )tensorrt_llm/_torch/auto_deploy/custom_ops/mamba/torch_backend_causal_conv.py (1)
1-11:⚠️ Potential issue | 🟡 MinorMissing SPDX copyright header.
This file is missing the required NVIDIA SPDX copyright header. Per coding guidelines, all TensorRT-LLM source files should contain an SPDX copyright header.
📝 Proposed fix
+# SPDX-FileCopyrightText: Copyright (c) 2022-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + """Custom op collection for cached causal conv1d in pure PyTorch.tests/unittest/_torch/auto_deploy/unit/singlegpu/transformations/library/test_kv_cache.py (1)
1-6:⚠️ Potential issue | 🟡 MinorMissing SPDX copyright header.
This test file is missing the required NVIDIA SPDX copyright header.
📝 Proposed fix
+# SPDX-FileCopyrightText: Copyright (c) 2022-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + from typing import List, Optionaltensorrt_llm/_torch/auto_deploy/custom_ops/mamba/torch_backend_mamba.py (1)
1-7:⚠️ Potential issue | 🟡 MinorMissing SPDX copyright header.
This file is missing the required NVIDIA SPDX copyright header.
📝 Proposed fix
+# SPDX-FileCopyrightText: Copyright (c) 2022-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + """Custom op collection for cached mamba2 ssm transform (linear attention) in pure PyTorch.tensorrt_llm/_torch/auto_deploy/custom_ops/mamba/flashinfer_backend_mamba.py (1)
1-7:⚠️ Potential issue | 🟡 MinorMissing SPDX copyright header.
This file is missing the required NVIDIA SPDX copyright header.
📝 Proposed fix
+# SPDX-FileCopyrightText: Copyright (c) 2022-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + from typing import Listtests/unittest/_torch/auto_deploy/unit/singlegpu/custom_ops/test_resource_handlers.py (1)
1-8:⚠️ Potential issue | 🟠 MajorAdd the NVIDIA copyright header (2026).
This file is missing the required NVIDIA header for TensorRT-LLM source files.
As per coding guidelines: All TensorRT-LLM source files (.cpp, .h, .cu, .py, and other source files) should contain an NVIDIA copyright header with the year of latest meaningful modification.📝 Suggested header
+ # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + # SPDX-License-Identifier: Apache-2.0tests/unittest/_torch/auto_deploy/unit/singlegpu/custom_ops/test_flashinfer_attention_op.py (1)
1-5:⚠️ Potential issue | 🟠 MajorAdd the NVIDIA copyright header (2026).
This file is missing the required NVIDIA header for TensorRT-LLM source files.
As per coding guidelines: All TensorRT-LLM source files (.cpp, .h, .cu, .py, and other source files) should contain an NVIDIA copyright header with the year of latest meaningful modification.📝 Suggested header
+ # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + # SPDX-License-Identifier: Apache-2.0tests/integration/defs/accuracy/test_llm_api_autodeploy.py (1)
1-2:⚠️ Potential issue | 🟠 MajorUpdate the NVIDIA copyright year to 2026.
This file was modified in 2026 but still lists 2025 in the header.
As per coding guidelines: All TensorRT-LLM source files (.cpp, .h, .cu, .py, and other source files) should contain an NVIDIA copyright header with the year of latest meaningful modification.📝 Suggested header tweak
-# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.tests/unittest/_torch/auto_deploy/unit/singlegpu/shim/test_cached_sequence_interface.py (1)
1-7:⚠️ Potential issue | 🟠 MajorAdd the NVIDIA copyright header (2026).
This file is missing the required NVIDIA header for TensorRT-LLM source files.
As per coding guidelines: All TensorRT-LLM source files (.cpp, .h, .cu, .py, and other source files) should contain an NVIDIA copyright header with the year of latest meaningful modification.📝 Suggested header
+ # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + # SPDX-License-Identifier: Apache-2.0tensorrt_llm/_torch/auto_deploy/custom_ops/flashinfer_attention.py (2)
1-5:⚠️ Potential issue | 🟠 MajorAdd the NVIDIA copyright header (2026).
This file is missing the required NVIDIA header for TensorRT-LLM source files.
As per coding guidelines: All TensorRT-LLM source files (.cpp, .h, .cu, .py, and other source files) should contain an NVIDIA copyright header with the year of latest meaningful modification.📝 Suggested header
+ # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + # SPDX-License-Identifier: Apache-2.0
462-469:⚠️ Potential issue | 🟡 MinorSilence unused-argument lint in fake op.
🧹 Suggested fix
def flashinfer_mha_with_cache_fake( @@ - kv_cache: torch.Tensor, + kv_cache: torch.Tensor, @@ ) -> torch.Tensor: + _ = kv_cache return torch.empty_like(q.contiguous())tensorrt_llm/_torch/auto_deploy/shim/interface.py (1)
111-116:⚠️ Potential issue | 🟠 Major
to()doesn't update unmanaged caches because the result isn't assigned.
Tensor.to(...)returns a new tensor; without assignment, caches stay on the old device.🛠️ Suggested fix
for name, cache in self._caches.items(): if name in self._unmanaged_resources: - cache.to(*args, **kwargs) + self._caches[name] = cache.to(*args, **kwargs)
🤖 Fix all issues with AI agents
In `@tensorrt_llm/_torch/auto_deploy/custom_ops/mla.py`:
- Around line 151-177: The calls to the Triton helpers are passing page indices
(`cache_loc`) instead of per-sequence slot indices; update the metadata and call
sites so `_decode_attention` and `_prefill_attention` receive `slot_idx`
(per-sequence slot indices) rather than `cache_loc` (page indices): rename the
metadata argument to `slot_idx`, change any slicing/usage that currently reads
`cache_loc` to use `slot_idx` (including the arguments passed into
`_decode_attention` and `_prefill_attention`), and update
`get_standard_metadata_args` to expose and return `slot_idx` so all call sites
use the correct slot indices for unpaged cache accesses and after slot
migration.
In `@tensorrt_llm/_torch/auto_deploy/shim/interface.py`:
- Around line 1-4: This file is missing the required NVIDIA copyright header
(2026); add the standard NVIDIA copyright/header block at the very top of the
file before any imports (above the existing import copy / import functools
lines) using the 2026 year and the canonical TensorRT-LLM header text used
across the repo so it appears in tensorrt_llm._torch.auto_deploy.shim.interface
(the module containing the shown imports and typing declarations).
In `@tests/integration/defs/accuracy/test_llm_api_autodeploy.py`:
- Around line 103-107: The chunked-prefill configuration sets
config["max_num_tokens"] = 512 which violates the requirement that
max_num_tokens must be strictly greater than max_batch_size (max_batch_size is
512 for flashinfer); update the code in the block guarded by
enable_chunked_prefill (where config dict is modified) to set
config["max_num_tokens"] to a value strictly greater than max_batch_size (e.g.,
max_batch_size + 1 or a constant like 513) so the constraint is satisfied.
- Around line 61-73: Annotate the mutable class attribute ATTN_BACKEND_CONFIGS
on the TestLlama3_1_8B class with typing.ClassVar to make it explicit that it is
a class-level shared constant; add the appropriate typing imports (e.g., from
typing import ClassVar, Dict, Any) and change the declaration to include a
ClassVar type annotation such as ATTN_BACKEND_CONFIGS: ClassVar[Dict[str,
Dict[str, Any]]] = {...} so the linter (RUF012) recognizes it as a class
variable.
In
`@tests/unittest/_torch/auto_deploy/unit/singlegpu/custom_ops/test_flashinfer_attention_op.py`:
- Around line 23-34: The function _attention_with_fp8_kv_cache currently
declares an unused parameter v; remove v from the function signature and update
any internal references or local variables accordingly (there are none inside),
and update all call sites/tests that pass v to stop supplying that argument (or
pass only q,k,kv_cache,k_scale,v_scale,prefill_seq_len,causal,mask in the new
order). Also apply the same removal to the other duplicate helper instance
mentioned (the similar function around the other occurrence) so both function
signatures and callers remain consistent.
🧹 Nitpick comments (10)
tensorrt_llm/_torch/auto_deploy/custom_ops/attention_interface.py (1)
14-14: Prefer namespace import fortyping.The current
from typing import ...import violates the namespace-import guideline; consider switching toimport typing as tand qualifying type names.As per coding guidelines, always maintain the namespace when importing Python modules, even if only one class or function from a module is used.
tensorrt_llm/_torch/auto_deploy/transform/library/kvcache_transformers.py (1)
19-205: Use module namespace import for.kvcache.Guidelines prefer maintaining module namespaces; consider importing the module and qualifying
_InsertCachedOperator.As per coding guidelines, always maintain the namespace when importing Python modules, even if only one class or function from a module is used.💡 Suggested change
-from .kvcache import _InsertCachedOperator +from . import kvcache @@ -class HFReplaceCachedAttn(_InsertCachedOperator): +class HFReplaceCachedAttn(kvcache._InsertCachedOperator):tensorrt_llm/_torch/auto_deploy/custom_ops/mla.py (1)
19-19: Prefer module namespace import for triton_attention.Switch to a module import and qualify
_decode_attention/_prefill_attentionto follow namespace guidelines.As per coding guidelines, always maintain the namespace when importing Python modules, even if only one class or function from a module is used.
tensorrt_llm/_torch/auto_deploy/transform/library/hidden_states.py (1)
44-44: Prefer module namespace import for.kvcache.Switch to
from . import kvcacheand qualify_InsertCachedOperatorto follow namespace import guidance.As per coding guidelines, always maintain the namespace when importing Python modules, even if only one class or function from a module is used.
tests/unittest/_torch/auto_deploy/unit/singlegpu/transformations/library/test_kv_cache.py (1)
295-297: Redundant import —KVPagedResourceHandleris already imported at module level.
KVPagedResourceHandleris already imported at line 11. This local import is unnecessary.♻️ Proposed fix
# Add a resource to verify initialize_resources is called - from tensorrt_llm._torch.auto_deploy.custom_ops.attention_interface import ( - KVPagedResourceHandler, - ) - dummy_cached_interface.add_resource( "kv_cache_0", KVPagedResourceHandler(8, 64, dtype=torch.float16) )tests/unittest/_torch/auto_deploy/unit/singlegpu/custom_ops/test_resource_handlers.py (1)
13-20: Keep module namespace in imports.Prefer module-level imports and qualify usages rather than importing symbols directly.
As per coding guidelines: Always maintain the namespace when importing Python modules, even if only one class or function from a module is used.
tests/unittest/_torch/auto_deploy/unit/singlegpu/custom_ops/test_flashinfer_attention_op.py (1)
6-6: Keep module namespace in imports.Prefer module-level imports and qualify usages rather than importing symbols directly.
As per coding guidelines: Always maintain the namespace when importing Python modules, even if only one class or function from a module is used.
tests/unittest/_torch/auto_deploy/unit/singlegpu/shim/test_cached_sequence_interface.py (1)
12-23: Keep module namespace in imports.Prefer module-level imports and qualify usages rather than importing symbols directly.
As per coding guidelines: Always maintain the namespace when importing Python modules, even if only one class or function from a module is used.
tensorrt_llm/_torch/auto_deploy/custom_ops/flashinfer_attention.py (1)
15-25: Keep module namespace in imports.Prefer module-level imports and qualify usages rather than importing symbols directly.
As per coding guidelines: Always maintain the namespace when importing Python modules, even if only one class or function from a module is used.
tensorrt_llm/_torch/auto_deploy/shim/interface.py (1)
13-24: Keep module namespace in imports.Prefer module-level imports and qualify usages rather than importing symbols directly.
As per coding guidelines: Always maintain the namespace when importing Python modules, even if only one class or function from a module is used.
|
PR_Github #34697 [ run ] completed with state
|
|
/bot skip --comment "All single-gpu and all AD multi-gpu tests are passing" |
|
PR_Github #34785 [ skip ] triggered by Bot. Commit: |
|
PR_Github #34785 [ skip ] completed with state |
…VIDIA#11149) Signed-off-by: Lucas Liebenwein <11156568+lucaslie@users.noreply.github.com>
5 participants
Description
fixes #10966
AutoDeploy Features
AutoDeploy Doc Update
features/torch/auto_deploy/advancedand moved everything tofeatures/auto_deploy/advancedTest Coverage
PR Checklist
Please review the following before submitting your PR:
PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.
GitHub Bot Help
/bot [-h] ['run', 'kill', 'skip', 'reuse-pipeline'] ...Provide a user friendly way for developers to interact with a Jenkins server.
Run
/bot [-h|--help]to print this help message.See details below for each supported subcommand.
Details
run [--reuse-test (optional)pipeline-id --disable-fail-fast --skip-test --stage-list "A10-PyTorch-1, xxx" --gpu-type "A30, H100_PCIe" --test-backend "pytorch, cpp" --add-multi-gpu-test --only-multi-gpu-test --disable-multi-gpu-test --post-merge --extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx" --detailed-log --debug(experimental)]Launch build/test pipelines. All previously running jobs will be killed.
--reuse-test (optional)pipeline-id(OPTIONAL) : Allow the new pipeline to reuse build artifacts and skip successful test stages from a specified pipeline or the last pipeline if no pipeline-id is indicated. If the Git commit ID has changed, this option will be always ignored. The DEFAULT behavior of the bot is to reuse build artifacts and successful test results from the last pipeline.--disable-reuse-test(OPTIONAL) : Explicitly prevent the pipeline from reusing build artifacts and skipping successful test stages from a previous pipeline. Ensure that all builds and tests are run regardless of previous successes.--disable-fail-fast(OPTIONAL) : Disable fail fast on build/tests/infra failures.--skip-test(OPTIONAL) : Skip all test stages, but still run build stages, package stages and sanity check stages. Note: Does NOT update GitHub check status.--stage-list "A10-PyTorch-1, xxx"(OPTIONAL) : Only run the specified test stages. Examples: "A10-PyTorch-1, xxx". Note: Does NOT update GitHub check status.--gpu-type "A30, H100_PCIe"(OPTIONAL) : Only run the test stages on the specified GPU types. Examples: "A30, H100_PCIe". Note: Does NOT update GitHub check status.--test-backend "pytorch, cpp"(OPTIONAL) : Skip test stages which don't match the specified backends. Only support [pytorch, cpp, tensorrt, triton]. Examples: "pytorch, cpp" (does not run test stages with tensorrt or triton backend). Note: Does NOT update GitHub pipeline status.--only-multi-gpu-test(OPTIONAL) : Only run the multi-GPU tests. Note: Does NOT update GitHub check status.--disable-multi-gpu-test(OPTIONAL) : Disable the multi-GPU tests. Note: Does NOT update GitHub check status.--add-multi-gpu-test(OPTIONAL) : Force run the multi-GPU tests in addition to running L0 pre-merge pipeline.--post-merge(OPTIONAL) : Run the L0 post-merge pipeline instead of the ordinary L0 pre-merge pipeline.--extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx"(OPTIONAL) : Run the ordinary L0 pre-merge pipeline and specified test stages. Examples: --extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx".--detailed-log(OPTIONAL) : Enable flushing out all logs to the Jenkins console. This will significantly increase the log volume and may slow down the job.--debug(OPTIONAL) : Experimental feature. Enable access to the CI container for debugging purpose. Note: Specify exactly one stage in thestage-listparameter to access the appropriate container environment. Note: Does NOT update GitHub check status.For guidance on mapping tests to stage names, see
docs/source/reference/ci-overview.mdand the
scripts/test_to_stage_mapping.pyhelper.kill
killKill all running builds associated with pull request.
skip
skip --comment COMMENTSkip testing for latest commit on pull request.
--comment "Reason for skipping build/test"is required. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.reuse-pipeline
reuse-pipelineReuse a previous pipeline to validate current commit. This action will also kill all currently running builds associated with the pull request. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.
Summary by CodeRabbit