[None][feat] Support multi-gpu running for nemotron-v3-nano and super - #10118
Conversation
📝 WalkthroughWalkthroughThis pull request refactors weight loading and quantization handling for NVFP4, FP8, and MoE scenarios. Changes include: introducing a Mamba2 mixer weight splitting helper for NemotronH, extending KV duplication for weight scales in NVFP4 mode, reworking expert weight placement with shard-based logic, adding post-load alignment hooks to multiple FusedMoE methods, and gating conditional unswizzle/padding in NVFP4LinearMethod. Changes
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45–60 minutes Areas requiring extra attention:
Pre-merge checks and finishing touches❌ Failed checks (3 warnings)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
tensorrt_llm/_torch/models/checkpoints/hf/weight_mapper.py (1)
72-82: Guard NVFP4 check viahas_nvfp4(or quant_config existence) to avoid fragile assumptionsThe new
duplicated_keyslogic is fine, but callingmodule.quant_config.quant_mode.has_nvfp4()assumes everymodulethat hits this path always has a non-Nonequant_configand aquant_modeattribute. That’s true forLinearin the quantized path, but this callback is generic and can be reused, which makes the assumption a bit brittle.To keep the check robust and re-use the existing encapsulation, consider:
- Using the
Linear.has_nvfp4property when available, or- Guarding
quant_configwithgetattr.For example:
Suggested change
- duplicated_keys = ["weight", "bias"] - if module.quant_config.quant_mode.has_nvfp4(): - duplicated_keys.append("weight_scale") + duplicated_keys = ["weight", "bias"] + # Reuse Linear's NVFP4 detection when available; otherwise fall back to quant_config if present. + has_nvfp4 = getattr(module, "has_nvfp4", + getattr(getattr(module, "quant_config", None), + "quant_mode", None).has_nvfp4() + if getattr(module, "quant_config", None) is not None + else False) + if has_nvfp4: + duplicated_keys.append("weight_scale")If you’re certain this callback only ever sees NVFP4-quantized
Linearmodules, you can keep behavior as-is, but it would be good to document that assumption.
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
tensorrt_llm/_torch/models/checkpoints/hf/nemotron_h_weight_mapper.py(4 hunks)tensorrt_llm/_torch/models/checkpoints/hf/weight_mapper.py(1 hunks)tensorrt_llm/_torch/modules/fused_moe/quantization.py(4 hunks)tensorrt_llm/_torch/modules/linear.py(1 hunks)
🧰 Additional context used
📓 Path-based instructions (2)
**/*.py
📄 CodeRabbit inference engine (CODING_GUIDELINES.md)
**/*.py: Code developed for TensorRT-LLM should conform to Python 3.8+
Indent Python code with 4 spaces. Do not use tabs
Always maintain the namespace when importing in Python, even if only one class or function from a module is used
Python files should use snake_case naming:some_file.py
Python classes should use PascalCase naming:class SomeClass
Python functions and methods should use snake_case naming:def my_awesome_function():
Python local variables should use snake_case naming:my_variable = ...
Python variable names that start with a number should be prefixed with 'k':k_99th_percentile = ...
Python global variables should use upper snake_case with prefix 'G':G_MY_GLOBAL = ...
Python constants should use upper snake_case naming:MY_CONSTANT = ...
Avoid shadowing variables declared in an outer scope in Python
Initialize all externally visible members of a Python class in the constructor
For Python interfaces that may be used outside a file, prefer docstrings over comments
Python comments should be reserved for code within a function, or interfaces that are local to a file
Use Google style docstrings in Python for classes and functions, which can be parsed by Sphinx
Python attributes and variables can be documented inline with type and description
Avoid using reflection in Python when functionality can be easily achieved without reflection
When using try-except blocks in Python, limit the except to the smallest set of errors possible
When using try-except blocks in Python to handle multiple possible variable types (duck-typing), keep the body of the try as small as possible, using the else block for logic
Files:
tensorrt_llm/_torch/models/checkpoints/hf/weight_mapper.pytensorrt_llm/_torch/modules/linear.pytensorrt_llm/_torch/models/checkpoints/hf/nemotron_h_weight_mapper.pytensorrt_llm/_torch/modules/fused_moe/quantization.py
**/*.{cpp,h,cu,cuh,py}
📄 CodeRabbit inference engine (CODING_GUIDELINES.md)
All TensorRT-LLM Open Source Software code should contain an NVIDIA copyright header that includes the year of its latest meaningful modification
Files:
tensorrt_llm/_torch/models/checkpoints/hf/weight_mapper.pytensorrt_llm/_torch/modules/linear.pytensorrt_llm/_torch/models/checkpoints/hf/nemotron_h_weight_mapper.pytensorrt_llm/_torch/modules/fused_moe/quantization.py
🧠 Learnings (11)
📓 Common learnings
Learnt from: djns99
Repo: NVIDIA/TensorRT-LLM PR: 6915
File: cpp/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_kernels.cu:4010-4012
Timestamp: 2025-08-14T23:23:27.449Z
Learning: For MOE (Mixture of Experts) code reviews in TensorRT-LLM, avoid repeatedly suggesting finalize fusion validation checks and safety assertions. The user djns99 has indicated these suggestions are repetitive and unwanted across multiple MOE-related changes.
Learnt from: jhaotingc
Repo: NVIDIA/TensorRT-LLM PR: 7856
File: cpp/tensorrt_llm/thop/fp8BlockScaleMoe.cpp:159-166
Timestamp: 2025-09-19T21:28:13.751Z
Learning: In TensorRT-LLM blockScaleMoe routing (cpp/tensorrt_llm/kernels/trtllmGenKernels/blockScaleMoe/runner.cu), the DeepSeek routing method performs reinterpret_cast<float*>(routingLogits) at line 89, which could cause issues if routing_logits are BF16. However, Qwen3-FP8 models use RenormalizeNaive routing method and are not affected by this dtype casting issue.
📚 Learning: 2025-08-08T05:10:38.906Z
Learnt from: sklevtsov-nvidia
Repo: NVIDIA/TensorRT-LLM PR: 3294
File: cpp/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/epilogue/fusion/sm90_visitor_scatter.hpp:0-0
Timestamp: 2025-08-08T05:10:38.906Z
Learning: The ScaledAccPerRowBiasPerColScaleScatter fusion in CUTLASS extensions (cpp/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/epilogue/fusion/sm90_visitor_scatter.hpp) is specifically designed for per-column scaling factors only, so it uses a fixed Stride<_0,_1,int64_t> rather than conditional stride logic.
Applied to files:
tensorrt_llm/_torch/modules/linear.py
📚 Learning: 2025-09-19T21:28:13.751Z
Learnt from: jhaotingc
Repo: NVIDIA/TensorRT-LLM PR: 7856
File: cpp/tensorrt_llm/thop/fp8BlockScaleMoe.cpp:159-166
Timestamp: 2025-09-19T21:28:13.751Z
Learning: In TensorRT-LLM blockScaleMoe routing (cpp/tensorrt_llm/kernels/trtllmGenKernels/blockScaleMoe/runner.cu), the DeepSeek routing method performs reinterpret_cast<float*>(routingLogits) at line 89, which could cause issues if routing_logits are BF16. However, Qwen3-FP8 models use RenormalizeNaive routing method and are not affected by this dtype casting issue.
Applied to files:
tensorrt_llm/_torch/modules/linear.pytensorrt_llm/_torch/modules/fused_moe/quantization.py
📚 Learning: 2025-08-19T12:45:11.997Z
Learnt from: amitz-nv
Repo: NVIDIA/TensorRT-LLM PR: 7033
File: tensorrt_llm/_torch/pyexecutor/model_engine.py:0-0
Timestamp: 2025-08-19T12:45:11.997Z
Learning: In tensorrt_llm/_torch/pyexecutor/model_engine.py, DoRA (Delta Orthogonal Rank Adaptation) functionality was removed from the PyTorch flow to eliminate issues with inverted DoRA detection logic. The original is_dora condition was checking if scaling_vec_pointer == 0, which was potentially incorrect.
Applied to files:
tensorrt_llm/_torch/modules/linear.py
📚 Learning: 2025-08-08T22:03:40.707Z
Learnt from: sklevtsov-nvidia
Repo: NVIDIA/TensorRT-LLM PR: 3294
File: cpp/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_kernels.cu:1198-1209
Timestamp: 2025-08-08T22:03:40.707Z
Learning: In the CUTLASS MoE kernels (cpp/tensorrt_llm/cutlass_extensions), when `layout_info.fusion` is set to `TmaWarpSpecializedGroupedGemmInput::EpilogueFusion::FINALIZE`, the `router_scales` parameter must be non-null by design. The fused finalize kernel epilogue does not perform nullptr checks and requires valid router scales to function correctly. This is an implicit contract that callers must satisfy when enabling the FINALIZE fusion mode.
Applied to files:
tensorrt_llm/_torch/modules/linear.pytensorrt_llm/_torch/modules/fused_moe/quantization.py
📚 Learning: 2025-09-29T15:14:28.503Z
Learnt from: amitz-nv
Repo: NVIDIA/TensorRT-LLM PR: 8063
File: tensorrt_llm/lora_manager.py:1080-1112
Timestamp: 2025-09-29T15:14:28.503Z
Learning: In tensorrt_llm/lora_manager.py, when calculating part_sizes for attn_qkv fused LoRA modules, the sizes are correctly multiplied by tp_size because model_config.num_heads and model_config.num_kv_heads are already divided by tp_size (per-TP-rank values), so multiplication is needed to get the original full concatenated dimension size. The interleave_fused_lora_weights_for_tp function provides proper validation with asserts for total size and TP divisibility.
Applied to files:
tensorrt_llm/_torch/modules/linear.pytensorrt_llm/_torch/modules/fused_moe/quantization.py
📚 Learning: 2025-09-29T15:14:28.503Z
Learnt from: amitz-nv
Repo: NVIDIA/TensorRT-LLM PR: 8063
File: tensorrt_llm/lora_manager.py:1080-1112
Timestamp: 2025-09-29T15:14:28.503Z
Learning: In tensorrt_llm/lora_manager.py, when calculating part_sizes for attn_qkv fused LoRA modules, the sizes are correctly multiplied by tp_size because model_config.num_heads and model_config.num_kv_heads are already divided by tp_size (per-TP-rank values), so multiplication is needed to get the original full concatenated dimension size. The interleave_fused_lora_weights_for_tp function provides proper validation.
Applied to files:
tensorrt_llm/_torch/modules/linear.pytensorrt_llm/_torch/models/checkpoints/hf/nemotron_h_weight_mapper.py
📚 Learning: 2025-08-09T20:57:04.084Z
Learnt from: sklevtsov-nvidia
Repo: NVIDIA/TensorRT-LLM PR: 3294
File: cpp/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_gemm_tma_warp_specialized_input.cu:118-127
Timestamp: 2025-08-09T20:57:04.084Z
Learning: In the CUTLASS MoE finalize fusion implementation (cpp/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_gemm_tma_warp_specialized_input.cu), when setting `fused_finalize_epilogue.stride_final_output` with shape `(hidden_size, num_output_tokens, 1)`, the `num_rows_in_final_output` should be set to `num_output_tokens` (not `hidden_size`) because of a swap+transpose operation that maps rows of the output tensor to `hidden_size` and columns to `num_output_tokens`.
Applied to files:
tensorrt_llm/_torch/modules/linear.py
📚 Learning: 2025-10-20T16:54:09.824Z
Learnt from: nvchenghaoz
Repo: NVIDIA/TensorRT-LLM PR: 8469
File: tensorrt_llm/_torch/auto_deploy/custom_ops/rms_norm.py:6-6
Timestamp: 2025-10-20T16:54:09.824Z
Learning: In tensorrt_llm/_torch/auto_deploy/custom_ops/rms_norm.py, the import `from ...modules.mamba.layernorm_gated import _layer_norm_fwd` is correct and should not be changed to modules.fla.layernorm_gated. The _layer_norm_fwd function exists in both modules/mamba/layernorm_gated.py and modules/fla/layernorm_gated.py, but the mamba version is the intended implementation for this use case.
Applied to files:
tensorrt_llm/_torch/modules/linear.py
📚 Learning: 2025-10-20T17:07:18.745Z
Learnt from: nvchenghaoz
Repo: NVIDIA/TensorRT-LLM PR: 8469
File: tensorrt_llm/_torch/auto_deploy/models/patches/nemotron_h.py:98-116
Timestamp: 2025-10-20T17:07:18.745Z
Learning: In NemotronH models (tensorrt_llm/_torch/auto_deploy/models/patches/nemotron_h.py), the gate (self.gate) returns topk_indices and topk_weights that are already in the correct shape to be passed directly to torch_ops.auto_deploy.torch_moe without needing to reshape them when hidden_states is flattened.
Applied to files:
tensorrt_llm/_torch/models/checkpoints/hf/nemotron_h_weight_mapper.py
📚 Learning: 2025-08-14T23:23:27.449Z
Learnt from: djns99
Repo: NVIDIA/TensorRT-LLM PR: 6915
File: cpp/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_kernels.cu:4010-4012
Timestamp: 2025-08-14T23:23:27.449Z
Learning: For MOE (Mixture of Experts) code reviews in TensorRT-LLM, avoid repeatedly suggesting finalize fusion validation checks and safety assertions. The user djns99 has indicated these suggestions are repetitive and unwanted across multiple MOE-related changes.
Applied to files:
tensorrt_llm/_torch/modules/fused_moe/quantization.py
🧬 Code graph analysis (3)
tensorrt_llm/_torch/models/checkpoints/hf/weight_mapper.py (1)
tensorrt_llm/_torch/modules/linear.py (1)
has_nvfp4(2185-2188)
tensorrt_llm/_torch/modules/linear.py (1)
tensorrt_llm/_torch/utils.py (1)
unswizzle_sf(168-183)
tensorrt_llm/_torch/models/checkpoints/hf/nemotron_h_weight_mapper.py (4)
tensorrt_llm/_torch/utils.py (1)
split(393-401)tensorrt_llm/_torch/distributed/communicator.py (1)
tp_size(64-65)tensorrt_llm/_torch/models/modeling_utils.py (1)
config(525-526)tensorrt_llm/models/modeling_utils.py (1)
quant_algo(550-551)
🪛 Ruff (0.14.8)
tensorrt_llm/_torch/modules/fused_moe/quantization.py
2116-2116: Local variable block_scales_vec_size is assigned to but never used
Remove assignment to unused variable block_scales_vec_size
(F841)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Pre-commit Check
🔇 Additional comments (4)
tensorrt_llm/_torch/modules/linear.py (1)
1152-1170: Re-check NVFP4 weight_scale gating condition; it likely disables the realignment path entirelyThe new guard:
scale_rows = fp4_utils.pad_up(module.out_features, 128) scale_cols = fp4_utils.pad_up(module.in_features // module.scaling_vector_size, 4) if scale_rows * scale_cols != module.weight_scale.shape[0]: ...uses exactly the same formula as
create_weightswhen allocatingmodule.weight_scale, so for the standard NVFP4 pathscale_rows * scale_colswill always equalweight_scale.numel(). That means the unswizzle → pad → interleave block will never run, even whenrow_pad_sizeorcol_pad_sizeformodule.weightare non-zero.If the intent is to fully bypass weight_scale padding/realignment in all NVFP4 cases, that’s fine but should be explicitly documented, since the surrounding comment still says “Pad weight and weight_scale tensors…”. If instead you still need the adjustment when GEMM alignment padding is applied, the condition likely needs to be tied to the padded dimensions rather than the original
pad_up(..., 128/4)sizes.Can you confirm which behavior is desired here, and whether you’ve exercised this on the NVFP4 + multi-GPU configs that motivated the change?
tensorrt_llm/_torch/models/checkpoints/hf/nemotron_h_weight_mapper.py (1)
18-39: Mamba2 in_proj split helper and Nemotron-H MoE NVFP4 routing look consistent; clarify a couple of assumptionsThe refactored
_split_mamba2_mixer_in_projand the Nemotron-H MoE remapping logic look structurally sound:
- The splits along
dim=0match the expected[z, x, b, c, dt]layout (2 * d_inner + 2 * n_groups * d_state + nheads).- Rebuilding per-TP-rank chunks with
split(...)mirrors existing patterns elsewhere in the codebase.- For MoE experts, forcing
w3(gate_proj) weights andweight_scaleto[:0]in the NVFP4 case is a clear way to encode “no gate weights for Nemotron-H MoE”.Two minor points worth calling out:
is_nvfp4 = self.config.quant_config.quant_algo == "NVFP4"assumes a string-valuedquant_algo. In other places,quant_algois often compared toQuantAlgo.*. If there’s any chancequant_algocan be an enum, it may be safer to normalize (e.g., compare tostr(quant_algo)or use the sameQuantAlgoconstant everywhere).The
if weights[name].shape:check to distinguish NVFP4 vs FP8weight_scalerelies on FP8 using 0‑D scalars (torch.Size([]), which is falsy) and NVFP4 using non-empty shapes. That’s a subtle convention; consider a short comment like “FP8 stores scalar weight_scale (0‑D), NVFP4 stores tensors” to make this more obvious for future readers.Functionally, the changes look correct given those assumptions.
Also applies to: 61-67, 81-83, 117-129
tensorrt_llm/_torch/modules/fused_moe/quantization.py (2)
478-487: Using actual shard sizes for w3/w1 placement is a solid robustness improvementSwitching to
src_w3_size_shard/src_w1_size_shardand usingnarrow(...)for both w3 and w1 avoids hard-coding split points and makes this helper resilient to asymmetric or padded shapes, as well as partial-loading scenarios.The logic is straightforward and preserves the existing layout (
[w3, w1]stacked along dim 0).
707-735: FP8QDQ MoE weight padding for CUTLASS backend looks dimensionally correctThe new
post_load_weights:
- Early-exits for non-
"CUTLASS"moe_backend, so it won’t impact other paths.- Pads
w3_w1_weightandw2_weightalong their last two dims using(0, col_pad_size, 0, row_pad_size), which is appropriate for the[expert, M, K]layout.- Uses
row_alignment=32,col_alignment=16forw3_w1_weight, and alignsw2_weight.shape[2]to the samerow_alignmentto stay consistent with FC1’s intermediate dimension.This is a clean, localized alignment fix; just keep in mind that if the kernels ever start consuming the MoE biases, you may need similar padding there as well.
eacdedc to
cc6a155
Compare
|
/bot run --disable-fail-fast |
|
PR_Github #29114 [ run ] triggered by Bot. Commit: |
|
PR_Github #29114 [ run ] completed with state
|
cc6a155 to
c9ee717
Compare
252a779 to
051c033
Compare
|
/bot run --disable-fail-fast |
|
PR_Github #29534 [ run ] triggered by Bot. Commit: |
051c033 to
ca98381
Compare
|
/bot run --disable-fail-fast |
|
PR_Github #29581 [ run ] triggered by Bot. Commit: |
|
PR_Github #29691 [ run ] completed with state
|
d39a487 to
b7fcc14
Compare
|
/bot run --disable-fail-fast |
|
PR_Github #29767 [ run ] triggered by Bot. Commit: |
|
PR_Github #29767 [ run ] completed with state
|
|
/bot run --disable-fail-fast |
|
PR_Github #29849 [ run ] triggered by Bot. Commit: |
|
PR_Github #29849 [ run ] completed with state
|
|
/bot run --disable-fail-fast |
|
PR_Github #29925 [ run ] triggered by Bot. Commit: |
|
PR_Github #29925 [ run ] completed with state
|
|
/bot run --stage-list "DGX_H100-2_GPUs-PyTorch-Others-1" --disable-fail-fast |
…on-3-super * Verified 1/2/4 GPUs with TP, TEP for nano-v3 bf16/fp8/nvfp4 ckpts. * Verified 1/2/4/8 GPUs with TP/TEP for super-v3 bf16 ckpt. * Support cudagraph. * Support multi-stream for MoE shared_experts and routed_experts. * Support pipeline parallelism. TODO: * Support attention_dp for multi-gpugs. (will raise error or hanging for now). Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
b7fcc14 to
d08b512
Compare
|
/bot run --stage-list "DGX_H100-2_GPUs-PyTorch-Others-1" --disable-fail-fast |
|
PR_Github #29963 [ run ] triggered by Bot. Commit: |
|
PR_Github #29964 [ run ] triggered by Bot. Commit: |
|
PR_Github #29963 [ run ] completed with state |
|
PR_Github #29964 [ run ] completed with state |
|
/bot skip |
GitHub Bot Help
Provide a user friendly way for developers to interact with a Jenkins server. Run See details below for each supported subcommand. Details
Launch build/test pipelines. All previously running jobs will be killed.
kill
Kill all running builds associated with pull request. skip
Skip testing for latest commit on pull request. reuse-pipeline
Reuse a previous pipeline to validate current commit. This action will also kill all currently running builds associated with the pull request. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break. |
|
/bot skip --comment "all tests are passed" |
|
PR_Github #29980 [ skip ] triggered by Bot. Commit: |
|
PR_Github #29980 [ skip ] completed with state |
…NVIDIA#10118) Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com> Signed-off-by: Daniil Kulko <kulkodaniil@gmail.com>
4 participants
Features
TODOs
Summary by CodeRabbit
Bug Fixes
Refactor
✏️ Tip: You can customize this high-level summary in your review settings.
Description
Test Coverage
PR Checklist
Please review the following before submitting your PR:
PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.
GitHub Bot Help
/bot [-h] ['run', 'kill', 'skip', 'reuse-pipeline'] ...Provide a user friendly way for developers to interact with a Jenkins server.
Run
/bot [-h|--help]to print this help message.See details below for each supported subcommand.
Details
run [--reuse-test (optional)pipeline-id --disable-fail-fast --skip-test --stage-list "A10-PyTorch-1, xxx" --gpu-type "A30, H100_PCIe" --test-backend "pytorch, cpp" --add-multi-gpu-test --only-multi-gpu-test --disable-multi-gpu-test --post-merge --extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx" --detailed-log --debug(experimental)]Launch build/test pipelines. All previously running jobs will be killed.
--reuse-test (optional)pipeline-id(OPTIONAL) : Allow the new pipeline to reuse build artifacts and skip successful test stages from a specified pipeline or the last pipeline if no pipeline-id is indicated. If the Git commit ID has changed, this option will be always ignored. The DEFAULT behavior of the bot is to reuse build artifacts and successful test results from the last pipeline.--disable-reuse-test(OPTIONAL) : Explicitly prevent the pipeline from reusing build artifacts and skipping successful test stages from a previous pipeline. Ensure that all builds and tests are run regardless of previous successes.--disable-fail-fast(OPTIONAL) : Disable fail fast on build/tests/infra failures.--skip-test(OPTIONAL) : Skip all test stages, but still run build stages, package stages and sanity check stages. Note: Does NOT update GitHub check status.--stage-list "A10-PyTorch-1, xxx"(OPTIONAL) : Only run the specified test stages. Examples: "A10-PyTorch-1, xxx". Note: Does NOT update GitHub check status.--gpu-type "A30, H100_PCIe"(OPTIONAL) : Only run the test stages on the specified GPU types. Examples: "A30, H100_PCIe". Note: Does NOT update GitHub check status.--test-backend "pytorch, cpp"(OPTIONAL) : Skip test stages which don't match the specified backends. Only support [pytorch, cpp, tensorrt, triton]. Examples: "pytorch, cpp" (does not run test stages with tensorrt or triton backend). Note: Does NOT update GitHub pipeline status.--only-multi-gpu-test(OPTIONAL) : Only run the multi-GPU tests. Note: Does NOT update GitHub check status.--disable-multi-gpu-test(OPTIONAL) : Disable the multi-GPU tests. Note: Does NOT update GitHub check status.--add-multi-gpu-test(OPTIONAL) : Force run the multi-GPU tests in addition to running L0 pre-merge pipeline.--post-merge(OPTIONAL) : Run the L0 post-merge pipeline instead of the ordinary L0 pre-merge pipeline.--extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx"(OPTIONAL) : Run the ordinary L0 pre-merge pipeline and specified test stages. Examples: --extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx".--detailed-log(OPTIONAL) : Enable flushing out all logs to the Jenkins console. This will significantly increase the log volume and may slow down the job.--debug(OPTIONAL) : Experimental feature. Enable access to the CI container for debugging purpose. Note: Specify exactly one stage in thestage-listparameter to access the appropriate container environment. Note: Does NOT update GitHub check status.For guidance on mapping tests to stage names, see
docs/source/reference/ci-overview.mdand the
scripts/test_to_stage_mapping.pyhelper.kill
killKill all running builds associated with pull request.
skip
skip --comment COMMENTSkip testing for latest commit on pull request.
--comment "Reason for skipping build/test"is required. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.reuse-pipeline
reuse-pipelineReuse a previous pipeline to validate current commit. This action will also kill all currently running builds associated with the pull request. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.