[https://nvbugs/5889841][fix] Add custom option class to allow subcommand help to work.

[FrankD412]

Summary by CodeRabbit

Release Notes

• Bug Fixes
  • Improved the bench command help system to function independently of model requirements, enabling users to access documentation without specifying a model argument.

This PR adds an option class that allows child commands to print their help without needing to specify the --model parameter for the root trtllm-bench command. Previously you were required to specify a model name (or random string) to get the help to print. This is no longer required:

trtllm-bench prepare-dataset --help yields:

[TensorRT-LLM] TensorRT LLM version: 1.3.0rc6
Usage: trtllm-bench prepare-dataset [OPTIONS] COMMAND [ARGS]...

  Prepare dataset for benchmarking with trtllm-bench.

Options:
  --output TEXT              Output json filename.  [default:
                             preprocessed_dataset.json]
  --random-seed INTEGER      random seed for token_ids  [default: 420]
  --task-id INTEGER          LoRA task id  [default: -1]
  --rand-task-id INTEGER...  Random LoRA Tasks
  --lora-dir TEXT            Directory containing LoRA adapters
  --log-level [info|debug]   Logging level.  [default: info]
  --trust-remote-code        Trust remote code.
  --help                     Show this message and exit.

Commands:
  real-dataset     Prepare dataset from real dataset.
  token-norm-dist  Prepare synthetic dataset by generating random tokens...
  token-unif-dist  Prepare synthetic dataset by generating random tokens...

Incorrect sub-options continue to yield appropriate errors.

user@06fb1629be1e:/workspace/benchmark_toolkit$ trtllm-bench -m meta-llama/Llama-3.1-8B   prepare-dataset   --stdout   --num-requests 200   --synthetic   --input-mean 256   --output-mean 256

[TensorRT-LLM] TensorRT LLM version: 1.3.0rc6
Usage: trtllm-bench prepare-dataset [OPTIONS] COMMAND [ARGS]...
Try 'trtllm-bench prepare-dataset --help' for help.

Error: No such option: --stdout Did you mean --output?

And missing the required argument when using subcommands still works too:

user@06fb1629be1e:/workspace/benchmark_toolkit$ trtllm-bench prepare-dataset   --stdout   --num-requests 200   --synthetic   --input-mean 256   --output-mean 256
...
[TensorRT-LLM] TensorRT LLM version: 1.3.0rc6
Usage: trtllm-bench [OPTIONS] COMMAND [ARGS]...
Try 'trtllm-bench --help' for help.

Error: Missing option '--model' / '-m'.

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

/bot [-h] ['run', 'kill', 'skip', 'reuse-pipeline'] ...

Provide a user friendly way for developers to interact with a Jenkins server.

Run /bot [-h|--help] to print this help message.

See details below for each supported subcommand.

Details

run [--reuse-test (optional)pipeline-id --disable-fail-fast --skip-test --stage-list "A10-PyTorch-1, xxx" --gpu-type "A30, H100_PCIe" --test-backend "pytorch, cpp" --add-multi-gpu-test --only-multi-gpu-test --disable-multi-gpu-test --post-merge --extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx" --detailed-log --debug(experimental)]

Launch build/test pipelines. All previously running jobs will be killed.

--reuse-test (optional)pipeline-id (OPTIONAL) : Allow the new pipeline to reuse build artifacts and skip successful test stages from a specified pipeline or the last pipeline if no pipeline-id is indicated. If the Git commit ID has changed, this option will be always ignored. The DEFAULT behavior of the bot is to reuse build artifacts and successful test results from the last pipeline.

--disable-reuse-test (OPTIONAL) : Explicitly prevent the pipeline from reusing build artifacts and skipping successful test stages from a previous pipeline. Ensure that all builds and tests are run regardless of previous successes.

--disable-fail-fast (OPTIONAL) : Disable fail fast on build/tests/infra failures.

--skip-test (OPTIONAL) : Skip all test stages, but still run build stages, package stages and sanity check stages. Note: Does NOT update GitHub check status.

--stage-list "A10-PyTorch-1, xxx" (OPTIONAL) : Only run the specified test stages. Examples: "A10-PyTorch-1, xxx". Note: Does NOT update GitHub check status.

--gpu-type "A30, H100_PCIe" (OPTIONAL) : Only run the test stages on the specified GPU types. Examples: "A30, H100_PCIe". Note: Does NOT update GitHub check status.

--test-backend "pytorch, cpp" (OPTIONAL) : Skip test stages which don't match the specified backends. Only support [pytorch, cpp, tensorrt, triton]. Examples: "pytorch, cpp" (does not run test stages with tensorrt or triton backend). Note: Does NOT update GitHub pipeline status.

--only-multi-gpu-test (OPTIONAL) : Only run the multi-GPU tests. Note: Does NOT update GitHub check status.

--disable-multi-gpu-test (OPTIONAL) : Disable the multi-GPU tests. Note: Does NOT update GitHub check status.

--add-multi-gpu-test (OPTIONAL) : Force run the multi-GPU tests in addition to running L0 pre-merge pipeline.

--post-merge (OPTIONAL) : Run the L0 post-merge pipeline instead of the ordinary L0 pre-merge pipeline.

--extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx" (OPTIONAL) : Run the ordinary L0 pre-merge pipeline and specified test stages. Examples: --extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx".

--detailed-log (OPTIONAL) : Enable flushing out all logs to the Jenkins console. This will significantly increase the log volume and may slow down the job.

--debug (OPTIONAL) : Experimental feature. Enable access to the CI container for debugging purpose. Note: Specify exactly one stage in the stage-list parameter to access the appropriate container environment. Note: Does NOT update GitHub check status.

For guidance on mapping tests to stage names, see docs/source/developer-guide/ci-overview.md
and the scripts/test_to_stage_mapping.py helper.

kill

kill

Kill all running builds associated with pull request.

skip

skip --comment COMMENT

Skip testing for latest commit on pull request. --comment "Reason for skipping build/test" is required. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.

reuse-pipeline

reuse-pipeline

Reuse a previous pipeline to validate current commit. This action will also kill all currently running builds associated with the pull request. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.

[FrankD412]

/bot run

[coderabbitai]

📝 Walkthrough

Walkthrough

This change makes the --model argument optional when requesting help in the bench command by introducing a custom NotRequiredForHelp click.Option subclass that exempts help requests from requiring the model parameter.

Changes

Cohort / File(s)	Summary
Bench Command Help Handling tensorrt_llm/commands/bench.py	Introduced NotRequiredForHelp class to override click.Option parsing behavior, allowing the --model option to be bypassed when --help is requested. Added early return guard in main function to handle None model values.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 1 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check	❓ Inconclusive	The PR description contains detailed context and examples explaining the issue and solution, but key sections from the template (Description, Test Coverage, PR Checklist items) are incomplete or missing.	Add a formal Description section explaining the issue and solution concisely. Specify the relevant test case(s) that validate this change under Test Coverage. Confirm which PR Checklist items apply to this change.

✅ Passed checks (1 passed)

Check name	Status	Explanation
Title check	✅ Passed	The pull request title clearly and concisely summarizes the main change: adding a custom option class to fix subcommand help functionality without requiring the root --model parameter.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

• 📝 Generate docstrings (stacked PR)
• 📝 Generate docstrings (commit on current branch)

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

Tip

Try Coding Plans. Let us write the prompt for your AI agent so you can ship faster (with fewer bugs).
Share your feedback on Discord.

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

tensorrt_llm/commands/bench.py (1)

59-66: ⚠️ Potential issue | 🟡 Minor

model type hint should be Optional[str] after this change

With NotRequiredForHelp, model can now be None at runtime (when --help is passed without --model), but the function signature still declares model: str. This misleads callers and type-checkers.

♻️ Proposed fix

 def main(
     ctx,
-    model: str,
+    model: Optional[str],
     model_path: Path,
     workspace: Path,
     log_level: str,
     revision: Optional[str],
 ) -> None:

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@tensorrt_llm/commands/bench.py` around lines 59 - 66, Update the type
annotation for the parameter model in the function main
(tensorrt_llm/commands/bench.py) from str to Optional[str] to reflect that it
can be None when NotRequiredForHelp is used; ensure Optional is imported from
typing if not already, and scan usages inside main (and any direct callers) for
places that assume model is a non-None str and guard or handle the None case
accordingly (e.g., conditional checks or early returns) so the signature and
runtime behavior stay consistent.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@tensorrt_llm/commands/bench.py`:
- Around line 17-21: The handle_parse_result method mutates the shared Click
Parameter instance by setting self.required = False when help flags are present,
causing the change to persist across invocations; update handle_parse_result
(the method name) to save the original required state (orig_required =
self.required) before changing it, set self.required = False only for the
duration of the call to super().handle_parse_result(ctx, opts, args), then
restore self.required = orig_required after the super() call so the module-level
option (e.g., the --model parameter) is not permanently altered.

---

Outside diff comments:
In `@tensorrt_llm/commands/bench.py`:
- Around line 59-66: Update the type annotation for the parameter model in the
function main (tensorrt_llm/commands/bench.py) from str to Optional[str] to
reflect that it can be None when NotRequiredForHelp is used; ensure Optional is
imported from typing if not already, and scan usages inside main (and any direct
callers) for places that assume model is a non-None str and guard or handle the
None case accordingly (e.g., conditional checks or early returns) so the
signature and runtime behavior stay consistent.

---

ℹ️ Review info

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 08318ab and 03f0000.

📒 Files selected for processing (1)

• tensorrt_llm/commands/bench.py

[tensorrt-cicd]

PR_Github #36827 [ run ] triggered by Bot. Commit: 03f0000 Link to invocation

[tensorrt-cicd]

PR_Github #36827 [ run ] completed with state FAILURE. Commit: 03f0000
/LLM/release-1.2/L0_MergeRequest_PR pipeline #396 completed with status: 'FAILURE'

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

[FrankD412]

/bot run

[tensorrt-cicd]

PR_Github #36829 [ run ] triggered by Bot. Commit: 03f0000 Link to invocation

[tensorrt-cicd]

PR_Github #36829 [ run ] completed with state SUCCESS. Commit: 03f0000
/LLM/release-1.2/L0_MergeRequest_PR pipeline #397 completed with status: 'SUCCESS'

Link to invocation