From 423692ce6f5e3024cc16494b17b507265d125439 Mon Sep 17 00:00:00 2001
From: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
Date: Sun, 3 Aug 2025 16:07:27 +0000
Subject: [PATCH 01/10] doc: Add DeepSeek R1 deployment guide.

Signed-off-by: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
---
 .pre-commit-config.yaml                       |   2 +-
 ...eployment-guide-for-trt-llm-deepseek-r1.md | 377 ++++++++++++++++++
 2 files changed, 378 insertions(+), 1 deletion(-)
 create mode 100644 examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md

diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index 00516b1afa79..74f830f07db4 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -27,7 +27,7 @@ repos:
         args: [--allow-multiple-documents]
         exclude: ".*/gitlab/.*.yml"
     -   id: trailing-whitespace
-        exclude: '\.patch$'
+        exclude: '\.(patch|md)$'
     -   id: check-toml
     -   id: mixed-line-ending
         args: [--fix=lf]
diff --git a/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md b/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md
new file mode 100644
index 000000000000..b85928fc73e5
--- /dev/null
+++ b/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md
@@ -0,0 +1,377 @@
+# Deployment Guide for TensorRT-LLM DeepSeek R1 FP8 and NVFP4
+
+# Introduction
+
+This deployment guide provides step-by-step instructions for running the DeepSeek R1 model using TensorRT-LLM with FP8 and NVFP4 quantization, optimized for NVIDIA GPUs. It covers the complete setup required; from accessing model weights and preparing the software environment to configuring TensorRT-LLM parameters, launching the server, and validating inference output.
+
+The guide is intended for developers and practitioners seeking high-throughput or low-latency inference using NVIDIA’s accelerated stack—starting with the PyTorch container from NGC, then installing TensorRT-LLM for model serving, FlashInfer for optimized CUDA kernels, and ModelOpt to enable FP8 and NVFP4 quantized execution.
+
+# Prerequisites
+
+GPU: NVIDIA Blackwell or Hopper Architecture  
+OS: Linux  
+Drivers: CUDA Driver 575 or Later  
+Docker with NVIDIA Container Toolkit installed  
+Python3 and python3-pip (Optional, for accuracy evaluation only)
+
+# Models
+
+* FP8 model: [DeepSeek-R1](https://huggingface.co/deepseek-ai/DeepSeek-R1)  
+* NVFP4 model: [DeepSeek-R1-FP4](https://huggingface.co/nvidia/DeepSeek-R1-FP4)
+
+
+Note that NVFP4 is only supported on NVIDIA Blackwell platform.
+
+# Deployment Steps
+
+## Run Docker Container
+
+Run the docker container using the TensorRT-LLM NVIDIA NGC image.
+
+```shell
+docker run --rm -it \
+--ipc=host \
+--gpus all \
+-p 8000:8000 \
+-v ~/.cache:/root/.cache:rw \
+--name tensorrt_llm \
+nvcr.io/nvidia/tensorrt-llm/release:1.0.0rc5 \
+/bin/bash
+```
+
+Note: 
+
+* You can mount additional directories and paths using the \-v \<local\_path\>:\<path\> flag if needed, such as mounting the downloaded weight paths.  
+* The command mounts your user .cache directory to save the downloaded model checkpoints which are saved to \~/.cache/huggingface/hub/ by default. This prevents having to redownload the weights each time you rerun the container. If the \~/.cache directory doesn’t exist please create it using  mkdir \~/.cache  
+* The command also maps port **8000** from the container to your host so you can access the LLM API endpoint from your host  
+* See the [https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tensorrt-llm/containers/release/tags](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tensorrt-llm/containers/release/tags) for all the available containers. The containers published in the main branch weekly have “rcN” suffix, while the monthly release with QA tests has no “rcN” suffix. Use the rc release to get the latest model and feature support.
+
+If you want to use latest main branch, you can choose to build from source to install TensorRT-LLM, the steps refer to [https://nvidia.github.io/TensorRT-LLM/latest/installation/build-from-source-linux.html](https://nvidia.github.io/TensorRT-LLM/latest/installation/build-from-source-linux.html) 
+
+## Creating the TRT-LLM Server config
+
+We create a YAML configuration file /tmp/config.yml for the TensorRT-LLM Server and populate it with the following recommended performance settings.
+
+```shell
+EXTRA_LLM_API_FILE=/tmp/config.yml
+
+cat << EOF > ${EXTRA_LLM_API_FILE}
+enable_attention_dp: true
+cuda_graph_config:
+  enable_padding: true
+  max_batch_size: 1024
+kv_cache_config:
+  dtype: fp8
+EOF
+```
+
+For FP8 model, we need extra `moe_config`:
+
+```shell
+EXTRA_LLM_API_FILE=/tmp/config.yml
+
+cat << EOF > ${EXTRA_LLM_API_FILE}
+enable_attention_dp: true
+cuda_graph_config:
+  enable_padding: true
+  max_batch_size: 1024
+kv_cache_config:
+  dtype: fp8
+moe_config:
+  backend: DEEPGEMM
+EOF
+```
+
+## Launch the TRT-LLM Server
+
+Below is an example command to launch the TRT-LLM server with the DeepSeek-R1 model from within the container. The command is specifically configured for the 1024/1024 Input/Output Sequence Length test. The explanation of each flag is shown in the “Configs and Parameters” section.
+
+```shell
+trtllm-serve deepseek-ai/DeepSeek-R1 \
+    --host 0.0.0.0 \
+    --port 8000 \
+    --backend pytorch \
+    --max_batch_size 1024 \
+    --max_num_tokens 2048 \
+    --max_seq_len 2048 \
+    --kv_cache_free_gpu_memory_fraction 0.9 \
+    --tp_size 4 \
+    --ep_size 4 \
+    --trust_remote_code \
+    --extra_llm_api_options ${EXTRA_LLM_API_FILE}
+```
+
+After the server is set up, the client can now send prompt requests to the server and receive results.
+
+## Configs and Parameters
+
+These options are used directly on the command line when you start the `trtllm-serve` process.
+#### `--tp_size`
+
+&emsp;**Description:** Sets the **tensor-parallel size**. This should typically match the number of GPUs you intend to use for a single model instance.
+
+#### `--ep_size`
+
+&emsp;**Description:** Sets the **expert-parallel size** for Mixture-of-Experts (MoE) models. Like `tp_size`, this should generally match the number of GPUs you're using. This setting has no effect on non-MoE models.
+
+#### `--kv_cache_free_gpu_memory_fraction`
+
+&emsp;**Description:** A value between 0.0 and 1.0 that specifies the fraction of free GPU memory to reserve for the KV cache after the model is loaded. Since memory usage can fluctuate, this buffer helps prevent out-of-memory (OOM) errors.
+
+&emsp;**Recommendation:** If you experience OOM errors, try reducing this value to **0.8** or lower.
+
+#### `--backend pytorch`
+
+&emsp;**Description:** Tells TensorRT-LLM to use the **pytorch** backend.
+
+#### `--max_batch_size`
+
+&emsp;**Description:** The maximum number of user requests that can be grouped into a single batch for processing.
+
+#### `--max_num_tokens`
+
+&emsp;**Description:** The maximum total number of tokens (across all requests) allowed inside a single scheduled batch.
+
+#### `--max_seq_len`
+
+&emsp;**Description:** The maximum possible sequence length for a single request, including both input and generated output tokens.
+
+#### `--trust_remote_code`
+
+&emsp;**Description:** Allows TensorRT-LLM to download models and tokenizers from Hugging Face. This flag is passed directly to the Hugging Face API.
+
+
+#### Extra LLM API Options (YAML Configuration)
+
+These options provide finer control over performance and are set within a YAML file passed to the trtllm-serve command via the \--extra\_llm\_api\_options argument.
+
+#### `kv_cache_config`
+
+&emsp;**Description**: A section for configuring the Key-Value (KV) cache.
+
+&emsp;**Options**: 
+
+&emsp;&emsp;dtype: Sets the data type for the KV cache.
+
+&emsp;&emsp;**Default**: auto (uses the data type specified in the model checkpoint).
+
+#### `cuda_graph_config`
+
+&emsp;**Description**: A section for configuring CUDA graphs to optimize performance.
+
+&emsp;**Options**:
+
+&emsp;&emsp;enable\_padding: If true, input batches are padded to the nearest cuda\_graph\_batch\_size. This can significantly improve performance.
+
+&emsp;&emsp;**Default**: false
+
+&emsp;&emsp;max\_batch\_size: Sets the maximum batch size for which a CUDA graph will be created.
+
+&emsp;&emsp;**Default**: 0
+
+&emsp;&emsp;**Recommendation**: Set this to the same value as the \--max\_batch\_size command-line option.
+
+&emsp;&emsp;batch\_sizes: A specific list of batch sizes to create CUDA graphs for.
+
+&emsp;&emsp;**Default**: None
+
+#### `moe_config`
+
+&emsp;**Description**: Configuration for Mixture-of-Experts (MoE) models.
+
+&emsp;**Options**:
+
+&emsp;&emsp;backend: The backend to use for MoE operations.
+
+&emsp;&emsp;**Default**: CUTLASS
+
+#### `attention_backend`
+
+&emsp;**Description**: The backend to use for attention calculations.
+
+&emsp;**Default**: TRTLLM
+
+See the [https://github.com/nvidia/TensorRT-LLM/blob/main/tensorrt\_llm/llmapi/llm\_args.py\#L1894](https://github.com/nvidia/TensorRT-LLM/blob/main/tensorrt_llm/llmapi/llm_args.py#L1894) TorchLlmArgs class for the full list of options which can be used in the extra\_llm\_api\_options`.`
+
+# Testing API Endpoint
+
+## Basic Test
+
+Start a new terminal on the host to test the TensorRT-LLM server you just launched. 
+
+You can query the health/readiness of the server using:
+
+```shell
+curl -s -o /dev/null -w "Status: %{http_code}\n" "http://localhost:8000/health"
+```
+
+When the `Status: 200` code is returned, the server is ready for queries. Note that the very first query may take longer due to initialization and compilation.
+
+After the TRT-LLM server is set up and shows Application startup complete, you can send requests to the server. 
+
+```shell
+curl http://localhost:8000/v1/completions -H "Content-Type: application/json"  -d '{
+      "model": "deepseek-ai/DeepSeek-R1",
+      "prompt": "Where is New York?",
+      "max_tokens": 16,
+      "temperature": 0
+}'
+```
+
+Here is an example response, showing that the TRT-LLM server returns “New York is a state located in the northeastern United States. It is bordered by”, completing the input sequence.
+
+```json
+{"id":"cmpl-bc1393d529ce485c961d9ffee5b25d72","object":"text_completion","created":1753843963,"model":"deepseek-ai/DeepSeek-R1","choices":[{"index":0,"text":" New York is a state located in the northeastern United States. It is bordered by","token_ids":null,"logprobs":null,"context_logits":null,"finish_reason":"length","stop_reason":null,"disaggregated_params":null}],"usage":{"prompt_tokens":6,"total_tokens":22,"completion_tokens":16},"prompt_token_ids":null}
+```
+
+## Troubleshooting Tips
+
+* If you encounter CUDA out-of-memory errors, try reducing max\_batch\_size or max\_seq\_len  
+* Ensure your model checkpoints are compatible with the expected format  
+* For performance issues, check GPU utilization with nvidia-smi while the server is running  
+* If the container fails to start, verify that the NVIDIA Container Toolkit is properly installed  
+* For connection issues, make sure port 8000 is not being used by another application
+
+## Running Evaluations to Verify Accuracy (Optional)
+
+We use the lm-eval tool to test the model’s accuracy. For more information see [https://github.com/EleutherAI/lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness).
+
+To run the evaluation harness exec into the running TensorRT-LLM container and install with this command:
+
+```shell
+docker exec -it tensorrt_llm /bin/bash
+
+pip install lm_eval
+```
+
+FP8 command for GSM8K
+
+* Note: The tokenizer will add BOS (beginning of sentence token) before input prompt by default which leads to accuracy regression on GSM8K task for DeepSeek R1 model. So, set add\_special\_tokens=False to avoid it.
+
+```
+MODEL_PATH=deepseek-ai/DeepSeek-R1
+
+lm_eval --model local-completions  --tasks gsm8k --batch_size 256 --gen_kwargs temperature=0.0,add_special_tokens=False --num_fewshot 5 --model_args model=${MODEL_PATH},base_url=http://localhost:8000/v1/completions,num_concurrent=32,max_retries=20,tokenized_requests=False --log_samples --output_path trtllm.fp8.gsm8k
+```
+
+Sample result in Blackwell. 
+
+```
+|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
+|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
+|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.9348|±  |0.0068|
+|     |       |strict-match    |     5|exact_match|↑  |0.8870|±  |0.0087|
+```
+
+FP4 command for GSM8K
+
+* Note: The tokenizer will add BOS before input prompt by default, which leads to accuracy regression on GSM8K task for DeepSeek R1 model. So set add\_special\_tokens=False to avoid it.
+
+```shell
+MODEL_PATH=nvidia/DeepSeek-R1-FP4
+
+lm_eval --model local-completions  --tasks gsm8k --batch_size 256 --gen_kwargs temperature=0.0,add_special_tokens=False --num_fewshot 5 --model_args model=${MODEL_PATH},base_url=http://localhost:8000/v1/completions,num_concurrent=32,max_retries=20,tokenized_requests=False --log_samples --output_path trtllm.fp4.gsm8k
+```
+
+Sample result in Blackwell
+
+```shell
+|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
+|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
+|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.9356|±  |0.0068|
+|     |       |strict-match    |     5|exact_match|↑  |0.8393|±  |0.0101|
+```
+
+# Benchmarking Performance
+
+To benchmark the performance of your TensorRT-LLM server you can leverage the built-in “benchmark\_serving.py” script. To do this first creating a wrapper [bench.sh](http://bench.sh) script.
+
+```shell
+cat <<EOF >  bench.sh
+concurrency_list="1 2 4 8 16 32 64 128 256"
+multi_round=5
+isl=1024
+osl=1024
+result_dir=/tmp/deepseek_r1_output
+
+for concurrency in ${concurrency_list}; do
+    num_prompts=$((concurrency * multi_round))
+    python -m tensorrt_llm.serve.scripts.benchmark_serving \
+        --model deepseek-ai/DeepSeek-R1 \
+        --backend openai \
+        --dataset-name "random" \
+        --random-input-len ${isl} \
+        --random-output-len ${osl} \
+        --random-prefix-len 0 \
+        --random-ids \
+        --num-prompts ${num_prompts} \
+        --max-concurrency ${concurrency} \
+        --ignore-eos \
+        --tokenize-on-client \
+        --percentile-metrics "ttft,tpot,itl,e2el"
+done
+EOF
+chmod +x bench.sh
+```
+
+To benchmark the FP4 model, replace \--model deepseek-ai/DeepSeek-R1 with \--model nvidia/DeepSeek-R1-FP4.
+
+If you want to save the results to a file add the following options.
+
+```shell
+--save-result \
+--result-dir "${result_dir}" \
+--result-filename "concurrency_${concurrency}.json"
+```
+
+For more benchmarking options see. [https://github.com/NVIDIA/TensorRT-LLM/blob/main/tensorrt\_llm/serve/scripts/benchmark\_serving.py](https://github.com/NVIDIA/TensorRT-LLM/blob/main/tensorrt_llm/serve/scripts/benchmark_serving.py) 
+
+Run bench.sh to begin a serving benchmark. This will take a long time if you run all the concurrencies mentioned in the above bench.sh script.
+
+```shell
+./bench.sh
+```
+
+Sample TensorRT-LLM serving benchmark output. Your results may vary due to ongoing software optimizations.
+
+```
+============ Serving Benchmark Result ============
+Successful requests:                      16
+Benchmark duration (s):                   17.66
+Total input tokens:                       16384
+Total generated tokens:                   16384
+Request throughput (req/s):               [result]
+Output token throughput (tok/s):          [result]
+Total Token throughput (tok/s):           [result]
+User throughput (tok/s):                  [result]
+---------------Time to First Token----------------
+Mean TTFT (ms):                           [result]
+Median TTFT (ms):                         [result]
+P99 TTFT (ms):                            [result]
+-----Time per Output Token (excl. 1st token)------
+Mean TPOT (ms):                           [result]
+Median TPOT (ms):                         [result]
+P99 TPOT (ms):                            [result]
+---------------Inter-token Latency----------------
+Mean ITL (ms):                            [result]
+Median ITL (ms):                          [result]
+P99 ITL (ms):                             [result]
+----------------End-to-end Latency----------------
+Mean E2EL (ms):                           [result]
+Median E2EL (ms):                         [result]
+P99 E2EL (ms):                            [result]
+==================================================
+```
+
+## Key Metrics
+
+* Median Time to First Token (TTFT)  
+  * The typical time elapsed from when a request is sent until the first output token is generated.  
+* Median Time Per Output Token (TPOT)  
+  * The typical time required to generate each token *after* the first one.   
+* Median Inter-Token Latency (ITL)  
+  * The typical time delay between the completion of one token and the completion of the next.  
+* Median End-to-End Latency (E2EL)  
+  * The typical total time from when a request is submitted until the final token of the response is received.   
+* Total Token Throughput  
+  * The combined rate at which the system processes both input (prompt) tokens and output (generated) tokens. 

From ecbd20b8c3cea90c5d7ed4d0e7bd33fa3b40d686 Mon Sep 17 00:00:00 2001
From: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
Date: Mon, 4 Aug 2025 01:56:25 +0000
Subject: [PATCH 02/10] Fix doc.

Signed-off-by: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
---
 .../deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md   | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md b/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md
index b85928fc73e5..09a09eaf4d21 100644
--- a/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md
+++ b/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md
@@ -95,8 +95,8 @@ trtllm-serve deepseek-ai/DeepSeek-R1 \
     --max_num_tokens 2048 \
     --max_seq_len 2048 \
     --kv_cache_free_gpu_memory_fraction 0.9 \
-    --tp_size 4 \
-    --ep_size 4 \
+    --tp_size 8 \
+    --ep_size 8 \
     --trust_remote_code \
     --extra_llm_api_options ${EXTRA_LLM_API_FILE}
 ```

From 531a7cad6dd357dfb87d4c56b3e3a96babcbcc52 Mon Sep 17 00:00:00 2001
From: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
Date: Mon, 4 Aug 2025 02:29:55 +0000
Subject: [PATCH 03/10] Address comments.

Signed-off-by: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
---
 ...eployment-guide-for-trt-llm-deepseek-r1.md | 28 +++++++++----------
 1 file changed, 14 insertions(+), 14 deletions(-)

diff --git a/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md b/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md
index 09a09eaf4d21..aa897c08d321 100644
--- a/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md
+++ b/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md
@@ -1,12 +1,12 @@
 # Deployment Guide for TensorRT-LLM DeepSeek R1 FP8 and NVFP4
 
-# Introduction
+## Introduction
 
 This deployment guide provides step-by-step instructions for running the DeepSeek R1 model using TensorRT-LLM with FP8 and NVFP4 quantization, optimized for NVIDIA GPUs. It covers the complete setup required; from accessing model weights and preparing the software environment to configuring TensorRT-LLM parameters, launching the server, and validating inference output.
 
 The guide is intended for developers and practitioners seeking high-throughput or low-latency inference using NVIDIA’s accelerated stack—starting with the PyTorch container from NGC, then installing TensorRT-LLM for model serving, FlashInfer for optimized CUDA kernels, and ModelOpt to enable FP8 and NVFP4 quantized execution.
 
-# Prerequisites
+## Prerequisites
 
 GPU: NVIDIA Blackwell or Hopper Architecture  
 OS: Linux  
@@ -14,7 +14,7 @@ Drivers: CUDA Driver 575 or Later
 Docker with NVIDIA Container Toolkit installed  
 Python3 and python3-pip (Optional, for accuracy evaluation only)
 
-# Models
+## Models
 
 * FP8 model: [DeepSeek-R1](https://huggingface.co/deepseek-ai/DeepSeek-R1)  
 * NVFP4 model: [DeepSeek-R1-FP4](https://huggingface.co/nvidia/DeepSeek-R1-FP4)
@@ -22,9 +22,9 @@ Python3 and python3-pip (Optional, for accuracy evaluation only)
 
 Note that NVFP4 is only supported on NVIDIA Blackwell platform.
 
-# Deployment Steps
+## Deployment Steps
 
-## Run Docker Container
+### Run Docker Container
 
 Run the docker container using the TensorRT-LLM NVIDIA NGC image.
 
@@ -48,7 +48,7 @@ Note:
 
 If you want to use latest main branch, you can choose to build from source to install TensorRT-LLM, the steps refer to [https://nvidia.github.io/TensorRT-LLM/latest/installation/build-from-source-linux.html](https://nvidia.github.io/TensorRT-LLM/latest/installation/build-from-source-linux.html) 
 
-## Creating the TRT-LLM Server config
+### Creating the TRT-LLM Server config
 
 We create a YAML configuration file /tmp/config.yml for the TensorRT-LLM Server and populate it with the following recommended performance settings.
 
@@ -82,7 +82,7 @@ moe_config:
 EOF
 ```
 
-## Launch the TRT-LLM Server
+### Launch the TRT-LLM Server
 
 Below is an example command to launch the TRT-LLM server with the DeepSeek-R1 model from within the container. The command is specifically configured for the 1024/1024 Input/Output Sequence Length test. The explanation of each flag is shown in the “Configs and Parameters” section.
 
@@ -103,7 +103,7 @@ trtllm-serve deepseek-ai/DeepSeek-R1 \
 
 After the server is set up, the client can now send prompt requests to the server and receive results.
 
-## Configs and Parameters
+### Configs and Parameters
 
 These options are used directly on the command line when you start the `trtllm-serve` process.
 #### `--tp_size`
@@ -193,9 +193,9 @@ These options provide finer control over performance and are set within a YAML f
 
 See the [https://github.com/nvidia/TensorRT-LLM/blob/main/tensorrt\_llm/llmapi/llm\_args.py\#L1894](https://github.com/nvidia/TensorRT-LLM/blob/main/tensorrt_llm/llmapi/llm_args.py#L1894) TorchLlmArgs class for the full list of options which can be used in the extra\_llm\_api\_options`.`
 
-# Testing API Endpoint
+## Testing API Endpoint
 
-## Basic Test
+### Basic Test
 
 Start a new terminal on the host to test the TensorRT-LLM server you just launched. 
 
@@ -224,7 +224,7 @@ Here is an example response, showing that the TRT-LLM server returns “New York
 {"id":"cmpl-bc1393d529ce485c961d9ffee5b25d72","object":"text_completion","created":1753843963,"model":"deepseek-ai/DeepSeek-R1","choices":[{"index":0,"text":" New York is a state located in the northeastern United States. It is bordered by","token_ids":null,"logprobs":null,"context_logits":null,"finish_reason":"length","stop_reason":null,"disaggregated_params":null}],"usage":{"prompt_tokens":6,"total_tokens":22,"completion_tokens":16},"prompt_token_ids":null}
 ```
 
-## Troubleshooting Tips
+### Troubleshooting Tips
 
 * If you encounter CUDA out-of-memory errors, try reducing max\_batch\_size or max\_seq\_len  
 * Ensure your model checkpoints are compatible with the expected format  
@@ -232,7 +232,7 @@ Here is an example response, showing that the TRT-LLM server returns “New York
 * If the container fails to start, verify that the NVIDIA Container Toolkit is properly installed  
 * For connection issues, make sure port 8000 is not being used by another application
 
-## Running Evaluations to Verify Accuracy (Optional)
+### Running Evaluations to Verify Accuracy (Optional)
 
 We use the lm-eval tool to test the model’s accuracy. For more information see [https://github.com/EleutherAI/lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness).
 
@@ -282,7 +282,7 @@ Sample result in Blackwell
 |     |       |strict-match    |     5|exact_match|↑  |0.8393|±  |0.0101|
 ```
 
-# Benchmarking Performance
+## Benchmarking Performance
 
 To benchmark the performance of your TensorRT-LLM server you can leverage the built-in “benchmark\_serving.py” script. To do this first creating a wrapper [bench.sh](http://bench.sh) script.
 
@@ -363,7 +363,7 @@ P99 E2EL (ms):                            [result]
 ==================================================
 ```
 
-## Key Metrics
+### Key Metrics
 
 * Median Time to First Token (TTFT)  
   * The typical time elapsed from when a request is sent until the first output token is generated.  

From d94211eac2f6e8e3fee4ab2d857a5f666f51eb6d Mon Sep 17 00:00:00 2001
From: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
Date: Mon, 4 Aug 2025 03:31:26 +0000
Subject: [PATCH 04/10] Address comments.

Signed-off-by: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
---
 ...deployment-guide-for-trt-llm-deepseek-r1.md | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md b/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md
index aa897c08d321..b4cfe39b9733 100644
--- a/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md
+++ b/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md
@@ -16,8 +16,8 @@ Python3 and python3-pip (Optional, for accuracy evaluation only)
 
 ## Models
 
-* FP8 model: [DeepSeek-R1](https://huggingface.co/deepseek-ai/DeepSeek-R1)  
-* NVFP4 model: [DeepSeek-R1-FP4](https://huggingface.co/nvidia/DeepSeek-R1-FP4)
+* FP8 model: [DeepSeek-R1-0528](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528)  
+* NVFP4 model: [DeepSeek-R1-0528-FP4](https://huggingface.co/nvidia/DeepSeek-R1-0528-FP4)
 
 
 Note that NVFP4 is only supported on NVIDIA Blackwell platform.
@@ -87,7 +87,7 @@ EOF
 Below is an example command to launch the TRT-LLM server with the DeepSeek-R1 model from within the container. The command is specifically configured for the 1024/1024 Input/Output Sequence Length test. The explanation of each flag is shown in the “Configs and Parameters” section.
 
 ```shell
-trtllm-serve deepseek-ai/DeepSeek-R1 \
+trtllm-serve deepseek-ai/DeepSeek-R1-0528 \
     --host 0.0.0.0 \
     --port 8000 \
     --backend pytorch \
@@ -211,7 +211,7 @@ After the TRT-LLM server is set up and shows Application startup complete, you c
 
 ```shell
 curl http://localhost:8000/v1/completions -H "Content-Type: application/json"  -d '{
-      "model": "deepseek-ai/DeepSeek-R1",
+      "model": "deepseek-ai/DeepSeek-R1-0528",
       "prompt": "Where is New York?",
       "max_tokens": 16,
       "temperature": 0
@@ -221,7 +221,7 @@ curl http://localhost:8000/v1/completions -H "Content-Type: application/json"  -
 Here is an example response, showing that the TRT-LLM server returns “New York is a state located in the northeastern United States. It is bordered by”, completing the input sequence.
 
 ```json
-{"id":"cmpl-bc1393d529ce485c961d9ffee5b25d72","object":"text_completion","created":1753843963,"model":"deepseek-ai/DeepSeek-R1","choices":[{"index":0,"text":" New York is a state located in the northeastern United States. It is bordered by","token_ids":null,"logprobs":null,"context_logits":null,"finish_reason":"length","stop_reason":null,"disaggregated_params":null}],"usage":{"prompt_tokens":6,"total_tokens":22,"completion_tokens":16},"prompt_token_ids":null}
+{"id":"cmpl-bc1393d529ce485c961d9ffee5b25d72","object":"text_completion","created":1753843963,"model":"deepseek-ai/DeepSeek-R1-0528","choices":[{"index":0,"text":" New York is a state located in the northeastern United States. It is bordered by","token_ids":null,"logprobs":null,"context_logits":null,"finish_reason":"length","stop_reason":null,"disaggregated_params":null}],"usage":{"prompt_tokens":6,"total_tokens":22,"completion_tokens":16},"prompt_token_ids":null}
 ```
 
 ### Troubleshooting Tips
@@ -249,7 +249,7 @@ FP8 command for GSM8K
 * Note: The tokenizer will add BOS (beginning of sentence token) before input prompt by default which leads to accuracy regression on GSM8K task for DeepSeek R1 model. So, set add\_special\_tokens=False to avoid it.
 
 ```
-MODEL_PATH=deepseek-ai/DeepSeek-R1
+MODEL_PATH=deepseek-ai/DeepSeek-R1-0528
 
 lm_eval --model local-completions  --tasks gsm8k --batch_size 256 --gen_kwargs temperature=0.0,add_special_tokens=False --num_fewshot 5 --model_args model=${MODEL_PATH},base_url=http://localhost:8000/v1/completions,num_concurrent=32,max_retries=20,tokenized_requests=False --log_samples --output_path trtllm.fp8.gsm8k
 ```
@@ -268,7 +268,7 @@ FP4 command for GSM8K
 * Note: The tokenizer will add BOS before input prompt by default, which leads to accuracy regression on GSM8K task for DeepSeek R1 model. So set add\_special\_tokens=False to avoid it.
 
 ```shell
-MODEL_PATH=nvidia/DeepSeek-R1-FP4
+MODEL_PATH=nvidia/DeepSeek-R1-0528-FP4
 
 lm_eval --model local-completions  --tasks gsm8k --batch_size 256 --gen_kwargs temperature=0.0,add_special_tokens=False --num_fewshot 5 --model_args model=${MODEL_PATH},base_url=http://localhost:8000/v1/completions,num_concurrent=32,max_retries=20,tokenized_requests=False --log_samples --output_path trtllm.fp4.gsm8k
 ```
@@ -297,7 +297,7 @@ result_dir=/tmp/deepseek_r1_output
 for concurrency in ${concurrency_list}; do
     num_prompts=$((concurrency * multi_round))
     python -m tensorrt_llm.serve.scripts.benchmark_serving \
-        --model deepseek-ai/DeepSeek-R1 \
+        --model deepseek-ai/DeepSeek-R1-0528 \
         --backend openai \
         --dataset-name "random" \
         --random-input-len ${isl} \
@@ -314,7 +314,7 @@ EOF
 chmod +x bench.sh
 ```
 
-To benchmark the FP4 model, replace \--model deepseek-ai/DeepSeek-R1 with \--model nvidia/DeepSeek-R1-FP4.
+To benchmark the FP4 model, replace \--model deepseek-ai/DeepSeek-R1-0528 with \--model nvidia/DeepSeek-R1-0528-FP4.
 
 If you want to save the results to a file add the following options.
 

From 37f5d3c133601599290f0d3b22ca3ca3b045de76 Mon Sep 17 00:00:00 2001
From: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
Date: Mon, 4 Aug 2025 04:54:39 +0000
Subject: [PATCH 05/10] Update config.

Signed-off-by: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
---
 .../deployment-guide-for-trt-llm-deepseek-r1.md  | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md b/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md
index b4cfe39b9733..56e5e49291d6 100644
--- a/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md
+++ b/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md
@@ -59,9 +59,13 @@ cat << EOF > ${EXTRA_LLM_API_FILE}
 enable_attention_dp: true
 cuda_graph_config:
   enable_padding: true
-  max_batch_size: 1024
+  max_batch_size: 128
 kv_cache_config:
   dtype: fp8
+stream_interval: 10
+speculative_config:
+  decoding_type: MTP
+  num_nextn_predict_layers: 3
 EOF
 ```
 
@@ -77,6 +81,10 @@ cuda_graph_config:
   max_batch_size: 1024
 kv_cache_config:
   dtype: fp8
+stream_interval: 10
+speculative_config:
+  decoding_type: MTP
+  num_nextn_predict_layers: 3
 moe_config:
   backend: DEEPGEMM
 EOF
@@ -92,9 +100,9 @@ trtllm-serve deepseek-ai/DeepSeek-R1-0528 \
     --port 8000 \
     --backend pytorch \
     --max_batch_size 1024 \
-    --max_num_tokens 2048 \
+    --max_num_tokens 5248 \
     --max_seq_len 2048 \
-    --kv_cache_free_gpu_memory_fraction 0.9 \
+    --kv_cache_free_gpu_memory_fraction 0.8 \
     --tp_size 8 \
     --ep_size 8 \
     --trust_remote_code \
@@ -288,7 +296,7 @@ To benchmark the performance of your TensorRT-LLM server you can leverage the bu
 
 ```shell
 cat <<EOF >  bench.sh
-concurrency_list="1 2 4 8 16 32 64 128 256"
+concurrency_list="32 64 128 256 512 1024 2048 4096"
 multi_round=5
 isl=1024
 osl=1024

From 2ee475abc2dac5b87fb938893ebde1368fd64c5c Mon Sep 17 00:00:00 2001
From: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
Date: Mon, 4 Aug 2025 05:18:46 +0000
Subject: [PATCH 06/10] Update config.

Signed-off-by: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
---
 .../deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md     | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md b/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md
index 56e5e49291d6..a2aebba061de 100644
--- a/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md
+++ b/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md
@@ -78,7 +78,7 @@ cat << EOF > ${EXTRA_LLM_API_FILE}
 enable_attention_dp: true
 cuda_graph_config:
   enable_padding: true
-  max_batch_size: 1024
+  max_batch_size: 128
 kv_cache_config:
   dtype: fp8
 stream_interval: 10

From e6696e16890d6d48ef2deea63069b70f013b20a5 Mon Sep 17 00:00:00 2001
From: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
Date: Mon, 4 Aug 2025 06:02:03 +0000
Subject: [PATCH 07/10] Update results.

Signed-off-by: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
---
 ...eployment-guide-for-trt-llm-deepseek-r1.md | 26 +++++++++----------
 1 file changed, 13 insertions(+), 13 deletions(-)

diff --git a/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md b/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md
index a2aebba061de..ead45fb8357a 100644
--- a/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md
+++ b/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md
@@ -252,7 +252,7 @@ docker exec -it tensorrt_llm /bin/bash
 pip install lm_eval
 ```
 
-FP8 command for GSM8K
+FP8 command for GSM8K:
 
 * Note: The tokenizer will add BOS (beginning of sentence token) before input prompt by default which leads to accuracy regression on GSM8K task for DeepSeek R1 model. So, set add\_special\_tokens=False to avoid it.
 
@@ -262,16 +262,16 @@ MODEL_PATH=deepseek-ai/DeepSeek-R1-0528
 lm_eval --model local-completions  --tasks gsm8k --batch_size 256 --gen_kwargs temperature=0.0,add_special_tokens=False --num_fewshot 5 --model_args model=${MODEL_PATH},base_url=http://localhost:8000/v1/completions,num_concurrent=32,max_retries=20,tokenized_requests=False --log_samples --output_path trtllm.fp8.gsm8k
 ```
 
-Sample result in Blackwell. 
+Sample result in Blackwell:
 
-```
-|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
-|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
-|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.9348|±  |0.0068|
-|     |       |strict-match    |     5|exact_match|↑  |0.8870|±  |0.0087|
+```shell
+|Tasks|Version|     Filter     |n-shot|  Metric   |   | Value |   |Stderr|
+|-----|------:|----------------|-----:|-----------|---|------:|---|-----:|
+|gsm8k|      3|flexible-extract|     5|exact_match|↑  |94.9204|±  |0.6048|
+|     |       |strict-match    |     5|exact_match|↑  |94.8446|±  |0.6091|
 ```
 
-FP4 command for GSM8K
+FP4 command for GSM8K:
 
 * Note: The tokenizer will add BOS before input prompt by default, which leads to accuracy regression on GSM8K task for DeepSeek R1 model. So set add\_special\_tokens=False to avoid it.
 
@@ -281,13 +281,13 @@ MODEL_PATH=nvidia/DeepSeek-R1-0528-FP4
 lm_eval --model local-completions  --tasks gsm8k --batch_size 256 --gen_kwargs temperature=0.0,add_special_tokens=False --num_fewshot 5 --model_args model=${MODEL_PATH},base_url=http://localhost:8000/v1/completions,num_concurrent=32,max_retries=20,tokenized_requests=False --log_samples --output_path trtllm.fp4.gsm8k
 ```
 
-Sample result in Blackwell
+Sample result in Blackwell:
 
 ```shell
-|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
-|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
-|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.9356|±  |0.0068|
-|     |       |strict-match    |     5|exact_match|↑  |0.8393|±  |0.0101|
+|Tasks|Version|     Filter     |n-shot|  Metric   |   | Value |   |Stderr|
+|-----|------:|----------------|-----:|-----------|---|------:|---|-----:|
+|gsm8k|      3|flexible-extract|     5|exact_match|↑  |94.9204|±  |0.6048|
+|     |       |strict-match    |     5|exact_match|↑  |94.8446|±  |0.6091|
 ```
 
 ## Benchmarking Performance

From 9ecd493d65c8cbf46a82f2ceb59aa570a834e8e1 Mon Sep 17 00:00:00 2001
From: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
Date: Mon, 4 Aug 2025 08:23:13 +0000
Subject: [PATCH 08/10] Update config and result.

Signed-off-by: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
---
 .../deployment-guide-for-trt-llm-deepseek-r1.md      | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md b/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md
index ead45fb8357a..476a607551cc 100644
--- a/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md
+++ b/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md
@@ -65,7 +65,7 @@ kv_cache_config:
 stream_interval: 10
 speculative_config:
   decoding_type: MTP
-  num_nextn_predict_layers: 3
+  num_nextn_predict_layers: 1
 EOF
 ```
 
@@ -84,7 +84,7 @@ kv_cache_config:
 stream_interval: 10
 speculative_config:
   decoding_type: MTP
-  num_nextn_predict_layers: 3
+  num_nextn_predict_layers: 1
 moe_config:
   backend: DEEPGEMM
 EOF
@@ -100,7 +100,7 @@ trtllm-serve deepseek-ai/DeepSeek-R1-0528 \
     --port 8000 \
     --backend pytorch \
     --max_batch_size 1024 \
-    --max_num_tokens 5248 \
+    --max_num_tokens 3200 \
     --max_seq_len 2048 \
     --kv_cache_free_gpu_memory_fraction 0.8 \
     --tp_size 8 \
@@ -126,7 +126,7 @@ These options are used directly on the command line when you start the `trtllm-s
 
 &emsp;**Description:** A value between 0.0 and 1.0 that specifies the fraction of free GPU memory to reserve for the KV cache after the model is loaded. Since memory usage can fluctuate, this buffer helps prevent out-of-memory (OOM) errors.
 
-&emsp;**Recommendation:** If you experience OOM errors, try reducing this value to **0.8** or lower.
+&emsp;**Recommendation:** If you experience OOM errors, try reducing this value to **0.7** or lower.
 
 #### `--backend pytorch`
 
@@ -199,7 +199,7 @@ These options provide finer control over performance and are set within a YAML f
 
 &emsp;**Default**: TRTLLM
 
-See the [https://github.com/nvidia/TensorRT-LLM/blob/main/tensorrt\_llm/llmapi/llm\_args.py\#L1894](https://github.com/nvidia/TensorRT-LLM/blob/main/tensorrt_llm/llmapi/llm_args.py#L1894) TorchLlmArgs class for the full list of options which can be used in the extra\_llm\_api\_options`.`
+See the [TorchLlmArgs class](https://nvidia.github.io/TensorRT-LLM/llm-api/reference.html#tensorrt_llm.llmapi.TorchLlmArgs) for the full list of options which can be used in the extra\_llm\_api\_options`.`
 
 ## Testing API Endpoint
 
@@ -229,7 +229,7 @@ curl http://localhost:8000/v1/completions -H "Content-Type: application/json"  -
 Here is an example response, showing that the TRT-LLM server returns “New York is a state located in the northeastern United States. It is bordered by”, completing the input sequence.
 
 ```json
-{"id":"cmpl-bc1393d529ce485c961d9ffee5b25d72","object":"text_completion","created":1753843963,"model":"deepseek-ai/DeepSeek-R1-0528","choices":[{"index":0,"text":" New York is a state located in the northeastern United States. It is bordered by","token_ids":null,"logprobs":null,"context_logits":null,"finish_reason":"length","stop_reason":null,"disaggregated_params":null}],"usage":{"prompt_tokens":6,"total_tokens":22,"completion_tokens":16},"prompt_token_ids":null}
+{"id":"cmpl-e728f08114c042309efeae4df86a50ca","object":"text_completion","created":1754294810,"model":"deepseek-ai/DeepSeek-R1-0528","choices":[{"index":0,"text":" / by Megan Stine ; illustrated by John Hinderliter.\n\nBook | Gross","token_ids":null,"logprobs":null,"context_logits":null,"finish_reason":"length","stop_reason":null,"disaggregated_params":null}],"usage":{"prompt_tokens":6,"total_tokens":22,"completion_tokens":16},"prompt_token_ids":null}
 ```
 
 ### Troubleshooting Tips

From b3f7dfe1acf879d14b05e5c4ab457ee828d7b6e3 Mon Sep 17 00:00:00 2001
From: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
Date: Mon, 4 Aug 2025 12:39:19 +0000
Subject: [PATCH 09/10] Update config and result.

Signed-off-by: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
---
 .../deployment-guide-for-trt-llm-deepseek-r1.md | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)

diff --git a/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md b/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md
index 476a607551cc..9ec4b10fc5fe 100644
--- a/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md
+++ b/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md
@@ -87,6 +87,7 @@ speculative_config:
   num_nextn_predict_layers: 1
 moe_config:
   backend: DEEPGEMM
+  max_num_tokens: 3200
 EOF
 ```
 
@@ -265,10 +266,10 @@ lm_eval --model local-completions  --tasks gsm8k --batch_size 256 --gen_kwargs t
 Sample result in Blackwell:
 
 ```shell
-|Tasks|Version|     Filter     |n-shot|  Metric   |   | Value |   |Stderr|
-|-----|------:|----------------|-----:|-----------|---|------:|---|-----:|
-|gsm8k|      3|flexible-extract|     5|exact_match|↑  |94.9204|±  |0.6048|
-|     |       |strict-match    |     5|exact_match|↑  |94.8446|±  |0.6091|
+|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
+|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
+|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.9538|±  |0.0058|
+|     |       |strict-match    |     5|exact_match|↑  |0.9500|±  |0.0060|
 ```
 
 FP4 command for GSM8K:
@@ -284,10 +285,10 @@ lm_eval --model local-completions  --tasks gsm8k --batch_size 256 --gen_kwargs t
 Sample result in Blackwell:
 
 ```shell
-|Tasks|Version|     Filter     |n-shot|  Metric   |   | Value |   |Stderr|
-|-----|------:|----------------|-----:|-----------|---|------:|---|-----:|
-|gsm8k|      3|flexible-extract|     5|exact_match|↑  |94.9204|±  |0.6048|
-|     |       |strict-match    |     5|exact_match|↑  |94.8446|±  |0.6091|
+|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
+|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
+|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.9462|±  |0.0062|
+|     |       |strict-match    |     5|exact_match|↑  |0.9447|±  |0.0063|
 ```
 
 ## Benchmarking Performance

From efa0cf42e1d7598130b012e1306e4f2f891d6981 Mon Sep 17 00:00:00 2001
From: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
Date: Wed, 6 Aug 2025 08:21:47 +0000
Subject: [PATCH 10/10] Rename doc.

Signed-off-by: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
---
 ...k-r1.md => quick-start-recipe-for-deepseek-r1-on-trt-llm.md} | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
 rename examples/models/core/deepseek_v3/{deployment-guide-for-trt-llm-deepseek-r1.md => quick-start-recipe-for-deepseek-r1-on-trt-llm.md} (99%)

diff --git a/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md b/examples/models/core/deepseek_v3/quick-start-recipe-for-deepseek-r1-on-trt-llm.md
similarity index 99%
rename from examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md
rename to examples/models/core/deepseek_v3/quick-start-recipe-for-deepseek-r1-on-trt-llm.md
index 9ec4b10fc5fe..6c019c8a8621 100644
--- a/examples/models/core/deepseek_v3/deployment-guide-for-trt-llm-deepseek-r1.md
+++ b/examples/models/core/deepseek_v3/quick-start-recipe-for-deepseek-r1-on-trt-llm.md
@@ -1,4 +1,4 @@
-# Deployment Guide for TensorRT-LLM DeepSeek R1 FP8 and NVFP4
+# Quick Start Recipe for DeepSeek R1 on TensorRT-LLM - Blackwell & Hopper Hardware
 
 ## Introduction