From c6b00d3ff73482233b77ccefb480f994c0ebda2b Mon Sep 17 00:00:00 2001 From: Kinjal Patel Date: Mon, 15 Jun 2026 19:51:17 +0000 Subject: [PATCH 1/2] updated readme to clarify simple_qat_train.py usecase Signed-off-by: Kinjal Patel --- examples/llm_qat/README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/examples/llm_qat/README.md b/examples/llm_qat/README.md index 30268698f3d..4b5d4fcf394 100644 --- a/examples/llm_qat/README.md +++ b/examples/llm_qat/README.md @@ -88,11 +88,13 @@ python export.py --pyt_ckpt_path qwen3-8b-qad-nvfp4 --export_path qwen3-8b-qad-d Exported checkpoints can be deployed on [TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM), [vLLM](https://github.com/vllm-project/vllm), or [SGLang](https://github.com/sgl-project/sglang). See [llm_ptq/README.md](../llm_ptq/README.md#deployment) for deployment instructions. For quick accuracy evaluation without exporting, see [Native Fake-Quantized Evaluation](#native-fake-quantized-evaluation). > [!NOTE] -> To see the full QAT flow in a single script (quantize + train + save), see [simple_qat_train.py](simple_qat_train.py): +> For a minimal end-to-end demo (quantize + train + save in one script), see [simple_qat_train.py](simple_qat_train.py). It runs on a **single GPU** only and is intended as a quick introduction to the QAT flow (without transformer trainer)—not for distributed training. > > ```sh > python simple_qat_train.py --model-path meta-llama/Llama-3.2-3B --recipe general/ptq/nvfp4_default-kv_fp8 > ``` +> +> For multi-GPU training (FSDP2, DDP, DeepSpeed), use [train.py](train.py) with `accelerate launch` as shown in the [Run End-to-End QAT/QAD Example](#Run-End-to-End-QAT/QAD-Example) commands above. > [!TIP] > For more performant QAD, please refer to [examples/megatron_bridge/README.md](../megatron_bridge/README.md) for example scripts for PTQ / QAD with Megatron-Bridge which is generally more performant than the Hugging Face scripts. From 0b7e7a15844677ab1ceac25c5dd6c11c43457ff9 Mon Sep 17 00:00:00 2001 From: Kinjal Patel Date: Mon, 15 Jun 2026 19:57:27 +0000 Subject: [PATCH 2/2] minor Signed-off-by: Kinjal Patel --- examples/llm_qat/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/examples/llm_qat/README.md b/examples/llm_qat/README.md index 4b5d4fcf394..7e46753e1e8 100644 --- a/examples/llm_qat/README.md +++ b/examples/llm_qat/README.md @@ -94,7 +94,7 @@ Exported checkpoints can be deployed on [TensorRT-LLM](https://github.com/NVIDIA > python simple_qat_train.py --model-path meta-llama/Llama-3.2-3B --recipe general/ptq/nvfp4_default-kv_fp8 > ``` > -> For multi-GPU training (FSDP2, DDP, DeepSpeed), use [train.py](train.py) with `accelerate launch` as shown in the [Run End-to-End QAT/QAD Example](#Run-End-to-End-QAT/QAD-Example) commands above. +> For multi-GPU training (FSDP2, DDP, DeepSpeed), use [train.py](train.py) with `accelerate launch` as shown in the [commands](#qat) above. > [!TIP] > For more performant QAD, please refer to [examples/megatron_bridge/README.md](../megatron_bridge/README.md) for example scripts for PTQ / QAD with Megatron-Bridge which is generally more performant than the Hugging Face scripts.