Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,10 @@ Changelog
**Deprecations**

- Deprecate the public ``QuantizationArgumentsWithConfig`` name in ``modelopt.torch.quantization.plugins.transformers_trainer``; it now aliases ``QuantizationArguments`` and will be removed in a future release.
- Deprecate ``examples/llm_autodeploy``. The AutoQuant + TensorRT-LLM AutoDeploy
workflow it demonstrates will be removed in a future release; use TensorRT-LLM's
`AutoDeploy <https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/auto_deploy>`_
directly together with ModelOpt PTQ in ``examples/llm_ptq``.

**Bug Fixes**

Expand Down
7 changes: 7 additions & 0 deletions examples/llm_autodeploy/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# Deploy AutoQuant Models with AutoDeploy

> [!WARNING]
> **Deprecated (ModelOpt 0.45).** This example is deprecated and will be removed in
> a future release (0.46). For mixed-precision deployment, use TensorRT-LLM's
> [AutoDeploy](https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/auto_deploy)
> directly, combined with ModelOpt PTQ/AutoQuant from
> [`examples/llm_ptq`](../llm_ptq/README.md).

This guide demonstrates how to deploy mixed-precision models using ModelOpt's AutoQuant and TRT-LLM's AutoDeploy.

[ModelOpt's AutoQuant](https://nvidia.github.io/Model-Optimizer/reference/generated/modelopt.torch.quantization.model_quant.html#modelopt.torch.quantization.model_quant.auto_quantize) is a post-training quantization (PTQ) algorithm that optimizes model quantization by selecting the best quantization format for each layer while adhering to user-defined compression constraints. This approach allows users to balance model accuracy and performance effectively.
Expand Down
Loading