Skip to content

[6294905] Fix --quant_cfg CLI parsing type in transformer trainer #1676

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kinjalpatel27 merged 3 commits into from
Jun 11, 2026
+2 −3
Merged

[6294905] Fix --quant_cfg CLI parsing type in transformer trainer #1676

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d09caba
Fix --quant_cfg CLI parsing type in transformer trainer
kinjalpatel27 Jun 10, 2026
2766285
minor
kinjalpatel27 Jun 10, 2026
9d49456
minor
kinjalpatel27 Jun 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion examples/llm_qat/ARGUMENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@
| Argument | Type | Default | Description |
|----------|------|---------|-------------|
| `--recipe` | `str` | `None` | Path to a quantization recipe YAML file (built-in or custom). Built-in recipes can be specified by relative path, e.g. 'general/ptq/nvfp4_default-kv_fp8'. Replaces the deprecated --quant_cfg flag. |
| `--quant_cfg` | `modelopt.torch.quantization.config.QuantizeConfig` | `None` | Deprecated: pre-quantize the model with a separate quantization step instead. Specify the quantization format for PTQ/QAT by name (e.g. NVFP4_DEFAULT_CFG). |
| `--quant_cfg` | `str` | `None` | Deprecated: pre-quantize the model with a separate quantization step instead. Specify the quantization format for PTQ/QAT by name (e.g. NVFP4_DEFAULT_CFG). |

@jenchen13 jenchen13 Jun 11, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if --quant-cfg is being deprecated, will it be removed in the future?

@kinjalpatel27 kinjalpatel27 Jun 11, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would think so, once we adapt to recipes completely, I think quant_cfg should be removed in the future.

@jenchen13 jenchen13 Jun 11, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like the deprecation note is more of a user-workflow opinion - to quantize with a separate script. Not whether to use config or recipe

@kinjalpatel27 kinjalpatel27 Jun 11, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess as we added Replaces the deprecated --quant_cfg flag. in --recipe, we can add something similar in quant_cfg to encourage users to use --recipe instead

| `--calib_size` | `int` | `512` | Specify the calibration size for quantization. The calibration dataset is used to setup the quantization scale parameters for PTQ/QAT. |
| `--compress` | `bool` | `False` | Whether to compress the model weights after quantization for QLoRA. This is useful for reducing the model size. |
| `--calib_batch_size` | `int` | `1` | Batch size for calibration data during quantization. |
Expand Down
3 changes: 1 addition & 2 deletions modelopt/torch/quantization/plugins/transformers_trainer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,6 @@
from modelopt.torch.opt.plugins.transformers import ModelOptHFArguments
from modelopt.torch.utils import get_module_device, print_rank_0

from ..config import QuantizeConfig
from ..nn import TensorQuantizer
from ..utils import (
calibrate_with_adapters,
Expand All @@ -58,7 +57,7 @@ class QuantizationArguments(ModelOptHFArguments):
),
},
)
quant_cfg: str | QuantizeConfig | None = field(
quant_cfg: str | None = field(
Comment thread
coderabbitai[bot] marked this conversation as resolved.
default=None,
metadata={
"help": (
Expand Down
Loading