From 01c83eaffd44d5ba27a4046f9ed914a3c6a6afeb Mon Sep 17 00:00:00 2001 From: Jingyu Xin Date: Mon, 22 Jun 2026 17:00:02 -0700 Subject: [PATCH 1/3] Deprecate examples/diffusers/eval image-quality evaluation example Per the project deprecation policy (1-release migration before removal), mark the diffusers eval example (ImageReward / CLIP-IQA / CLIP metrics) as deprecated: add a notice to a new eval/README.md and to the diffusers README Evaluate Accuracy section, plus a 0.45 Changelog Deprecations entry. The example code is kept intact and will be removed in a follow-up after the 0.45 release. Co-Authored-By: Claude Opus 4.8 (1M context) Signed-off-by: Jingyu Xin --- CHANGELOG.rst | 1 + examples/diffusers/README.md | 2 ++ examples/diffusers/eval/README.md | 24 ++++++++++++++++++++++++ 3 files changed, 27 insertions(+) create mode 100644 examples/diffusers/eval/README.md diff --git a/CHANGELOG.rst b/CHANGELOG.rst index 31fac352b96..5892d02d22d 100755 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -73,6 +73,7 @@ Changelog **Deprecations** - Deprecate the public ``QuantizationArgumentsWithConfig`` name in ``modelopt.torch.quantization.plugins.transformers_trainer``; it now aliases ``QuantizationArguments`` and will be removed in a future release. +- Deprecate the ``examples/diffusers/eval`` image-quality evaluation example (ImageReward / CLIP-IQA / CLIP metrics); it is no longer maintained and will be removed in a future release. **Bug Fixes** diff --git a/examples/diffusers/README.md b/examples/diffusers/README.md index 8fc32d7a324..69c286bb0e2 100644 --- a/examples/diffusers/README.md +++ b/examples/diffusers/README.md @@ -474,6 +474,8 @@ Stable Diffusion pipelines rely heavily on random sampling operations, which inc ## Evaluate Accuracy +> **Deprecated:** This evaluation example is no longer maintained and will be removed in a future release. There is no public replacement at this time; evaluation support within the TensorRT-LLM and SGLang frameworks is planned. See [`eval/README.md`](./eval/README.md). + This simple code demonstrates how to evaluate images generated by diffusion (or other generative) models using popular metrics such as [imagereward](https://arxiv.org/abs/2304.05977), [clip-iqa](https://arxiv.org/abs/2207.12396), and [clip](https://arxiv.org/abs/2104.08718). ### Install Requirements diff --git a/examples/diffusers/eval/README.md b/examples/diffusers/eval/README.md new file mode 100644 index 00000000000..595b53efa8f --- /dev/null +++ b/examples/diffusers/eval/README.md @@ -0,0 +1,24 @@ +# Diffusers Image-Quality Evaluation (Deprecated) + +> **Deprecated:** This evaluation example is no longer maintained and will be removed in a future release. There is no public replacement at this time; evaluation support within the TensorRT-LLM and SGLang frameworks is planned. + +These scripts evaluate images generated by diffusion (or other generative) models using popular metrics such as [ImageReward](https://arxiv.org/abs/2304.05977), [CLIP-IQA](https://arxiv.org/abs/2207.12396), and [CLIP](https://arxiv.org/abs/2104.08718). + +## Install Requirements + +```bash +pip install -r requirements.txt +``` + +## Usage + +Prepare a JSON file with your prompts and corresponding images, then run: + +```bash +python main.py --data-path {PATH_TO_THE_IMAGE_JSON_PATH} --metrics imagereward +``` + +- `--data-path`: Path to your JSON file. +- `--metrics`: One or more metrics to compute (e.g. `imagereward`, `clip-iqa`, `clip`). + +See the [Evaluate Accuracy](../README.md#evaluate-accuracy) section of the diffusers README for the input JSON format and sample results. From c429882388f8fa65322ac360e3f73e4bfee5c433 Mon Sep 17 00:00:00 2001 From: Jingyu Xin Date: Mon, 22 Jun 2026 17:10:49 -0700 Subject: [PATCH 2/3] Drop forward-looking eval-support note from deprecation notices Co-Authored-By: Claude Opus 4.8 (1M context) Signed-off-by: Jingyu Xin --- examples/diffusers/README.md | 2 +- examples/diffusers/eval/README.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/examples/diffusers/README.md b/examples/diffusers/README.md index 69c286bb0e2..24061023b30 100644 --- a/examples/diffusers/README.md +++ b/examples/diffusers/README.md @@ -474,7 +474,7 @@ Stable Diffusion pipelines rely heavily on random sampling operations, which inc ## Evaluate Accuracy -> **Deprecated:** This evaluation example is no longer maintained and will be removed in a future release. There is no public replacement at this time; evaluation support within the TensorRT-LLM and SGLang frameworks is planned. See [`eval/README.md`](./eval/README.md). +> **Deprecated:** This evaluation example is no longer maintained and will be removed in a future release. There is no public replacement at this time. See [`eval/README.md`](./eval/README.md). This simple code demonstrates how to evaluate images generated by diffusion (or other generative) models using popular metrics such as [imagereward](https://arxiv.org/abs/2304.05977), [clip-iqa](https://arxiv.org/abs/2207.12396), and [clip](https://arxiv.org/abs/2104.08718). diff --git a/examples/diffusers/eval/README.md b/examples/diffusers/eval/README.md index 595b53efa8f..56a8dd67961 100644 --- a/examples/diffusers/eval/README.md +++ b/examples/diffusers/eval/README.md @@ -1,6 +1,6 @@ # Diffusers Image-Quality Evaluation (Deprecated) -> **Deprecated:** This evaluation example is no longer maintained and will be removed in a future release. There is no public replacement at this time; evaluation support within the TensorRT-LLM and SGLang frameworks is planned. +> **Deprecated:** This evaluation example is no longer maintained and will be removed in a future release. There is no public replacement at this time. These scripts evaluate images generated by diffusion (or other generative) models using popular metrics such as [ImageReward](https://arxiv.org/abs/2304.05977), [CLIP-IQA](https://arxiv.org/abs/2207.12396), and [CLIP](https://arxiv.org/abs/2104.08718). From f9fd9188d15428c890263852f4193ffc821e5159 Mon Sep 17 00:00:00 2001 From: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com> Date: Mon, 22 Jun 2026 20:39:55 -0700 Subject: [PATCH 3/3] minor Signed-off-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com> --- CHANGELOG.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.rst b/CHANGELOG.rst index 5892d02d22d..2cd57b4a8b6 100755 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -73,7 +73,7 @@ Changelog **Deprecations** - Deprecate the public ``QuantizationArgumentsWithConfig`` name in ``modelopt.torch.quantization.plugins.transformers_trainer``; it now aliases ``QuantizationArguments`` and will be removed in a future release. -- Deprecate the ``examples/diffusers/eval`` image-quality evaluation example (ImageReward / CLIP-IQA / CLIP metrics); it is no longer maintained and will be removed in a future release. +- Deprecate the ``examples/diffusers/eval`` image-quality evaluation example (ImageReward / CLIP-IQA / CLIP metrics); it is no longer maintained and will be removed in the next release. **Bug Fixes**