diff --git a/CHANGELOG.rst b/CHANGELOG.rst index 31fac352b96..2cd57b4a8b6 100755 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -73,6 +73,7 @@ Changelog **Deprecations** - Deprecate the public ``QuantizationArgumentsWithConfig`` name in ``modelopt.torch.quantization.plugins.transformers_trainer``; it now aliases ``QuantizationArguments`` and will be removed in a future release. +- Deprecate the ``examples/diffusers/eval`` image-quality evaluation example (ImageReward / CLIP-IQA / CLIP metrics); it is no longer maintained and will be removed in the next release. **Bug Fixes** diff --git a/examples/diffusers/README.md b/examples/diffusers/README.md index 8fc32d7a324..24061023b30 100644 --- a/examples/diffusers/README.md +++ b/examples/diffusers/README.md @@ -474,6 +474,8 @@ Stable Diffusion pipelines rely heavily on random sampling operations, which inc ## Evaluate Accuracy +> **Deprecated:** This evaluation example is no longer maintained and will be removed in a future release. There is no public replacement at this time. See [`eval/README.md`](./eval/README.md). + This simple code demonstrates how to evaluate images generated by diffusion (or other generative) models using popular metrics such as [imagereward](https://arxiv.org/abs/2304.05977), [clip-iqa](https://arxiv.org/abs/2207.12396), and [clip](https://arxiv.org/abs/2104.08718). ### Install Requirements diff --git a/examples/diffusers/eval/README.md b/examples/diffusers/eval/README.md new file mode 100644 index 00000000000..56a8dd67961 --- /dev/null +++ b/examples/diffusers/eval/README.md @@ -0,0 +1,24 @@ +# Diffusers Image-Quality Evaluation (Deprecated) + +> **Deprecated:** This evaluation example is no longer maintained and will be removed in a future release. There is no public replacement at this time. + +These scripts evaluate images generated by diffusion (or other generative) models using popular metrics such as [ImageReward](https://arxiv.org/abs/2304.05977), [CLIP-IQA](https://arxiv.org/abs/2207.12396), and [CLIP](https://arxiv.org/abs/2104.08718). + +## Install Requirements + +```bash +pip install -r requirements.txt +``` + +## Usage + +Prepare a JSON file with your prompts and corresponding images, then run: + +```bash +python main.py --data-path {PATH_TO_THE_IMAGE_JSON_PATH} --metrics imagereward +``` + +- `--data-path`: Path to your JSON file. +- `--metrics`: One or more metrics to compute (e.g. `imagereward`, `clip-iqa`, `clip`). + +See the [Evaluate Accuracy](../README.md#evaluate-accuracy) section of the diffusers README for the input JSON format and sample results.