From 04ca80543f1ec30eb1cd02a15b10e17e081aaad6 Mon Sep 17 00:00:00 2001 From: stevhliu Date: Tue, 28 Jul 2026 18:04:14 -0700 Subject: [PATCH 1/3] docs --- .ai/AGENTS.md | 2 ++ docs/source/en/conceptual/contribution.md | 15 +++++---------- 2 files changed, 7 insertions(+), 10 deletions(-) diff --git a/.ai/AGENTS.md b/.ai/AGENTS.md index 18c9d0215fd0..ae611feb4bb6 100644 --- a/.ai/AGENTS.md +++ b/.ai/AGENTS.md @@ -37,6 +37,8 @@ Strive to write code as simple and explicit as possible. Task-specific guides live in `.ai/skills/` and are loaded on demand by AI agents. Available skills include: - [model-integration](./skills/model-integration/SKILL.md) (adding/converting pipelines) +- [custom-blocks](./skills/custom-blocks/SKILL.md) (packaging a `ModularPipelineBlocks` subclass for the Hub) +- [diffusers-cli](./skills/diffusers-cli/SKILL.md) (running pipelines, inspecting schemas, and using the Diffusers CLI) - [self-review](./skills/self-review/SKILL.md) (pre-PR self-review against the project rules) ## Self-review before a PR diff --git a/docs/source/en/conceptual/contribution.md b/docs/source/en/conceptual/contribution.md index 7f7d8bd77f29..f9d377907e76 100644 --- a/docs/source/en/conceptual/contribution.md +++ b/docs/source/en/conceptual/contribution.md @@ -589,13 +589,8 @@ The repository keeps AI-agent configuration in [`.ai/`](https://github.com/huggi ### AI-assisted and agentic contributions -AI-assisted contributions are welcome, but they must be coordinated, scoped, and verified to keep review load manageable. PRs that do not follow these guidelines may be closed without detailed review. - -- **Coordinate before opening a PR.** Find or open an issue, review similar PRs (open and recently closed), and wait for an explicit acknowledgment from a maintainer on that issue before opening a PR. This gives us a chance to discuss scope, avoid duplicate work, and confirm the approach. -- **Fix patterns, not one-offs.** If you spot an recurring issue, search the codebase for similar instances and open a *single* issue with a clear, systematic scope (e.g. "fix mutable defaults across all schedulers") rather than many issues or PRs for individual instances. -- **Self-review before opening.** Run the [`self-review`](https://github.com/huggingface/diffusers/blob/main/.ai/skills/self-review/SKILL.md) skill — it reviews your diff against [`.ai/review-rules.md`](https://github.com/huggingface/diffusers/blob/main/.ai/review-rules.md), the same rubric the `@claude` CI reviewer uses — and address what it reports — it's a helper, not authoritative, and can be wrong. Focus on the blocking issues that make sense to you, and clean up dead/unused code as much as possible. If you disagree with a suggestion, it's fine to leave it for the reviewer to discuss after the PR is opened — the notes you share (see below) tell the reviewer it was a deliberate call. -- **Share your self-review notes.** Please post the final self-review report — the round that reflects the diff you're submitting — on the PR, in the description or as a comment, including findings you intentionally did not fix and why. It helps the reviewer see what has already been checked and which calls were deliberate, and usually saves a few rounds of back-and-forth. -- **Include in the PR description:** - - A **coordination link** to the issue or discussion where a maintainer acknowledged the work. - - The **test commands you ran** and their results (paste relevant output, not just "tests pass"). - - Your **self-review notes** (or a link to the PR comment containing them), as described above. +AI agents are welcome tools for contributing to Diffusers. We encourage you to set up your agent with the Diffusers [agent guide](https://github.com/huggingface/diffusers/blob/main/.ai/AGENTS.md) and use the relevant task-specific skills, such as `model-integration` and `self-review`. Run `make codex` or `make claude` to use the skills with your agent, then follow the guide to scope, implement, test, and review your contribution. You remain responsible for understanding, testing, and maintaining the changes in your PR. + +If you are a model author or part of a team that officially maintains a model, we encourage you to use agents for a new model integration. Follow the repository's [recommended setup](https://github.com/huggingface/diffusers/blob/main/.ai/AGENTS.md) and use the [`model-integration`](https://github.com/huggingface/diffusers/blob/main/.ai/skills/model-integration/SKILL.md) skill. Coordinate the scope with maintainers before opening a PR. + +If you are contributing a model to Diffusers for the first time as a community contributor, we generally recommend starting with a custom implementation that loads code from the Hub. This gives users access to the model while its integration into the core library is evaluated. See the [custom models](../using-diffusers/automodel) and [custom modular blocks](../modular_diffusers/custom_blocks) guides for supported patterns. From 5913d231704180fdff4c806cef61f08772a8bcf2 Mon Sep 17 00:00:00 2001 From: stevhliu Date: Tue, 28 Jul 2026 18:14:04 -0700 Subject: [PATCH 2/3] update --- README.md | 2 +- benchmarks/README.md | 2 +- docs/source/en/advanced_inference/outpaint.md | 10 +- .../en/api/models/allegro_transformer3d.md | 2 +- docs/source/en/api/models/autoencoder_dc.md | 2 +- .../models/autoencoder_kl_hunyuan_video.md | 2 +- .../models/autoencoder_kl_hunyuan_video15.md | 2 +- .../api/models/autoencoder_kl_hunyuanimage.md | 2 +- .../autoencoder_kl_hunyuanimage_refiner.md | 2 +- .../en/api/models/autoencoder_kl_kvae.md | 2 +- .../api/models/autoencoder_kl_kvae_video.md | 2 +- .../en/api/models/autoencoder_kl_wan.md | 2 +- docs/source/en/api/models/autoencoder_tiny.md | 8 +- .../en/api/models/autoencoderkl_allegro.md | 2 +- .../api/models/autoencoderkl_audio_ltx_2.md | 2 +- .../en/api/models/autoencoderkl_cogvideox.md | 2 +- .../en/api/models/autoencoderkl_ltx_2.md | 2 +- .../en/api/models/autoencoderkl_ltx_video.md | 2 +- .../en/api/models/autoencoderkl_magvit.md | 2 +- .../en/api/models/autoencoderkl_mochi.md | 2 +- .../api/models/chronoedit_transformer_3d.md | 2 +- .../en/api/models/cogvideox_transformer3d.md | 2 +- .../api/models/cogview3plus_transformer2d.md | 2 +- .../en/api/models/cogview4_transformer2d.md | 2 +- .../en/api/models/consisid_transformer3d.md | 2 +- docs/source/en/api/models/controlnet.md | 2 +- .../en/api/models/controlnet_hunyuandit.md | 2 +- .../en/api/models/controlnet_sparsectrl.md | 4 +- .../en/api/models/cosmos3_omni_transformer.md | 2 +- .../en/api/models/cosmos_transformer3d.md | 2 +- .../api/models/easyanimate_transformer3d.md | 2 +- .../en/api/models/helios_transformer3d.md | 6 +- .../api/models/hidream_image_transformer.md | 4 +- .../models/hunyuan_video15_transformer_3d.md | 2 +- .../models/hunyuan_video_transformer_3d.md | 2 +- .../api/models/hunyuanimage_transformer_2d.md | 2 +- .../api/models/longcat_image_transformer2d.md | 2 +- .../en/api/models/ltx2_video_transformer3d.md | 2 +- .../en/api/models/ltx_video_transformer3d.md | 2 +- .../en/api/models/lumina2_transformer2d.md | 2 +- .../en/api/models/mochi_transformer3d.md | 2 +- .../api/models/motif_video_transformer_3d.md | 2 +- .../en/api/models/omnigen_transformer.md | 2 +- .../en/api/models/ovisimage_transformer2d.md | 2 +- .../en/api/models/qwenimage_transformer2d.md | 2 +- .../en/api/models/sana_transformer2d.md | 2 +- .../en/api/models/sana_video_transformer3d.md | 2 +- .../api/models/skyreels_v2_transformer_3d.md | 2 +- .../en/api/models/transformer_joyimage.md | 2 +- .../models/transformer_joyimage_edit_plus.md | 2 +- .../api/models/wan_animate_transformer_3d.md | 2 +- .../en/api/models/wan_transformer_3d.md | 2 +- docs/source/en/api/pipelines/ace_step.md | 2 +- docs/source/en/api/pipelines/allegro.md | 6 +- docs/source/en/api/pipelines/anima.md | 2 +- docs/source/en/api/pipelines/animatediff.md | 54 ++--- docs/source/en/api/pipelines/anyflow.md | 8 +- docs/source/en/api/pipelines/aura_flow.md | 10 +- docs/source/en/api/pipelines/chroma.md | 6 +- docs/source/en/api/pipelines/chronoedit.md | 24 +-- docs/source/en/api/pipelines/cogvideox.md | 8 +- docs/source/en/api/pipelines/consisid.md | 2 +- .../en/api/pipelines/consistency_models.md | 2 +- .../en/api/pipelines/control_flux_inpaint.md | 6 +- docs/source/en/api/pipelines/cosmos.md | 2 +- docs/source/en/api/pipelines/cosmos3.md | 58 +++--- docs/source/en/api/pipelines/deepfloyd_if.md | 32 +-- docs/source/en/api/pipelines/dreamlite.md | 8 +- docs/source/en/api/pipelines/easyanimate.md | 4 +- docs/source/en/api/pipelines/ernie_image.md | 4 +- docs/source/en/api/pipelines/flux.md | 44 ++-- docs/source/en/api/pipelines/framepack.md | 18 +- docs/source/en/api/pipelines/glm_image.md | 4 +- docs/source/en/api/pipelines/helios.md | 20 +- docs/source/en/api/pipelines/hunyuan_video.md | 6 +- .../en/api/pipelines/hunyuan_video15.md | 2 +- docs/source/en/api/pipelines/hunyuandit.md | 2 +- .../source/en/api/pipelines/hunyuanimage21.md | 6 +- docs/source/en/api/pipelines/ideogram4.md | 8 +- docs/source/en/api/pipelines/joyimage_edit.md | 2 +- .../en/api/pipelines/joyimage_edit_plus.md | 2 +- docs/source/en/api/pipelines/kandinsky.md | 62 +++--- .../en/api/pipelines/kandinsky5_video.md | 10 +- docs/source/en/api/pipelines/kolors.md | 6 +- docs/source/en/api/pipelines/krea2.md | 8 +- .../pipelines/latent_consistency_models.md | 38 ++-- docs/source/en/api/pipelines/latte.md | 8 +- .../en/api/pipelines/longcat_audio_dit.md | 2 +- docs/source/en/api/pipelines/longcat_image.md | 2 +- docs/source/en/api/pipelines/ltx2.md | 18 +- docs/source/en/api/pipelines/ltx_video.md | 24 +-- docs/source/en/api/pipelines/lumina.md | 8 +- docs/source/en/api/pipelines/lumina2.md | 8 +- docs/source/en/api/pipelines/marigold.md | 30 +-- docs/source/en/api/pipelines/mochi.md | 10 +- docs/source/en/api/pipelines/motif_video.md | 6 +- docs/source/en/api/pipelines/omnigen.md | 14 +- docs/source/en/api/pipelines/pag.md | 26 +-- docs/source/en/api/pipelines/pixart.md | 2 +- docs/source/en/api/pipelines/pixart_sigma.md | 2 +- docs/source/en/api/pipelines/prx.md | 10 +- docs/source/en/api/pipelines/prx_pixel.md | 4 +- docs/source/en/api/pipelines/qwenimage.md | 6 +- docs/source/en/api/pipelines/sana.md | 8 +- docs/source/en/api/pipelines/sana_sprint.md | 8 +- docs/source/en/api/pipelines/sana_video.md | 10 +- docs/source/en/api/pipelines/shap_e.md | 10 +- docs/source/en/api/pipelines/skyreels_v2.md | 12 +- docs/source/en/api/pipelines/stable_audio.md | 6 +- .../source/en/api/pipelines/stable_cascade.md | 12 +- .../pipelines/stable_diffusion/sdxl_turbo.md | 6 +- .../stable_diffusion/stable_diffusion_2.md | 8 +- .../stable_diffusion/stable_diffusion_3.md | 34 +-- .../stable_diffusion/stable_diffusion_xl.md | 30 +-- .../en/api/pipelines/stable_diffusion/svd.md | 4 +- docs/source/en/api/pipelines/stable_unclip.md | 8 +- docs/source/en/api/pipelines/visualcloze.md | 8 +- docs/source/en/api/pipelines/wan.md | 42 ++-- docs/source/en/api/pipelines/z_image.md | 4 +- docs/source/en/api/schedulers/ddim.md | 2 +- docs/source/en/conceptual/evaluation.md | 8 +- docs/source/en/hybrid_inference/overview.md | 10 +- .../modular_diffusers/components_manager.md | 4 +- .../en/modular_diffusers/custom_blocks.md | 4 +- .../en/modular_diffusers/modular_pipeline.md | 26 +-- .../source/en/modular_diffusers/quickstart.md | 6 +- .../en/optimization/attention_backends.md | 4 +- docs/source/en/optimization/cache.md | 10 +- docs/source/en/optimization/cache_dit.md | 2 +- docs/source/en/optimization/deepcache.md | 2 +- docs/source/en/optimization/fp16.md | 16 +- docs/source/en/optimization/habana.md | 2 +- docs/source/en/optimization/memory.md | 30 +-- docs/source/en/optimization/mps.md | 2 +- docs/source/en/optimization/para_attn.md | 18 +- docs/source/en/optimization/pruna.md | 4 +- .../en/optimization/speed-memory-optims.md | 8 +- docs/source/en/optimization/tgate.md | 10 +- docs/source/en/optimization/tome.md | 2 +- docs/source/en/optimization/xdit.md | 2 +- docs/source/en/quantization/autoround.md | 10 +- docs/source/en/quantization/bitsandbytes.md | 42 ++-- docs/source/en/quantization/gguf.md | 8 +- docs/source/en/quantization/modelopt.md | 10 +- docs/source/en/quantization/nunchaku.md | 2 +- docs/source/en/quantization/overview.md | 6 +- docs/source/en/quantization/quanto.md | 14 +- docs/source/en/quantization/torchao.md | 12 +- docs/source/en/quicktour.md | 14 +- docs/source/en/stable_diffusion.md | 10 +- docs/source/en/training/cogvideox.md | 2 +- docs/source/en/training/controlnet.md | 4 +- docs/source/en/training/custom_diffusion.md | 4 +- .../en/training/distributed_inference.md | 18 +- docs/source/en/training/dreambooth.md | 2 +- docs/source/en/training/instructpix2pix.md | 2 +- docs/source/en/training/kandinsky.md | 16 +- docs/source/en/training/lcm_distill.md | 4 +- docs/source/en/training/lora.md | 2 +- docs/source/en/training/sdxl.md | 2 +- docs/source/en/training/t2i_adapters.md | 4 +- docs/source/en/training/text2image.md | 2 +- docs/source/en/training/text_inversion.md | 2 +- docs/source/en/training/wuerstchen.md | 2 +- docs/source/en/tutorials/autopipeline.md | 6 +- .../en/tutorials/using_peft_for_inference.md | 36 ++-- docs/source/en/using-diffusers/automodel.md | 8 +- .../en/using-diffusers/batched_inference.md | 6 +- docs/source/en/using-diffusers/callback.md | 4 +- .../conditional_image_generation.md | 24 +-- docs/source/en/using-diffusers/controlnet.md | 26 +-- .../custom_pipeline_overview.md | 12 +- docs/source/en/using-diffusers/depth2img.md | 2 +- docs/source/en/using-diffusers/dreambooth.md | 2 +- .../en/using-diffusers/image_quality.md | 8 +- docs/source/en/using-diffusers/img2img.md | 38 ++-- docs/source/en/using-diffusers/inpaint.md | 44 ++-- docs/source/en/using-diffusers/ip_adapter.md | 40 ++-- docs/source/en/using-diffusers/loading.md | 24 +-- .../en/using-diffusers/other-formats.md | 14 +- docs/source/en/using-diffusers/schedulers.md | 12 +- docs/source/en/using-diffusers/t2i_adapter.md | 14 +- .../source/en/using-diffusers/text-img2vid.md | 36 ++-- .../textual_inversion_inference.md | 4 +- .../en/using-diffusers/weighted_prompts.md | 2 +- docs/source/ja/stable_diffusion.md | 4 +- docs/source/ja/tutorials/autopipeline.md | 12 +- .../stable_diffusion/stable_diffusion_xl.md | 24 +-- docs/source/ko/conceptual/evaluation.md | 8 +- docs/source/ko/optimization/fp16.md | 20 +- docs/source/ko/optimization/tome.md | 2 +- docs/source/ko/optimization/torch2.0.md | 22 +- docs/source/ko/stable_diffusion.md | 4 +- docs/source/ko/training/controlnet.md | 4 +- docs/source/ko/training/custom_diffusion.md | 6 +- .../ko/training/distributed_inference.md | 4 +- docs/source/ko/training/dreambooth.md | 2 +- docs/source/ko/training/instructpix2pix.md | 2 +- docs/source/ko/training/lora.md | 2 +- docs/source/ko/training/text2image.md | 2 +- docs/source/ko/training/text_inversion.md | 2 +- docs/source/ko/using-diffusers/depth2img.md | 2 +- docs/source/ko/using-diffusers/diffedit.md | 8 +- docs/source/ko/using-diffusers/img2img.md | 2 +- docs/source/ko/using-diffusers/inpaint.md | 2 +- docs/source/ko/using-diffusers/kandinsky.md | 62 +++--- docs/source/ko/using-diffusers/loading.md | 10 +- .../ko/using-diffusers/loading_adapters.md | 24 +-- .../ko/using-diffusers/other-formats.md | 2 +- docs/source/ko/using-diffusers/schedulers.md | 2 +- docs/source/ko/using-diffusers/sdxl_turbo.md | 6 +- docs/source/ko/using-diffusers/shap-e.md | 10 +- docs/source/ko/using-diffusers/svd.md | 4 +- .../textual_inversion_inference.md | 2 +- docs/source/pt/stable_diffusion.md | 10 +- docs/source/zh/conceptual/evaluation.md | 8 +- docs/source/zh/hybrid_inference/vae_encode.md | 2 +- .../zh/modular_diffusers/modular_pipeline.md | 16 +- .../source/zh/modular_diffusers/quickstart.md | 10 +- docs/source/zh/optimization/cache.md | 4 +- docs/source/zh/optimization/deepcache.md | 2 +- docs/source/zh/optimization/fp16.md | 16 +- docs/source/zh/optimization/habana.md | 2 +- docs/source/zh/optimization/memory.md | 34 +-- docs/source/zh/optimization/mps.md | 2 +- docs/source/zh/optimization/para_attn.md | 18 +- docs/source/zh/optimization/pruna.md | 4 +- .../zh/optimization/speed-memory-optims.md | 8 +- docs/source/zh/optimization/tgate.md | 10 +- docs/source/zh/optimization/tome.md | 2 +- docs/source/zh/optimization/xdit.md | 2 +- docs/source/zh/stable_diffusion.md | 4 +- docs/source/zh/training/controlnet.md | 4 +- .../zh/training/distributed_inference.md | 12 +- docs/source/zh/training/dreambooth.md | 2 +- docs/source/zh/training/instructpix2pix.md | 2 +- docs/source/zh/training/kandinsky.md | 16 +- docs/source/zh/training/lora.md | 2 +- docs/source/zh/training/text2image.md | 2 +- docs/source/zh/training/text_inversion.md | 2 +- docs/source/zh/training/wuerstchen.md | 2 +- docs/source/zh/tutorials/autopipeline.md | 6 +- .../zh/tutorials/using_peft_for_inference.md | 36 ++-- docs/source/zh/using-diffusers/anyflow.md | 12 +- docs/source/zh/using-diffusers/consisid.md | 2 +- docs/source/zh/using-diffusers/helios.md | 6 +- docs/source/zh/using-diffusers/schedulers.md | 12 +- .../advanced_diffusion_training/README.md | 10 +- .../README_flux.md | 4 +- examples/cogvideo/README.md | 2 +- examples/cogview4-control/README.md | 4 +- examples/community/README.md | 193 +++++++++--------- .../community/README_community_scripts.md | 10 +- examples/controlnet/README.md | 4 +- examples/controlnet/README_flux.md | 4 +- examples/controlnet/README_sd3.md | 2 +- examples/controlnet/README_sdxl.md | 10 +- examples/cosmos3/README.md | 2 +- examples/custom_diffusion/README.md | 6 +- examples/dreambooth/README.md | 6 +- examples/dreambooth/README_ideogram4.md | 2 +- examples/dreambooth/README_krea2.md | 2 +- examples/dreambooth/README_sdxl.md | 6 +- examples/dreambooth/README_z_image.md | 2 +- examples/flux-control/README.md | 4 +- examples/instruct_pix2pix/README.md | 2 +- examples/instruct_pix2pix/README_sdxl.md | 2 +- examples/kandinsky2_2/text_to_image/README.md | 12 +- examples/model_search/README.md | 4 +- examples/profiling/README.md | 2 +- examples/research_projects/anytext/README.md | 4 +- .../research_projects/colossalai/README.md | 2 +- .../flux_lora_quantization/README.md | 6 +- examples/research_projects/lora/README.md | 2 +- .../multi_subject_dreambooth/README.md | 2 +- .../multi_token_textual_inversion/README.md | 2 +- .../promptdiffusion/README.md | 4 +- .../training/text_to_image/README.md | 2 +- examples/t2i_adapter/README_sdxl.md | 4 +- examples/text_to_image/README.md | 10 +- examples/text_to_image/README_sdxl.md | 4 +- examples/textual_inversion/README.md | 2 +- examples/textual_inversion/README_sdxl.md | 2 +- 283 files changed, 1294 insertions(+), 1295 deletions(-) diff --git a/README.md b/README.md index 7c641f780c7b..f9f494e393c6 100644 --- a/README.md +++ b/README.md @@ -65,7 +65,7 @@ Generating outputs is super easy with 🤗 Diffusers. To generate an image from from diffusers import DiffusionPipeline import torch -pipeline = DiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16) +pipeline = DiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16) pipeline.to("cuda") pipeline("An image of a squirrel in Picasso style").images[0] ``` diff --git a/benchmarks/README.md b/benchmarks/README.md index afab1b0de354..636fa9ab0270 100644 --- a/benchmarks/README.md +++ b/benchmarks/README.md @@ -55,7 +55,7 @@ BenchmarkScenario( model_cls=FluxTransformer2DModel, model_init_kwargs={ "pretrained_model_name_or_path": CKPT_ID, - "torch_dtype": torch.bfloat16, + "dtype": torch.bfloat16, "subfolder": "transformer", "quantization_config": BitsAndBytesConfig(load_in_8bit=True), }, diff --git a/docs/source/en/advanced_inference/outpaint.md b/docs/source/en/advanced_inference/outpaint.md index bd0680b0fbdb..0613543f2aa2 100644 --- a/docs/source/en/advanced_inference/outpaint.md +++ b/docs/source/en/advanced_inference/outpaint.md @@ -111,15 +111,15 @@ Load the inpainting ControlNet, ZoeDepth model, VAE and pass them to the [`Stabl ```py controlnets = [ ControlNetModel.from_pretrained( - "destitech/controlnet-inpaint-dreamer-sdxl", torch_dtype=torch.float16, variant="fp16" + "destitech/controlnet-inpaint-dreamer-sdxl", dtype=torch.float16, variant="fp16" ), ControlNetModel.from_pretrained( - "diffusers/controlnet-zoe-depth-sdxl-1.0", torch_dtype=torch.float16 + "diffusers/controlnet-zoe-depth-sdxl-1.0", dtype=torch.float16 ), ] -vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16).to("cuda") +vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", dtype=torch.float16).to("cuda") pipeline = StableDiffusionXLControlNetPipeline.from_pretrained( - "SG161222/RealVisXL_V4.0", torch_dtype=torch.float16, variant="fp16", controlnet=controlnets, vae=vae + "SG161222/RealVisXL_V4.0", dtype=torch.float16, variant="fp16", controlnet=controlnets, vae=vae ).to("cuda") def generate_image(prompt, negative_prompt, inpaint_image, zoe_image, seed: int = None): @@ -173,7 +173,7 @@ Now that you have an initial outpainted image, load the [`StableDiffusionXLInpai ```py pipeline = StableDiffusionXLInpaintPipeline.from_pretrained( "OzzyGT/RealVisXL_V4.0_inpainting", - torch_dtype=torch.float16, + dtype=torch.float16, variant="fp16", vae=vae, ).to("cuda") diff --git a/docs/source/en/api/models/allegro_transformer3d.md b/docs/source/en/api/models/allegro_transformer3d.md index e23d75a4d304..48dd886b24eb 100644 --- a/docs/source/en/api/models/allegro_transformer3d.md +++ b/docs/source/en/api/models/allegro_transformer3d.md @@ -18,7 +18,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import AllegroTransformer3DModel -transformer = AllegroTransformer3DModel.from_pretrained("rhymes-ai/Allegro", subfolder="transformer", torch_dtype=torch.bfloat16).to("cuda") +transformer = AllegroTransformer3DModel.from_pretrained("rhymes-ai/Allegro", subfolder="transformer", dtype=torch.bfloat16).to("cuda") ``` ## AllegroTransformer3DModel diff --git a/docs/source/en/api/models/autoencoder_dc.md b/docs/source/en/api/models/autoencoder_dc.md index fd53ec0ef66f..aff90cf7b596 100644 --- a/docs/source/en/api/models/autoencoder_dc.md +++ b/docs/source/en/api/models/autoencoder_dc.md @@ -36,7 +36,7 @@ Load a model in Diffusers format with [`~ModelMixin.from_pretrained`]. ```python from diffusers import AutoencoderDC -ae = AutoencoderDC.from_pretrained("mit-han-lab/dc-ae-f32c32-sana-1.0-diffusers", torch_dtype=torch.float32).to("cuda") +ae = AutoencoderDC.from_pretrained("mit-han-lab/dc-ae-f32c32-sana-1.0-diffusers", dtype=torch.float32).to("cuda") ``` ## Load a model in Diffusers via `from_single_file` diff --git a/docs/source/en/api/models/autoencoder_kl_hunyuan_video.md b/docs/source/en/api/models/autoencoder_kl_hunyuan_video.md index b173f9f51e36..48a24c15f1d4 100644 --- a/docs/source/en/api/models/autoencoder_kl_hunyuan_video.md +++ b/docs/source/en/api/models/autoencoder_kl_hunyuan_video.md @@ -18,7 +18,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import AutoencoderKLHunyuanVideo -vae = AutoencoderKLHunyuanVideo.from_pretrained("hunyuanvideo-community/HunyuanVideo", subfolder="vae", torch_dtype=torch.float16) +vae = AutoencoderKLHunyuanVideo.from_pretrained("hunyuanvideo-community/HunyuanVideo", subfolder="vae", dtype=torch.float16) ``` ## AutoencoderKLHunyuanVideo diff --git a/docs/source/en/api/models/autoencoder_kl_hunyuan_video15.md b/docs/source/en/api/models/autoencoder_kl_hunyuan_video15.md index e82fe31380a5..445aa3eaa7a8 100644 --- a/docs/source/en/api/models/autoencoder_kl_hunyuan_video15.md +++ b/docs/source/en/api/models/autoencoder_kl_hunyuan_video15.md @@ -18,7 +18,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import AutoencoderKLHunyuanVideo15 -vae = AutoencoderKLHunyuanVideo15.from_pretrained("hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-480p_t2v", subfolder="vae", torch_dtype=torch.float32) +vae = AutoencoderKLHunyuanVideo15.from_pretrained("hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-480p_t2v", subfolder="vae", dtype=torch.float32) # make sure to enable tiling to avoid OOM vae.enable_tiling() diff --git a/docs/source/en/api/models/autoencoder_kl_hunyuanimage.md b/docs/source/en/api/models/autoencoder_kl_hunyuanimage.md index 60dd2b3ab155..756a63815f3d 100644 --- a/docs/source/en/api/models/autoencoder_kl_hunyuanimage.md +++ b/docs/source/en/api/models/autoencoder_kl_hunyuanimage.md @@ -18,7 +18,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import AutoencoderKLHunyuanImage -vae = AutoencoderKLHunyuanImage.from_pretrained("hunyuanvideo-community/HunyuanImage-2.1-Diffusers", subfolder="vae", torch_dtype=torch.bfloat16) +vae = AutoencoderKLHunyuanImage.from_pretrained("hunyuanvideo-community/HunyuanImage-2.1-Diffusers", subfolder="vae", dtype=torch.bfloat16) ``` ## AutoencoderKLHunyuanImage diff --git a/docs/source/en/api/models/autoencoder_kl_hunyuanimage_refiner.md b/docs/source/en/api/models/autoencoder_kl_hunyuanimage_refiner.md index 5e1dd5e2a24a..8b6181f668e6 100644 --- a/docs/source/en/api/models/autoencoder_kl_hunyuanimage_refiner.md +++ b/docs/source/en/api/models/autoencoder_kl_hunyuanimage_refiner.md @@ -18,7 +18,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import AutoencoderKLHunyuanImageRefiner -vae = AutoencoderKLHunyuanImageRefiner.from_pretrained("hunyuanvideo-community/HunyuanImage-2.1-Refiner-Diffusers", subfolder="vae", torch_dtype=torch.bfloat16) +vae = AutoencoderKLHunyuanImageRefiner.from_pretrained("hunyuanvideo-community/HunyuanImage-2.1-Refiner-Diffusers", subfolder="vae", dtype=torch.bfloat16) ``` ## AutoencoderKLHunyuanImageRefiner diff --git a/docs/source/en/api/models/autoencoder_kl_kvae.md b/docs/source/en/api/models/autoencoder_kl_kvae.md index 39cbb4c85c5f..a6c509bd5f98 100644 --- a/docs/source/en/api/models/autoencoder_kl_kvae.md +++ b/docs/source/en/api/models/autoencoder_kl_kvae.md @@ -22,7 +22,7 @@ The model can be loaded with the following code snippet. import torch from diffusers import AutoencoderKLKVAE -vae = AutoencoderKLKVAE.from_pretrained("kandinskylab/KVAE-2D-1.0", subfolder="diffusers", torch_dtype=torch.bfloat16) +vae = AutoencoderKLKVAE.from_pretrained("kandinskylab/KVAE-2D-1.0", subfolder="diffusers", dtype=torch.bfloat16) ``` ## AutoencoderKLKVAE diff --git a/docs/source/en/api/models/autoencoder_kl_kvae_video.md b/docs/source/en/api/models/autoencoder_kl_kvae_video.md index 0120dc2adc51..c0ce3bb36341 100644 --- a/docs/source/en/api/models/autoencoder_kl_kvae_video.md +++ b/docs/source/en/api/models/autoencoder_kl_kvae_video.md @@ -22,7 +22,7 @@ The model can be loaded with the following code snippet. import torch from diffusers import AutoencoderKLKVAEVideo -vae = AutoencoderKLKVAEVideo.from_pretrained("kandinskylab/KVAE-3D-1.0", subfolder="diffusers", torch_dtype=torch.float16) +vae = AutoencoderKLKVAEVideo.from_pretrained("kandinskylab/KVAE-3D-1.0", subfolder="diffusers", dtype=torch.float16) ``` ## AutoencoderKLKVAEVideo diff --git a/docs/source/en/api/models/autoencoder_kl_wan.md b/docs/source/en/api/models/autoencoder_kl_wan.md index 341e5e9a8736..23d5fa68251c 100644 --- a/docs/source/en/api/models/autoencoder_kl_wan.md +++ b/docs/source/en/api/models/autoencoder_kl_wan.md @@ -18,7 +18,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import AutoencoderKLWan -vae = AutoencoderKLWan.from_pretrained("Wan-AI/Wan2.1-T2V-1.3B-Diffusers", subfolder="vae", torch_dtype=torch.float32) +vae = AutoencoderKLWan.from_pretrained("Wan-AI/Wan2.1-T2V-1.3B-Diffusers", subfolder="vae", dtype=torch.float32) ``` ## AutoencoderKLWan diff --git a/docs/source/en/api/models/autoencoder_tiny.md b/docs/source/en/api/models/autoencoder_tiny.md index 19603f3a88a5..120c0273a3d0 100644 --- a/docs/source/en/api/models/autoencoder_tiny.md +++ b/docs/source/en/api/models/autoencoder_tiny.md @@ -21,9 +21,9 @@ import torch from diffusers import DiffusionPipeline, AutoencoderTiny pipe = DiffusionPipeline.from_pretrained( - "stabilityai/stable-diffusion-2-1-base", torch_dtype=torch.float16 + "stabilityai/stable-diffusion-2-1-base", dtype=torch.float16 ) -pipe.vae = AutoencoderTiny.from_pretrained("madebyollin/taesd", torch_dtype=torch.float16) +pipe.vae = AutoencoderTiny.from_pretrained("madebyollin/taesd", dtype=torch.float16) pipe = pipe.to("cuda") prompt = "slice of delicious New York-style berry cheesecake" @@ -38,9 +38,9 @@ import torch from diffusers import DiffusionPipeline, AutoencoderTiny pipe = DiffusionPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16 + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16 ) -pipe.vae = AutoencoderTiny.from_pretrained("madebyollin/taesdxl", torch_dtype=torch.float16) +pipe.vae = AutoencoderTiny.from_pretrained("madebyollin/taesdxl", dtype=torch.float16) pipe = pipe.to("cuda") prompt = "slice of delicious New York-style berry cheesecake" diff --git a/docs/source/en/api/models/autoencoderkl_allegro.md b/docs/source/en/api/models/autoencoderkl_allegro.md index 8c79479039fa..2f5220231026 100644 --- a/docs/source/en/api/models/autoencoderkl_allegro.md +++ b/docs/source/en/api/models/autoencoderkl_allegro.md @@ -18,7 +18,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import AutoencoderKLAllegro -vae = AutoencoderKLAllegro.from_pretrained("rhymes-ai/Allegro", subfolder="vae", torch_dtype=torch.float32).to("cuda") +vae = AutoencoderKLAllegro.from_pretrained("rhymes-ai/Allegro", subfolder="vae", dtype=torch.float32).to("cuda") ``` ## AutoencoderKLAllegro diff --git a/docs/source/en/api/models/autoencoderkl_audio_ltx_2.md b/docs/source/en/api/models/autoencoderkl_audio_ltx_2.md index d0024474e9e0..b3c4ea88175a 100644 --- a/docs/source/en/api/models/autoencoderkl_audio_ltx_2.md +++ b/docs/source/en/api/models/autoencoderkl_audio_ltx_2.md @@ -18,7 +18,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import AutoencoderKLLTX2Audio -vae = AutoencoderKLLTX2Audio.from_pretrained("Lightricks/LTX-2", subfolder="vae", torch_dtype=torch.float32).to("cuda") +vae = AutoencoderKLLTX2Audio.from_pretrained("Lightricks/LTX-2", subfolder="vae", dtype=torch.float32).to("cuda") ``` ## AutoencoderKLLTX2Audio diff --git a/docs/source/en/api/models/autoencoderkl_cogvideox.md b/docs/source/en/api/models/autoencoderkl_cogvideox.md index 2c5411a0647c..184739fc7157 100644 --- a/docs/source/en/api/models/autoencoderkl_cogvideox.md +++ b/docs/source/en/api/models/autoencoderkl_cogvideox.md @@ -18,7 +18,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import AutoencoderKLCogVideoX -vae = AutoencoderKLCogVideoX.from_pretrained("THUDM/CogVideoX-2b", subfolder="vae", torch_dtype=torch.float16).to("cuda") +vae = AutoencoderKLCogVideoX.from_pretrained("THUDM/CogVideoX-2b", subfolder="vae", dtype=torch.float16).to("cuda") ``` ## AutoencoderKLCogVideoX diff --git a/docs/source/en/api/models/autoencoderkl_ltx_2.md b/docs/source/en/api/models/autoencoderkl_ltx_2.md index 1dbf516c017a..4fc08e9f471a 100644 --- a/docs/source/en/api/models/autoencoderkl_ltx_2.md +++ b/docs/source/en/api/models/autoencoderkl_ltx_2.md @@ -18,7 +18,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import AutoencoderKLLTX2Video -vae = AutoencoderKLLTX2Video.from_pretrained("Lightricks/LTX-2", subfolder="vae", torch_dtype=torch.float32).to("cuda") +vae = AutoencoderKLLTX2Video.from_pretrained("Lightricks/LTX-2", subfolder="vae", dtype=torch.float32).to("cuda") ``` ## AutoencoderKLLTX2Video diff --git a/docs/source/en/api/models/autoencoderkl_ltx_video.md b/docs/source/en/api/models/autoencoderkl_ltx_video.md index 9c2384ca53a1..541d15ee32e4 100644 --- a/docs/source/en/api/models/autoencoderkl_ltx_video.md +++ b/docs/source/en/api/models/autoencoderkl_ltx_video.md @@ -18,7 +18,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import AutoencoderKLLTXVideo -vae = AutoencoderKLLTXVideo.from_pretrained("Lightricks/LTX-Video", subfolder="vae", torch_dtype=torch.float32).to("cuda") +vae = AutoencoderKLLTXVideo.from_pretrained("Lightricks/LTX-Video", subfolder="vae", dtype=torch.float32).to("cuda") ``` ## AutoencoderKLLTXVideo diff --git a/docs/source/en/api/models/autoencoderkl_magvit.md b/docs/source/en/api/models/autoencoderkl_magvit.md index 7c1060ddd435..762578eadb6f 100644 --- a/docs/source/en/api/models/autoencoderkl_magvit.md +++ b/docs/source/en/api/models/autoencoderkl_magvit.md @@ -18,7 +18,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import AutoencoderKLMagvit -vae = AutoencoderKLMagvit.from_pretrained("alibaba-pai/EasyAnimateV5.1-12b-zh", subfolder="vae", torch_dtype=torch.float16).to("cuda") +vae = AutoencoderKLMagvit.from_pretrained("alibaba-pai/EasyAnimateV5.1-12b-zh", subfolder="vae", dtype=torch.float16).to("cuda") ``` ## AutoencoderKLMagvit diff --git a/docs/source/en/api/models/autoencoderkl_mochi.md b/docs/source/en/api/models/autoencoderkl_mochi.md index fef6645a18fa..746b1b84c30b 100644 --- a/docs/source/en/api/models/autoencoderkl_mochi.md +++ b/docs/source/en/api/models/autoencoderkl_mochi.md @@ -18,7 +18,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import AutoencoderKLMochi -vae = AutoencoderKLMochi.from_pretrained("genmo/mochi-1-preview", subfolder="vae", torch_dtype=torch.float32).to("cuda") +vae = AutoencoderKLMochi.from_pretrained("genmo/mochi-1-preview", subfolder="vae", dtype=torch.float32).to("cuda") ``` ## AutoencoderKLMochi diff --git a/docs/source/en/api/models/chronoedit_transformer_3d.md b/docs/source/en/api/models/chronoedit_transformer_3d.md index 94982821795d..804eb7412397 100644 --- a/docs/source/en/api/models/chronoedit_transformer_3d.md +++ b/docs/source/en/api/models/chronoedit_transformer_3d.md @@ -20,7 +20,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import ChronoEditTransformer3DModel -transformer = ChronoEditTransformer3DModel.from_pretrained("nvidia/ChronoEdit-14B-Diffusers", subfolder="transformer", torch_dtype=torch.bfloat16) +transformer = ChronoEditTransformer3DModel.from_pretrained("nvidia/ChronoEdit-14B-Diffusers", subfolder="transformer", dtype=torch.bfloat16) ``` ## ChronoEditTransformer3DModel diff --git a/docs/source/en/api/models/cogvideox_transformer3d.md b/docs/source/en/api/models/cogvideox_transformer3d.md index 5d50e5dca651..1b09ba3d5ebc 100644 --- a/docs/source/en/api/models/cogvideox_transformer3d.md +++ b/docs/source/en/api/models/cogvideox_transformer3d.md @@ -18,7 +18,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import CogVideoXTransformer3DModel -transformer = CogVideoXTransformer3DModel.from_pretrained("THUDM/CogVideoX-2b", subfolder="transformer", torch_dtype=torch.float16).to("cuda") +transformer = CogVideoXTransformer3DModel.from_pretrained("THUDM/CogVideoX-2b", subfolder="transformer", dtype=torch.float16).to("cuda") ``` ## CogVideoXTransformer3DModel diff --git a/docs/source/en/api/models/cogview3plus_transformer2d.md b/docs/source/en/api/models/cogview3plus_transformer2d.md index 1fe574a7fb2f..2135926cc9cb 100644 --- a/docs/source/en/api/models/cogview3plus_transformer2d.md +++ b/docs/source/en/api/models/cogview3plus_transformer2d.md @@ -18,7 +18,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import CogView3PlusTransformer2DModel -transformer = CogView3PlusTransformer2DModel.from_pretrained("THUDM/CogView3Plus-3b", subfolder="transformer", torch_dtype=torch.bfloat16).to("cuda") +transformer = CogView3PlusTransformer2DModel.from_pretrained("THUDM/CogView3Plus-3b", subfolder="transformer", dtype=torch.bfloat16).to("cuda") ``` ## CogView3PlusTransformer2DModel diff --git a/docs/source/en/api/models/cogview4_transformer2d.md b/docs/source/en/api/models/cogview4_transformer2d.md index e87fbc680968..ed7b7abae8b2 100644 --- a/docs/source/en/api/models/cogview4_transformer2d.md +++ b/docs/source/en/api/models/cogview4_transformer2d.md @@ -18,7 +18,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import CogView4Transformer2DModel -transformer = CogView4Transformer2DModel.from_pretrained("THUDM/CogView4-6B", subfolder="transformer", torch_dtype=torch.bfloat16).to("cuda") +transformer = CogView4Transformer2DModel.from_pretrained("THUDM/CogView4-6B", subfolder="transformer", dtype=torch.bfloat16).to("cuda") ``` ## CogView4Transformer2DModel diff --git a/docs/source/en/api/models/consisid_transformer3d.md b/docs/source/en/api/models/consisid_transformer3d.md index 0531d475d2fb..98b49aee883c 100644 --- a/docs/source/en/api/models/consisid_transformer3d.md +++ b/docs/source/en/api/models/consisid_transformer3d.md @@ -18,7 +18,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import ConsisIDTransformer3DModel -transformer = ConsisIDTransformer3DModel.from_pretrained("BestWishYsh/ConsisID-preview", subfolder="transformer", torch_dtype=torch.bfloat16).to("cuda") +transformer = ConsisIDTransformer3DModel.from_pretrained("BestWishYsh/ConsisID-preview", subfolder="transformer", dtype=torch.bfloat16).to("cuda") ``` ## ConsisIDTransformer3DModel diff --git a/docs/source/en/api/models/controlnet.md b/docs/source/en/api/models/controlnet.md index 0821d63fd152..785ec6bbd0d3 100644 --- a/docs/source/en/api/models/controlnet.md +++ b/docs/source/en/api/models/controlnet.md @@ -43,7 +43,7 @@ from diffusers import ControlNetModel, UNet2DConditionModel lora_id = "stabilityai/control-lora" lora_filename = "control-LoRAs-rank128/control-lora-canny-rank128.safetensors" -unet = UNet2DConditionModel.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", subfolder="unet", torch_dtype=torch.bfloat16).to("cuda") +unet = UNet2DConditionModel.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", subfolder="unet", dtype=torch.bfloat16).to("cuda") controlnet = ControlNetModel.from_unet(unet).to(device="cuda", dtype=torch.bfloat16) controlnet.load_lora_adapter(lora_id, weight_name=lora_filename, prefix=None, controlnet_config=controlnet.config) ``` diff --git a/docs/source/en/api/models/controlnet_hunyuandit.md b/docs/source/en/api/models/controlnet_hunyuandit.md index 2ea5ab4b88d4..9e0e84bf57d6 100644 --- a/docs/source/en/api/models/controlnet_hunyuandit.md +++ b/docs/source/en/api/models/controlnet_hunyuandit.md @@ -29,7 +29,7 @@ This code is implemented by Tencent Hunyuan Team. You can find pre-trained check ```py from diffusers import HunyuanDiT2DControlNetModel import torch -controlnet = HunyuanDiT2DControlNetModel.from_pretrained("Tencent-Hunyuan/HunyuanDiT-v1.1-ControlNet-Diffusers-Pose", torch_dtype=torch.float16) +controlnet = HunyuanDiT2DControlNetModel.from_pretrained("Tencent-Hunyuan/HunyuanDiT-v1.1-ControlNet-Diffusers-Pose", dtype=torch.float16) ``` ## HunyuanDiT2DControlNetModel diff --git a/docs/source/en/api/models/controlnet_sparsectrl.md b/docs/source/en/api/models/controlnet_sparsectrl.md index 0aa9848d0d2b..2322c9dd2739 100644 --- a/docs/source/en/api/models/controlnet_sparsectrl.md +++ b/docs/source/en/api/models/controlnet_sparsectrl.md @@ -29,10 +29,10 @@ from diffusers import SparseControlNetModel # fp32 variant in float16 # 1. Scribble checkpoint -controlnet = SparseControlNetModel.from_pretrained("guoyww/animatediff-sparsectrl-scribble", torch_dtype=torch.float16) +controlnet = SparseControlNetModel.from_pretrained("guoyww/animatediff-sparsectrl-scribble", dtype=torch.float16) # 2. RGB checkpoint -controlnet = SparseControlNetModel.from_pretrained("guoyww/animatediff-sparsectrl-rgb", torch_dtype=torch.float16) +controlnet = SparseControlNetModel.from_pretrained("guoyww/animatediff-sparsectrl-rgb", dtype=torch.float16) # For loading fp16 variant, pass `variant="fp16"` as an additional parameter ``` diff --git a/docs/source/en/api/models/cosmos3_omni_transformer.md b/docs/source/en/api/models/cosmos3_omni_transformer.md index 585276d81009..81f3709c859e 100644 --- a/docs/source/en/api/models/cosmos3_omni_transformer.md +++ b/docs/source/en/api/models/cosmos3_omni_transformer.md @@ -25,7 +25,7 @@ import torch from diffusers import Cosmos3OmniTransformer transformer = Cosmos3OmniTransformer.from_pretrained( - "nvidia/Cosmos3-Nano", subfolder="transformer", torch_dtype=torch.bfloat16 + "nvidia/Cosmos3-Nano", subfolder="transformer", dtype=torch.bfloat16 ) ``` diff --git a/docs/source/en/api/models/cosmos_transformer3d.md b/docs/source/en/api/models/cosmos_transformer3d.md index eb52c82e5f05..42efabc71763 100644 --- a/docs/source/en/api/models/cosmos_transformer3d.md +++ b/docs/source/en/api/models/cosmos_transformer3d.md @@ -18,7 +18,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import CosmosTransformer3DModel -transformer = CosmosTransformer3DModel.from_pretrained("nvidia/Cosmos-1.0-Diffusion-7B-Text2World", subfolder="transformer", torch_dtype=torch.bfloat16) +transformer = CosmosTransformer3DModel.from_pretrained("nvidia/Cosmos-1.0-Diffusion-7B-Text2World", subfolder="transformer", dtype=torch.bfloat16) ``` ## CosmosTransformer3DModel diff --git a/docs/source/en/api/models/easyanimate_transformer3d.md b/docs/source/en/api/models/easyanimate_transformer3d.md index 66670eb632d4..72311466076f 100644 --- a/docs/source/en/api/models/easyanimate_transformer3d.md +++ b/docs/source/en/api/models/easyanimate_transformer3d.md @@ -18,7 +18,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import EasyAnimateTransformer3DModel -transformer = EasyAnimateTransformer3DModel.from_pretrained("alibaba-pai/EasyAnimateV5.1-12b-zh", subfolder="transformer", torch_dtype=torch.float16).to("cuda") +transformer = EasyAnimateTransformer3DModel.from_pretrained("alibaba-pai/EasyAnimateV5.1-12b-zh", subfolder="transformer", dtype=torch.float16).to("cuda") ``` ## EasyAnimateTransformer3DModel diff --git a/docs/source/en/api/models/helios_transformer3d.md b/docs/source/en/api/models/helios_transformer3d.md index 302b91d6c829..59ecc484995b 100644 --- a/docs/source/en/api/models/helios_transformer3d.md +++ b/docs/source/en/api/models/helios_transformer3d.md @@ -19,11 +19,11 @@ The model can be loaded with the following code snippet. from diffusers import HeliosTransformer3DModel # Best Quality -transformer = HeliosTransformer3DModel.from_pretrained("BestWishYsh/Helios-Base", subfolder="transformer", torch_dtype=torch.bfloat16) +transformer = HeliosTransformer3DModel.from_pretrained("BestWishYsh/Helios-Base", subfolder="transformer", dtype=torch.bfloat16) # Intermediate Weight -transformer = HeliosTransformer3DModel.from_pretrained("BestWishYsh/Helios-Mid", subfolder="transformer", torch_dtype=torch.bfloat16) +transformer = HeliosTransformer3DModel.from_pretrained("BestWishYsh/Helios-Mid", subfolder="transformer", dtype=torch.bfloat16) # Best Efficiency -transformer = HeliosTransformer3DModel.from_pretrained("BestWishYsh/Helios-Distilled", subfolder="transformer", torch_dtype=torch.bfloat16) +transformer = HeliosTransformer3DModel.from_pretrained("BestWishYsh/Helios-Distilled", subfolder="transformer", dtype=torch.bfloat16) ``` ## HeliosTransformer3DModel diff --git a/docs/source/en/api/models/hidream_image_transformer.md b/docs/source/en/api/models/hidream_image_transformer.md index b562c2f8543c..535ef98b6722 100644 --- a/docs/source/en/api/models/hidream_image_transformer.md +++ b/docs/source/en/api/models/hidream_image_transformer.md @@ -18,7 +18,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import HiDreamImageTransformer2DModel -transformer = HiDreamImageTransformer2DModel.from_pretrained("HiDream-ai/HiDream-I1-Full", subfolder="transformer", torch_dtype=torch.bfloat16) +transformer = HiDreamImageTransformer2DModel.from_pretrained("HiDream-ai/HiDream-I1-Full", subfolder="transformer", dtype=torch.bfloat16) ``` ## Loading GGUF quantized checkpoints for HiDream-I1 @@ -33,7 +33,7 @@ ckpt_path = "https://huggingface.co/city96/HiDream-I1-Dev-gguf/blob/main/hidream transformer = HiDreamImageTransformer2DModel.from_single_file( ckpt_path, quantization_config=GGUFQuantizationConfig(compute_dtype=torch.bfloat16), - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) ``` diff --git a/docs/source/en/api/models/hunyuan_video15_transformer_3d.md b/docs/source/en/api/models/hunyuan_video15_transformer_3d.md index 5ad4c6f4643f..5403d45d1af0 100644 --- a/docs/source/en/api/models/hunyuan_video15_transformer_3d.md +++ b/docs/source/en/api/models/hunyuan_video15_transformer_3d.md @@ -18,7 +18,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import HunyuanVideo15Transformer3DModel -transformer = HunyuanVideo15Transformer3DModel.from_pretrained("hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-480p_t2v" subfolder="transformer", torch_dtype=torch.bfloat16) +transformer = HunyuanVideo15Transformer3DModel.from_pretrained("hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-480p_t2v" subfolder="transformer", dtype=torch.bfloat16) ``` ## HunyuanVideo15Transformer3DModel diff --git a/docs/source/en/api/models/hunyuan_video_transformer_3d.md b/docs/source/en/api/models/hunyuan_video_transformer_3d.md index 77d30e5553bc..5671709ec597 100644 --- a/docs/source/en/api/models/hunyuan_video_transformer_3d.md +++ b/docs/source/en/api/models/hunyuan_video_transformer_3d.md @@ -18,7 +18,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import HunyuanVideoTransformer3DModel -transformer = HunyuanVideoTransformer3DModel.from_pretrained("hunyuanvideo-community/HunyuanVideo", subfolder="transformer", torch_dtype=torch.bfloat16) +transformer = HunyuanVideoTransformer3DModel.from_pretrained("hunyuanvideo-community/HunyuanVideo", subfolder="transformer", dtype=torch.bfloat16) ``` ## HunyuanVideoTransformer3DModel diff --git a/docs/source/en/api/models/hunyuanimage_transformer_2d.md b/docs/source/en/api/models/hunyuanimage_transformer_2d.md index 2dd8e6b52d52..6c3bc9f6467a 100644 --- a/docs/source/en/api/models/hunyuanimage_transformer_2d.md +++ b/docs/source/en/api/models/hunyuanimage_transformer_2d.md @@ -18,7 +18,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import HunyuanImageTransformer2DModel -transformer = HunyuanImageTransformer2DModel.from_pretrained("hunyuanvideo-community/HunyuanImage-2.1-Diffusers", subfolder="transformer", torch_dtype=torch.bfloat16) +transformer = HunyuanImageTransformer2DModel.from_pretrained("hunyuanvideo-community/HunyuanImage-2.1-Diffusers", subfolder="transformer", dtype=torch.bfloat16) ``` ## HunyuanImageTransformer2DModel diff --git a/docs/source/en/api/models/longcat_image_transformer2d.md b/docs/source/en/api/models/longcat_image_transformer2d.md index f40b2583e68b..21a3dd458932 100644 --- a/docs/source/en/api/models/longcat_image_transformer2d.md +++ b/docs/source/en/api/models/longcat_image_transformer2d.md @@ -17,7 +17,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import LongCatImageTransformer2DModel -transformer = LongCatImageTransformer2DModel.from_pretrained("meituan-longcat/LongCat-Image ", subfolder="transformer", torch_dtype=torch.bfloat16) +transformer = LongCatImageTransformer2DModel.from_pretrained("meituan-longcat/LongCat-Image ", subfolder="transformer", dtype=torch.bfloat16) ``` ## LongCatImageTransformer2DModel diff --git a/docs/source/en/api/models/ltx2_video_transformer3d.md b/docs/source/en/api/models/ltx2_video_transformer3d.md index 9faab8695468..279564360d6a 100644 --- a/docs/source/en/api/models/ltx2_video_transformer3d.md +++ b/docs/source/en/api/models/ltx2_video_transformer3d.md @@ -18,7 +18,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import LTX2VideoTransformer3DModel -transformer = LTX2VideoTransformer3DModel.from_pretrained("Lightricks/LTX-2", subfolder="transformer", torch_dtype=torch.bfloat16).to("cuda") +transformer = LTX2VideoTransformer3DModel.from_pretrained("Lightricks/LTX-2", subfolder="transformer", dtype=torch.bfloat16).to("cuda") ``` ## LTX2VideoTransformer3DModel diff --git a/docs/source/en/api/models/ltx_video_transformer3d.md b/docs/source/en/api/models/ltx_video_transformer3d.md index 5a2a1af9d821..d04b12c9090c 100644 --- a/docs/source/en/api/models/ltx_video_transformer3d.md +++ b/docs/source/en/api/models/ltx_video_transformer3d.md @@ -18,7 +18,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import LTXVideoTransformer3DModel -transformer = LTXVideoTransformer3DModel.from_pretrained("Lightricks/LTX-Video", subfolder="transformer", torch_dtype=torch.bfloat16).to("cuda") +transformer = LTXVideoTransformer3DModel.from_pretrained("Lightricks/LTX-Video", subfolder="transformer", dtype=torch.bfloat16).to("cuda") ``` ## LTXVideoTransformer3DModel diff --git a/docs/source/en/api/models/lumina2_transformer2d.md b/docs/source/en/api/models/lumina2_transformer2d.md index 2c85325f73f5..407250c31d44 100644 --- a/docs/source/en/api/models/lumina2_transformer2d.md +++ b/docs/source/en/api/models/lumina2_transformer2d.md @@ -18,7 +18,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import Lumina2Transformer2DModel -transformer = Lumina2Transformer2DModel.from_pretrained("Alpha-VLLM/Lumina-Image-2.0", subfolder="transformer", torch_dtype=torch.bfloat16) +transformer = Lumina2Transformer2DModel.from_pretrained("Alpha-VLLM/Lumina-Image-2.0", subfolder="transformer", dtype=torch.bfloat16) ``` ## Lumina2Transformer2DModel diff --git a/docs/source/en/api/models/mochi_transformer3d.md b/docs/source/en/api/models/mochi_transformer3d.md index 6c8c21ce91cd..2329d3bedb77 100644 --- a/docs/source/en/api/models/mochi_transformer3d.md +++ b/docs/source/en/api/models/mochi_transformer3d.md @@ -18,7 +18,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import MochiTransformer3DModel -transformer = MochiTransformer3DModel.from_pretrained("genmo/mochi-1-preview", subfolder="transformer", torch_dtype=torch.float16).to("cuda") +transformer = MochiTransformer3DModel.from_pretrained("genmo/mochi-1-preview", subfolder="transformer", dtype=torch.float16).to("cuda") ``` ## MochiTransformer3DModel diff --git a/docs/source/en/api/models/motif_video_transformer_3d.md b/docs/source/en/api/models/motif_video_transformer_3d.md index 011058832ee2..a25597ca6d78 100644 --- a/docs/source/en/api/models/motif_video_transformer_3d.md +++ b/docs/source/en/api/models/motif_video_transformer_3d.md @@ -20,7 +20,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import MotifVideoTransformer3DModel -transformer = MotifVideoTransformer3DModel.from_pretrained("Motif-Technologies/Motif-Video-2B", subfolder="transformer", torch_dtype=torch.bfloat16) +transformer = MotifVideoTransformer3DModel.from_pretrained("Motif-Technologies/Motif-Video-2B", subfolder="transformer", dtype=torch.bfloat16) ``` ## MotifVideoTransformer3DModel diff --git a/docs/source/en/api/models/omnigen_transformer.md b/docs/source/en/api/models/omnigen_transformer.md index a2584bc8e76d..50918abb6527 100644 --- a/docs/source/en/api/models/omnigen_transformer.md +++ b/docs/source/en/api/models/omnigen_transformer.md @@ -22,7 +22,7 @@ The abstract from the paper is: import torch from diffusers import OmniGenTransformer2DModel -transformer = OmniGenTransformer2DModel.from_pretrained("Shitao/OmniGen-v1-diffusers", subfolder="transformer", torch_dtype=torch.bfloat16) +transformer = OmniGenTransformer2DModel.from_pretrained("Shitao/OmniGen-v1-diffusers", subfolder="transformer", dtype=torch.bfloat16) ``` ## OmniGenTransformer2DModel diff --git a/docs/source/en/api/models/ovisimage_transformer2d.md b/docs/source/en/api/models/ovisimage_transformer2d.md index 484652404af3..03fc8589502e 100644 --- a/docs/source/en/api/models/ovisimage_transformer2d.md +++ b/docs/source/en/api/models/ovisimage_transformer2d.md @@ -16,7 +16,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import OvisImageTransformer2DModel -transformer = OvisImageTransformer2DModel.from_pretrained("AIDC-AI/Ovis-Image-7B", subfolder="transformer", torch_dtype=torch.bfloat16) +transformer = OvisImageTransformer2DModel.from_pretrained("AIDC-AI/Ovis-Image-7B", subfolder="transformer", dtype=torch.bfloat16) ``` ## OvisImageTransformer2DModel diff --git a/docs/source/en/api/models/qwenimage_transformer2d.md b/docs/source/en/api/models/qwenimage_transformer2d.md index c78623084e1c..bad711605481 100644 --- a/docs/source/en/api/models/qwenimage_transformer2d.md +++ b/docs/source/en/api/models/qwenimage_transformer2d.md @@ -16,7 +16,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import QwenImageTransformer2DModel -transformer = QwenImageTransformer2DModel.from_pretrained("Qwen/QwenImage-20B", subfolder="transformer", torch_dtype=torch.bfloat16) +transformer = QwenImageTransformer2DModel.from_pretrained("Qwen/QwenImage-20B", subfolder="transformer", dtype=torch.bfloat16) ``` ## QwenImageTransformer2DModel diff --git a/docs/source/en/api/models/sana_transformer2d.md b/docs/source/en/api/models/sana_transformer2d.md index e3e5fde3a79e..4397f54e292f 100644 --- a/docs/source/en/api/models/sana_transformer2d.md +++ b/docs/source/en/api/models/sana_transformer2d.md @@ -22,7 +22,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import SanaTransformer2DModel -transformer = SanaTransformer2DModel.from_pretrained("Efficient-Large-Model/Sana_1600M_1024px_BF16_diffusers", subfolder="transformer", torch_dtype=torch.bfloat16) +transformer = SanaTransformer2DModel.from_pretrained("Efficient-Large-Model/Sana_1600M_1024px_BF16_diffusers", subfolder="transformer", dtype=torch.bfloat16) ``` ## SanaTransformer2DModel diff --git a/docs/source/en/api/models/sana_video_transformer3d.md b/docs/source/en/api/models/sana_video_transformer3d.md index 0cf1451a2d39..175c2460dda4 100644 --- a/docs/source/en/api/models/sana_video_transformer3d.md +++ b/docs/source/en/api/models/sana_video_transformer3d.md @@ -23,7 +23,7 @@ The model can be loaded with the following code snippet. from diffusers import SanaVideoTransformer3DModel import torch -transformer = SanaVideoTransformer3DModel.from_pretrained("Efficient-Large-Model/SANA-Video_2B_480p_diffusers", subfolder="transformer", torch_dtype=torch.bfloat16) +transformer = SanaVideoTransformer3DModel.from_pretrained("Efficient-Large-Model/SANA-Video_2B_480p_diffusers", subfolder="transformer", dtype=torch.bfloat16) ``` ## SanaVideoTransformer3DModel diff --git a/docs/source/en/api/models/skyreels_v2_transformer_3d.md b/docs/source/en/api/models/skyreels_v2_transformer_3d.md index c1c8c2c7bcce..6edfa69b1116 100644 --- a/docs/source/en/api/models/skyreels_v2_transformer_3d.md +++ b/docs/source/en/api/models/skyreels_v2_transformer_3d.md @@ -18,7 +18,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import SkyReelsV2Transformer3DModel -transformer = SkyReelsV2Transformer3DModel.from_pretrained("Skywork/SkyReels-V2-DF-1.3B-540P-Diffusers", subfolder="transformer", torch_dtype=torch.bfloat16) +transformer = SkyReelsV2Transformer3DModel.from_pretrained("Skywork/SkyReels-V2-DF-1.3B-540P-Diffusers", subfolder="transformer", dtype=torch.bfloat16) ``` ## SkyReelsV2Transformer3DModel diff --git a/docs/source/en/api/models/transformer_joyimage.md b/docs/source/en/api/models/transformer_joyimage.md index 8b18ab6d5b6a..a07b6721579b 100644 --- a/docs/source/en/api/models/transformer_joyimage.md +++ b/docs/source/en/api/models/transformer_joyimage.md @@ -17,7 +17,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import JoyImageEditTransformer3DModel -transformer = JoyImageEditTransformer3DModel.from_pretrained("jdopensource/JoyAI-Image-Edit-Diffusers", subfolder="transformer", torch_dtype=torch.bfloat16) +transformer = JoyImageEditTransformer3DModel.from_pretrained("jdopensource/JoyAI-Image-Edit-Diffusers", subfolder="transformer", dtype=torch.bfloat16) ``` ## JoyImageEditTransformer3DModel diff --git a/docs/source/en/api/models/transformer_joyimage_edit_plus.md b/docs/source/en/api/models/transformer_joyimage_edit_plus.md index 776c53eaf20c..da37346a82ea 100644 --- a/docs/source/en/api/models/transformer_joyimage_edit_plus.md +++ b/docs/source/en/api/models/transformer_joyimage_edit_plus.md @@ -17,7 +17,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import JoyImageEditPlusTransformer3DModel -transformer = JoyImageEditPlusTransformer3DModel.from_pretrained("jdopensource/JoyAI-Image-Edit-Plus-Diffusers", subfolder="transformer", torch_dtype=torch.bfloat16) +transformer = JoyImageEditPlusTransformer3DModel.from_pretrained("jdopensource/JoyAI-Image-Edit-Plus-Diffusers", subfolder="transformer", dtype=torch.bfloat16) ``` ## JoyImageEditPlusTransformer3DModel diff --git a/docs/source/en/api/models/wan_animate_transformer_3d.md b/docs/source/en/api/models/wan_animate_transformer_3d.md index cc7b3f0c408c..35c671b21f7f 100644 --- a/docs/source/en/api/models/wan_animate_transformer_3d.md +++ b/docs/source/en/api/models/wan_animate_transformer_3d.md @@ -18,7 +18,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import WanAnimateTransformer3DModel -transformer = WanAnimateTransformer3DModel.from_pretrained("Wan-AI/Wan2.2-Animate-14B-Diffusers", subfolder="transformer", torch_dtype=torch.bfloat16) +transformer = WanAnimateTransformer3DModel.from_pretrained("Wan-AI/Wan2.2-Animate-14B-Diffusers", subfolder="transformer", dtype=torch.bfloat16) ``` ## WanAnimateTransformer3DModel diff --git a/docs/source/en/api/models/wan_transformer_3d.md b/docs/source/en/api/models/wan_transformer_3d.md index c218166584c6..aad35b9c84ec 100644 --- a/docs/source/en/api/models/wan_transformer_3d.md +++ b/docs/source/en/api/models/wan_transformer_3d.md @@ -18,7 +18,7 @@ The model can be loaded with the following code snippet. ```python from diffusers import WanTransformer3DModel -transformer = WanTransformer3DModel.from_pretrained("Wan-AI/Wan2.1-T2V-1.3B-Diffusers", subfolder="transformer", torch_dtype=torch.bfloat16) +transformer = WanTransformer3DModel.from_pretrained("Wan-AI/Wan2.1-T2V-1.3B-Diffusers", subfolder="transformer", dtype=torch.bfloat16) ``` ## WanTransformer3DModel diff --git a/docs/source/en/api/pipelines/ace_step.md b/docs/source/en/api/pipelines/ace_step.md index df6af1406fa3..ca1047092e0e 100644 --- a/docs/source/en/api/pipelines/ace_step.md +++ b/docs/source/en/api/pipelines/ace_step.md @@ -54,7 +54,7 @@ import torch import soundfile as sf from diffusers import AceStepPipeline -pipe = AceStepPipeline.from_pretrained("ACE-Step/acestep-v15-xl-turbo-diffusers", torch_dtype=torch.bfloat16) +pipe = AceStepPipeline.from_pretrained("ACE-Step/acestep-v15-xl-turbo-diffusers", dtype=torch.bfloat16) pipe = pipe.to("cuda") audio = pipe( diff --git a/docs/source/en/api/pipelines/allegro.md b/docs/source/en/api/pipelines/allegro.md index a981fb1f94f7..829f8fa55d0a 100644 --- a/docs/source/en/api/pipelines/allegro.md +++ b/docs/source/en/api/pipelines/allegro.md @@ -37,7 +37,7 @@ text_encoder_8bit = T5EncoderModel.from_pretrained( "rhymes-ai/Allegro", subfolder="text_encoder", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) quant_config = DiffusersBitsAndBytesConfig(load_in_8bit=True) @@ -45,14 +45,14 @@ transformer_8bit = AllegroTransformer3DModel.from_pretrained( "rhymes-ai/Allegro", subfolder="transformer", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) pipeline = AllegroPipeline.from_pretrained( "rhymes-ai/Allegro", text_encoder=text_encoder_8bit, transformer=transformer_8bit, - torch_dtype=torch.float16, + dtype=torch.float16, device_map="balanced", ) diff --git a/docs/source/en/api/pipelines/anima.md b/docs/source/en/api/pipelines/anima.md index b66eeb2a29b7..5ac544f3caaa 100644 --- a/docs/source/en/api/pipelines/anima.md +++ b/docs/source/en/api/pipelines/anima.md @@ -21,7 +21,7 @@ import torch from diffusers import ModularPipeline pipe = ModularPipeline.from_pretrained("circlestone-labs/Anima-Base-v1.0-Diffusers") -pipe.load_components(torch_dtype=torch.bfloat16) +pipe.load_components(dtype=torch.bfloat16) pipe.to("cuda") image = pipe(prompt="masterpiece, best quality, 1girl, solo, city lights").images[0] diff --git a/docs/source/en/api/pipelines/animatediff.md b/docs/source/en/api/pipelines/animatediff.md index 60057e15eedc..60c3fa5fa66d 100644 --- a/docs/source/en/api/pipelines/animatediff.md +++ b/docs/source/en/api/pipelines/animatediff.md @@ -53,10 +53,10 @@ from diffusers import AnimateDiffPipeline, DDIMScheduler, MotionAdapter from diffusers.utils import export_to_gif # Load the motion adapter -adapter = MotionAdapter.from_pretrained("guoyww/animatediff-motion-adapter-v1-5-2", torch_dtype=torch.float16) +adapter = MotionAdapter.from_pretrained("guoyww/animatediff-motion-adapter-v1-5-2", dtype=torch.float16) # load SD 1.5 based finetuned model model_id = "SG161222/Realistic_Vision_V5.1_noVAE" -pipe = AnimateDiffPipeline.from_pretrained(model_id, motion_adapter=adapter, torch_dtype=torch.float16) +pipe = AnimateDiffPipeline.from_pretrained(model_id, motion_adapter=adapter, dtype=torch.float16) scheduler = DDIMScheduler.from_pretrained( model_id, subfolder="scheduler", @@ -120,12 +120,12 @@ from controlnet_aux.processor import ZoeDetector # Download controlnets from https://huggingface.co/lllyasviel/ControlNet-v1-1 to use .from_single_file # Download Diffusers-format controlnets, such as https://huggingface.co/lllyasviel/sd-controlnet-depth, to use .from_pretrained() -controlnet = ControlNetModel.from_single_file("control_v11f1p_sd15_depth.pth", torch_dtype=torch.float16) +controlnet = ControlNetModel.from_single_file("control_v11f1p_sd15_depth.pth", dtype=torch.float16) # We use AnimateLCM for this example but one can use the original motion adapters as well (for example, https://huggingface.co/guoyww/animatediff-motion-adapter-v1-5-3) motion_adapter = MotionAdapter.from_pretrained("wangfuyun/AnimateLCM") -vae = AutoencoderKL.from_pretrained("stabilityai/sd-vae-ft-mse", torch_dtype=torch.float16) +vae = AutoencoderKL.from_pretrained("stabilityai/sd-vae-ft-mse", dtype=torch.float16) pipe: AnimateDiffControlNetPipeline = AnimateDiffControlNetPipeline.from_pretrained( "SG161222/Realistic_Vision_V5.1_noVAE", motion_adapter=motion_adapter, @@ -213,9 +213,9 @@ lora_adapter_id = "guoyww/animatediff-motion-lora-v1-5-3" vae_id = "stabilityai/sd-vae-ft-mse" device = "cuda" -motion_adapter = MotionAdapter.from_pretrained(motion_adapter_id, torch_dtype=torch.float16).to(device) -controlnet = SparseControlNetModel.from_pretrained(controlnet_id, torch_dtype=torch.float16).to(device) -vae = AutoencoderKL.from_pretrained(vae_id, torch_dtype=torch.float16).to(device) +motion_adapter = MotionAdapter.from_pretrained(motion_adapter_id, dtype=torch.float16).to(device) +controlnet = SparseControlNetModel.from_pretrained(controlnet_id, dtype=torch.float16).to(device) +vae = AutoencoderKL.from_pretrained(vae_id, dtype=torch.float16).to(device) scheduler = DPMSolverMultistepScheduler.from_pretrained( model_id, subfolder="scheduler", @@ -229,7 +229,7 @@ pipe = AnimateDiffSparseControlNetPipeline.from_pretrained( controlnet=controlnet, vae=vae, scheduler=scheduler, - torch_dtype=torch.float16, + dtype=torch.float16, ).to(device) pipe.load_lora_weights(lora_adapter_id, adapter_name="motion_lora") pipe.fuse_lora(lora_scale=1.0) @@ -309,9 +309,9 @@ lora_adapter_id = "guoyww/animatediff-motion-lora-v1-5-3" vae_id = "stabilityai/sd-vae-ft-mse" device = "cuda" -motion_adapter = MotionAdapter.from_pretrained(motion_adapter_id, torch_dtype=torch.float16).to(device) -controlnet = SparseControlNetModel.from_pretrained(controlnet_id, torch_dtype=torch.float16).to(device) -vae = AutoencoderKL.from_pretrained(vae_id, torch_dtype=torch.float16).to(device) +motion_adapter = MotionAdapter.from_pretrained(motion_adapter_id, dtype=torch.float16).to(device) +controlnet = SparseControlNetModel.from_pretrained(controlnet_id, dtype=torch.float16).to(device) +vae = AutoencoderKL.from_pretrained(vae_id, dtype=torch.float16).to(device) scheduler = DPMSolverMultistepScheduler.from_pretrained( model_id, subfolder="scheduler", @@ -325,7 +325,7 @@ pipe = AnimateDiffSparseControlNetPipeline.from_pretrained( controlnet=controlnet, vae=vae, scheduler=scheduler, - torch_dtype=torch.float16, + dtype=torch.float16, ).to(device) pipe.load_lora_weights(lora_adapter_id, adapter_name="motion_lora") @@ -375,7 +375,7 @@ from diffusers.models import MotionAdapter from diffusers import AnimateDiffSDXLPipeline, DDIMScheduler from diffusers.utils import export_to_gif -adapter = MotionAdapter.from_pretrained("guoyww/animatediff-motion-adapter-sdxl-beta", torch_dtype=torch.float16) +adapter = MotionAdapter.from_pretrained("guoyww/animatediff-motion-adapter-sdxl-beta", dtype=torch.float16) model_id = "stabilityai/stable-diffusion-xl-base-1.0" scheduler = DDIMScheduler.from_pretrained( @@ -390,7 +390,7 @@ pipe = AnimateDiffSDXLPipeline.from_pretrained( model_id, motion_adapter=adapter, scheduler=scheduler, - torch_dtype=torch.float16, + dtype=torch.float16, variant="fp16", ).to("cuda") @@ -426,10 +426,10 @@ from io import BytesIO from PIL import Image # Load the motion adapter -adapter = MotionAdapter.from_pretrained("guoyww/animatediff-motion-adapter-v1-5-2", torch_dtype=torch.float16) +adapter = MotionAdapter.from_pretrained("guoyww/animatediff-motion-adapter-v1-5-2", dtype=torch.float16) # load SD 1.5 based finetuned model model_id = "SG161222/Realistic_Vision_V5.1_noVAE" -pipe = AnimateDiffVideoToVideoPipeline.from_pretrained(model_id, motion_adapter=adapter, torch_dtype=torch.float16) +pipe = AnimateDiffVideoToVideoPipeline.from_pretrained(model_id, motion_adapter=adapter, dtype=torch.float16) scheduler = DDIMScheduler.from_pretrained( model_id, subfolder="scheduler", @@ -539,11 +539,11 @@ from diffusers.utils import export_to_gif, load_video from diffusers import AutoencoderKL, ControlNetModel, MotionAdapter, LCMScheduler # Load the ControlNet -controlnet = ControlNetModel.from_pretrained("lllyasviel/sd-controlnet-openpose", torch_dtype=torch.float16) +controlnet = ControlNetModel.from_pretrained("lllyasviel/sd-controlnet-openpose", dtype=torch.float16) # Load the motion adapter motion_adapter = MotionAdapter.from_pretrained("wangfuyun/AnimateLCM") # Load SD 1.5 based finetuned model -vae = AutoencoderKL.from_pretrained("stabilityai/sd-vae-ft-mse", torch_dtype=torch.float16) +vae = AutoencoderKL.from_pretrained("stabilityai/sd-vae-ft-mse", dtype=torch.float16) pipe = AnimateDiffVideoToVideoControlNetPipeline.from_pretrained( "SG161222/Realistic_Vision_V5.1_noVAE", motion_adapter=motion_adapter, @@ -621,10 +621,10 @@ from diffusers import AnimateDiffPipeline, DDIMScheduler, MotionAdapter from diffusers.utils import export_to_gif # Load the motion adapter -adapter = MotionAdapter.from_pretrained("guoyww/animatediff-motion-adapter-v1-5-2", torch_dtype=torch.float16) +adapter = MotionAdapter.from_pretrained("guoyww/animatediff-motion-adapter-v1-5-2", dtype=torch.float16) # load SD 1.5 based finetuned model model_id = "SG161222/Realistic_Vision_V5.1_noVAE" -pipe = AnimateDiffPipeline.from_pretrained(model_id, motion_adapter=adapter, torch_dtype=torch.float16) +pipe = AnimateDiffPipeline.from_pretrained(model_id, motion_adapter=adapter, dtype=torch.float16) pipe.load_lora_weights( "guoyww/animatediff-motion-lora-zoom-out", adapter_name="zoom-out" ) @@ -690,10 +690,10 @@ from diffusers import AnimateDiffPipeline, DDIMScheduler, MotionAdapter from diffusers.utils import export_to_gif # Load the motion adapter -adapter = MotionAdapter.from_pretrained("guoyww/animatediff-motion-adapter-v1-5-2", torch_dtype=torch.float16) +adapter = MotionAdapter.from_pretrained("guoyww/animatediff-motion-adapter-v1-5-2", dtype=torch.float16) # load SD 1.5 based finetuned model model_id = "SG161222/Realistic_Vision_V5.1_noVAE" -pipe = AnimateDiffPipeline.from_pretrained(model_id, motion_adapter=adapter, torch_dtype=torch.float16) +pipe = AnimateDiffPipeline.from_pretrained(model_id, motion_adapter=adapter, dtype=torch.float16) pipe.load_lora_weights( "diffusers/animatediff-motion-lora-zoom-out", adapter_name="zoom-out", @@ -761,7 +761,7 @@ from diffusers.utils import export_to_gif adapter = MotionAdapter.from_pretrained("guoyww/animatediff-motion-adapter-v1-5-2") model_id = "SG161222/Realistic_Vision_V5.1_noVAE" -pipe = AnimateDiffPipeline.from_pretrained(model_id, motion_adapter=adapter, torch_dtype=torch.float16).to("cuda") +pipe = AnimateDiffPipeline.from_pretrained(model_id, motion_adapter=adapter, dtype=torch.float16).to("cuda") pipe.scheduler = DDIMScheduler.from_pretrained( model_id, subfolder="scheduler", @@ -938,10 +938,10 @@ from diffusers.utils import export_to_video, load_image # Load pipeline dtype = torch.float16 -motion_adapter = MotionAdapter.from_pretrained("wangfuyun/AnimateLCM", torch_dtype=dtype) -vae = AutoencoderKL.from_pretrained("stabilityai/sd-vae-ft-mse", torch_dtype=dtype) +motion_adapter = MotionAdapter.from_pretrained("wangfuyun/AnimateLCM", dtype=dtype) +vae = AutoencoderKL.from_pretrained("stabilityai/sd-vae-ft-mse", dtype=dtype) -pipe = AnimateDiffPipeline.from_pretrained("emilianJR/epiCRealism", motion_adapter=motion_adapter, vae=vae, torch_dtype=dtype) +pipe = AnimateDiffPipeline.from_pretrained("emilianJR/epiCRealism", motion_adapter=motion_adapter, vae=vae, dtype=dtype) pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config, beta_schedule="linear") pipe.load_lora_weights( @@ -1002,7 +1002,7 @@ from diffusers import MotionAdapter ckpt_path = "https://huggingface.co/Lightricks/LongAnimateDiff/blob/main/lt_long_mm_32_frames.ckpt" -adapter = MotionAdapter.from_single_file(ckpt_path, torch_dtype=torch.float16) +adapter = MotionAdapter.from_single_file(ckpt_path, dtype=torch.float16) pipe = AnimateDiffPipeline.from_pretrained("emilianJR/epiCRealism", motion_adapter=adapter) ``` diff --git a/docs/source/en/api/pipelines/anyflow.md b/docs/source/en/api/pipelines/anyflow.md index 9e496a61113f..7667f8fc1ffa 100644 --- a/docs/source/en/api/pipelines/anyflow.md +++ b/docs/source/en/api/pipelines/anyflow.md @@ -48,7 +48,7 @@ from diffusers import AnyFlowPipeline from diffusers.utils import export_to_video pipe = AnyFlowPipeline.from_pretrained( - "nvidia/AnyFlow-Wan2.1-T2V-1.3B-Diffusers", torch_dtype=torch.bfloat16 + "nvidia/AnyFlow-Wan2.1-T2V-1.3B-Diffusers", dtype=torch.bfloat16 ).to("cuda") prompt = ( @@ -84,7 +84,7 @@ from diffusers import AnyFlowFARPipeline from diffusers.utils import export_to_video pipe = AnyFlowFARPipeline.from_pretrained( - "nvidia/AnyFlow-FAR-Wan2.1-1.3B-Diffusers", torch_dtype=torch.bfloat16 + "nvidia/AnyFlow-FAR-Wan2.1-1.3B-Diffusers", dtype=torch.bfloat16 ).to("cuda") prompt = ( @@ -105,7 +105,7 @@ from diffusers import AnyFlowFARPipeline from diffusers.utils import export_to_video, load_image pipe = AnyFlowFARPipeline.from_pretrained( - "nvidia/AnyFlow-FAR-Wan2.1-1.3B-Diffusers", torch_dtype=torch.bfloat16 + "nvidia/AnyFlow-FAR-Wan2.1-1.3B-Diffusers", dtype=torch.bfloat16 ).to("cuda") # Example conditioning image from the AnyFlow repo. @@ -139,7 +139,7 @@ from diffusers import AnyFlowFARPipeline from diffusers.utils import export_to_video, load_video pipe = AnyFlowFARPipeline.from_pretrained( - "nvidia/AnyFlow-FAR-Wan2.1-1.3B-Diffusers", torch_dtype=torch.bfloat16 + "nvidia/AnyFlow-FAR-Wan2.1-1.3B-Diffusers", dtype=torch.bfloat16 ).to("cuda") # Example conditioning clip from the AnyFlow repo — take the first 9 frames (3 latent frames at VAE temporal stride 4). diff --git a/docs/source/en/api/pipelines/aura_flow.md b/docs/source/en/api/pipelines/aura_flow.md index 67951859b962..7f88d28c8b40 100644 --- a/docs/source/en/api/pipelines/aura_flow.md +++ b/docs/source/en/api/pipelines/aura_flow.md @@ -35,7 +35,7 @@ text_encoder_8bit = T5EncoderModel.from_pretrained( "fal/AuraFlow", subfolder="text_encoder", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) quant_config = DiffusersBitsAndBytesConfig(load_in_8bit=True) @@ -43,14 +43,14 @@ transformer_8bit = AuraFlowTransformer2DModel.from_pretrained( "fal/AuraFlow", subfolder="transformer", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) pipeline = AuraFlowPipeline.from_pretrained( "fal/AuraFlow", text_encoder=text_encoder_8bit, transformer=transformer_8bit, - torch_dtype=torch.float16, + dtype=torch.float16, device_map="balanced", ) @@ -72,13 +72,13 @@ from diffusers import ( transformer = AuraFlowTransformer2DModel.from_single_file( "https://huggingface.co/city96/AuraFlow-v0.3-gguf/blob/main/aura_flow_0.3-Q2_K.gguf", quantization_config=GGUFQuantizationConfig(compute_dtype=torch.bfloat16), - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) pipeline = AuraFlowPipeline.from_pretrained( "fal/AuraFlow-v0.3", transformer=transformer, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) prompt = "a cute pony in a field of flowers" diff --git a/docs/source/en/api/pipelines/chroma.md b/docs/source/en/api/pipelines/chroma.md index 2b3b50c25e80..9ca1ff4dee68 100644 --- a/docs/source/en/api/pipelines/chroma.md +++ b/docs/source/en/api/pipelines/chroma.md @@ -33,7 +33,7 @@ Original model checkpoints for Chroma can be found here: import torch from diffusers import ChromaPipeline -pipe = ChromaPipeline.from_pretrained("lodestones/Chroma1-HD", torch_dtype=torch.bfloat16) +pipe = ChromaPipeline.from_pretrained("lodestones/Chroma1-HD", dtype=torch.bfloat16) pipe.enable_model_cpu_offload() prompt = [ @@ -67,9 +67,9 @@ from diffusers import ChromaTransformer2DModel, ChromaPipeline model_id = "lodestones/Chroma1-HD" dtype = torch.bfloat16 -transformer = ChromaTransformer2DModel.from_single_file("https://huggingface.co/lodestones/Chroma1-HD/blob/main/Chroma1-HD.safetensors", torch_dtype=dtype) +transformer = ChromaTransformer2DModel.from_single_file("https://huggingface.co/lodestones/Chroma1-HD/blob/main/Chroma1-HD.safetensors", dtype=dtype) -pipe = ChromaPipeline.from_pretrained(model_id, transformer=transformer, torch_dtype=dtype) +pipe = ChromaPipeline.from_pretrained(model_id, transformer=transformer, dtype=dtype) pipe.enable_model_cpu_offload() prompt = [ diff --git a/docs/source/en/api/pipelines/chronoedit.md b/docs/source/en/api/pipelines/chronoedit.md index 5e7057f9ccb8..2305b5799d5e 100644 --- a/docs/source/en/api/pipelines/chronoedit.md +++ b/docs/source/en/api/pipelines/chronoedit.md @@ -46,10 +46,10 @@ from transformers import CLIPVisionModel from PIL import Image model_id = "nvidia/ChronoEdit-14B-Diffusers" -image_encoder = CLIPVisionModel.from_pretrained(model_id, subfolder="image_encoder", torch_dtype=torch.float32) -vae = AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", torch_dtype=torch.float32) -transformer = ChronoEditTransformer3DModel.from_pretrained(model_id, subfolder="transformer", torch_dtype=torch.bfloat16) -pipe = ChronoEditPipeline.from_pretrained(model_id, image_encoder=image_encoder, transformer=transformer, vae=vae, torch_dtype=torch.bfloat16) +image_encoder = CLIPVisionModel.from_pretrained(model_id, subfolder="image_encoder", dtype=torch.float32) +vae = AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", dtype=torch.float32) +transformer = ChronoEditTransformer3DModel.from_pretrained(model_id, subfolder="transformer", dtype=torch.bfloat16) +pipe = ChronoEditPipeline.from_pretrained(model_id, image_encoder=image_encoder, transformer=transformer, vae=vae, dtype=torch.bfloat16) pipe.to("cuda") image = load_image( @@ -110,10 +110,10 @@ from transformers import CLIPVisionModel from PIL import Image model_id = "nvidia/ChronoEdit-14B-Diffusers" -image_encoder = CLIPVisionModel.from_pretrained(model_id, subfolder="image_encoder", torch_dtype=torch.float32) -vae = AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", torch_dtype=torch.float32) -transformer = ChronoEditTransformer3DModel.from_pretrained(model_id, subfolder="transformer", torch_dtype=torch.bfloat16) -pipe = ChronoEditPipeline.from_pretrained(model_id, image_encoder=image_encoder, transformer=transformer, vae=vae, torch_dtype=torch.bfloat16) +image_encoder = CLIPVisionModel.from_pretrained(model_id, subfolder="image_encoder", dtype=torch.float32) +vae = AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", dtype=torch.float32) +transformer = ChronoEditTransformer3DModel.from_pretrained(model_id, subfolder="transformer", dtype=torch.bfloat16) +pipe = ChronoEditPipeline.from_pretrained(model_id, image_encoder=image_encoder, transformer=transformer, vae=vae, dtype=torch.bfloat16) pipe.load_lora_weights("nvidia/ChronoEdit-14B-Diffusers", weight_name="lora/chronoedit_distill_lora.safetensors", adapter_name="distill") pipe.fuse_lora(adapter_names=["distill"], lora_scale=1.0) pipe.scheduler = UniPCMultistepScheduler.from_config(pipe.scheduler.config, flow_shift=2.0) @@ -161,10 +161,10 @@ from transformers import CLIPVisionModel from PIL import Image model_id = "nvidia/ChronoEdit-14B-Diffusers" -image_encoder = CLIPVisionModel.from_pretrained(model_id, subfolder="image_encoder", torch_dtype=torch.float32) -vae = AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", torch_dtype=torch.float32) -transformer = ChronoEditTransformer3DModel.from_pretrained(model_id, subfolder="transformer", torch_dtype=torch.bfloat16) -pipe = ChronoEditPipeline.from_pretrained(model_id, image_encoder=image_encoder, transformer=transformer, vae=vae, torch_dtype=torch.bfloat16) +image_encoder = CLIPVisionModel.from_pretrained(model_id, subfolder="image_encoder", dtype=torch.float32) +vae = AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", dtype=torch.float32) +transformer = ChronoEditTransformer3DModel.from_pretrained(model_id, subfolder="transformer", dtype=torch.bfloat16) +pipe = ChronoEditPipeline.from_pretrained(model_id, image_encoder=image_encoder, transformer=transformer, vae=vae, dtype=torch.bfloat16) pipe.load_lora_weights("nvidia/ChronoEdit-14B-Diffusers-Paint-Brush-Lora", weight_name="paintbrush_lora_diffusers.safetensors", adapter_name="paintbrush") pipe.load_lora_weights("nvidia/ChronoEdit-14B-Diffusers", weight_name="lora/chronoedit_distill_lora.safetensors", adapter_name="distill") pipe.fuse_lora(adapter_names=["paintbrush", "distill"], lora_scale=1.0) diff --git a/docs/source/en/api/pipelines/cogvideox.md b/docs/source/en/api/pipelines/cogvideox.md index b296bbe25582..09c7405bf0aa 100644 --- a/docs/source/en/api/pipelines/cogvideox.md +++ b/docs/source/en/api/pipelines/cogvideox.md @@ -56,7 +56,7 @@ pipeline_quant_config = PipelineQuantizationConfig( transformer = AutoModel.from_pretrained( "THUDM/CogVideoX-5b", subfolder="transformer", - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) transformer.enable_layerwise_casting( storage_dtype=torch.float8_e4m3fn, compute_dtype=torch.bfloat16 @@ -66,7 +66,7 @@ pipeline = CogVideoXPipeline.from_pretrained( "THUDM/CogVideoX-5b", transformer=transformer, quantization_config=pipeline_quant_config, - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) pipeline.to("cuda") @@ -102,7 +102,7 @@ from diffusers.utils import export_to_video pipeline = CogVideoXPipeline.from_pretrained( "THUDM/CogVideoX-2b", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") # torch.compile @@ -144,7 +144,7 @@ export_to_video(video, "output.mp4", fps=8) pipeline = CogVideoXPipeline.from_pretrained( "THUDM/CogVideoX-5b", - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) pipeline.to("cuda") diff --git a/docs/source/en/api/pipelines/consisid.md b/docs/source/en/api/pipelines/consisid.md index 6ef336d7c8e5..1889e16f099f 100644 --- a/docs/source/en/api/pipelines/consisid.md +++ b/docs/source/en/api/pipelines/consisid.md @@ -67,7 +67,7 @@ snapshot_download(repo_id="BestWishYsh/ConsisID-preview", local_dir="BestWishYsh face_helper_1, face_helper_2, face_clip_model, face_main_model, eva_transform_mean, eva_transform_std = prepare_face_models("BestWishYsh/ConsisID-preview", device="cuda", dtype=torch.bfloat16) # Load consisid base model -pipe = ConsisIDPipeline.from_pretrained("BestWishYsh/ConsisID-preview", torch_dtype=torch.bfloat16) +pipe = ConsisIDPipeline.from_pretrained("BestWishYsh/ConsisID-preview", dtype=torch.bfloat16) pipe.to("cuda") ``` diff --git a/docs/source/en/api/pipelines/consistency_models.md b/docs/source/en/api/pipelines/consistency_models.md index 4f7b2f0fb501..e09f6fc5bc5d 100644 --- a/docs/source/en/api/pipelines/consistency_models.md +++ b/docs/source/en/api/pipelines/consistency_models.md @@ -33,7 +33,7 @@ For an additional speed-up, use `torch.compile` to generate multiple images in < device = "cuda" # Load the cd_bedroom256_lpips checkpoint. model_id_or_path = "openai/diffusers-cd_bedroom256_lpips" - pipe = ConsistencyModelPipeline.from_pretrained(model_id_or_path, torch_dtype=torch.float16) + pipe = ConsistencyModelPipeline.from_pretrained(model_id_or_path, dtype=torch.float16) pipe.to(device) + pipe.unet = torch.compile(pipe.unet, mode="reduce-overhead", fullgraph=True) diff --git a/docs/source/en/api/pipelines/control_flux_inpaint.md b/docs/source/en/api/pipelines/control_flux_inpaint.md index 4b087f20efcd..a0105c3a941b 100644 --- a/docs/source/en/api/pipelines/control_flux_inpaint.md +++ b/docs/source/en/api/pipelines/control_flux_inpaint.md @@ -41,15 +41,15 @@ import numpy as np pipe = FluxControlInpaintPipeline.from_pretrained( "black-forest-labs/FLUX.1-Depth-dev", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) # use following lines if you have GPU constraints # --------------------------------------------------------------- transformer = FluxTransformer2DModel.from_pretrained( - "sayakpaul/FLUX.1-Depth-dev-nf4", subfolder="transformer", torch_dtype=torch.bfloat16 + "sayakpaul/FLUX.1-Depth-dev-nf4", subfolder="transformer", dtype=torch.bfloat16 ) text_encoder_2 = T5EncoderModel.from_pretrained( - "sayakpaul/FLUX.1-Depth-dev-nf4", subfolder="text_encoder_2", torch_dtype=torch.bfloat16 + "sayakpaul/FLUX.1-Depth-dev-nf4", subfolder="text_encoder_2", dtype=torch.bfloat16 ) pipe.transformer = transformer pipe.text_encoder_2 = text_encoder_2 diff --git a/docs/source/en/api/pipelines/cosmos.md b/docs/source/en/api/pipelines/cosmos.md index d4851997b9ce..4b60ae28e45f 100644 --- a/docs/source/en/api/pipelines/cosmos.md +++ b/docs/source/en/api/pipelines/cosmos.md @@ -30,7 +30,7 @@ from diffusers.utils import export_to_video model_id = "nvidia/Cosmos-Predict2.5-2B" pipe = Cosmos2_5_PredictBasePipeline.from_pretrained( - model_id, revision="diffusers/base/post-trained", torch_dtype=torch.bfloat16 + model_id, revision="diffusers/base/post-trained", dtype=torch.bfloat16 ) pipe.to("cuda") diff --git a/docs/source/en/api/pipelines/cosmos3.md b/docs/source/en/api/pipelines/cosmos3.md index 271b6ae412d5..b4acfb7f528a 100644 --- a/docs/source/en/api/pipelines/cosmos3.md +++ b/docs/source/en/api/pipelines/cosmos3.md @@ -100,7 +100,7 @@ json_prompt = json.load(open("assets/example_t2v_prompt.json")) negative_prompt = json.load(open("assets/negative_prompt.json")) pipe = Cosmos3OmniPipeline.from_pretrained( - "nvidia/Cosmos3-Nano", torch_dtype=torch.bfloat16, device_map="cuda" + "nvidia/Cosmos3-Nano", dtype=torch.bfloat16, device_map="cuda" ) pipe.scheduler = UniPCMultistepScheduler.from_config( pipe.scheduler.config, flow_shift=10.0, use_karras_sigmas=False @@ -135,7 +135,7 @@ json_prompt = json.load(open("assets/example_t2v_prompt.json")) negative_prompt = json.load(open("assets/negative_prompt.json")) pipe = Cosmos3OmniPipeline.from_pretrained( - "nvidia/Cosmos3-Super", torch_dtype=torch.bfloat16, device_map="cuda" + "nvidia/Cosmos3-Super", dtype=torch.bfloat16, device_map="cuda" ) pipe.scheduler = UniPCMultistepScheduler.from_config( pipe.scheduler.config, flow_shift=10.0, use_karras_sigmas=False @@ -174,7 +174,7 @@ from diffusers import Cosmos3OmniPipeline json_prompt = json.load(open("assets/example_t2i_prompt.json")) pipe = Cosmos3OmniPipeline.from_pretrained( - "nvidia/Cosmos3-Nano", torch_dtype=torch.bfloat16, device_map="cuda" + "nvidia/Cosmos3-Nano", dtype=torch.bfloat16, device_map="cuda" ) result = pipe(prompt=json.dumps(json_prompt), num_frames=1, height=720, width=1280) @@ -193,7 +193,7 @@ from diffusers import Cosmos3OmniPipeline json_prompt = json.load(open("assets/example_t2i_prompt.json")) pipe = Cosmos3OmniPipeline.from_pretrained( - "nvidia/Cosmos3-Super", torch_dtype=torch.bfloat16, device_map="cuda" + "nvidia/Cosmos3-Super", dtype=torch.bfloat16, device_map="cuda" ) result = pipe(prompt=json.dumps(json_prompt), num_frames=1, height=720, width=1280) @@ -221,7 +221,7 @@ json_prompt = json.load(open("assets/example_i2v_prompt.json")) negative_prompt = json.load(open("assets/negative_prompt_i2v.json")) pipe = Cosmos3OmniPipeline.from_pretrained( - "nvidia/Cosmos3-Nano", torch_dtype=torch.bfloat16, device_map="cuda" + "nvidia/Cosmos3-Nano", dtype=torch.bfloat16, device_map="cuda" ) image = load_image( @@ -255,7 +255,7 @@ json_prompt = json.load(open("assets/example_i2v_prompt.json")) negative_prompt = json.load(open("assets/negative_prompt_i2v.json")) pipe = Cosmos3OmniPipeline.from_pretrained( - "nvidia/Cosmos3-Super", torch_dtype=torch.bfloat16, device_map="cuda" + "nvidia/Cosmos3-Super", dtype=torch.bfloat16, device_map="cuda" ) image = load_image( @@ -299,7 +299,7 @@ json_prompt = json.load(open("assets/example_v2v_prompt.json")) negative_prompt = json.load(open("assets/negative_prompt_i2v.json")) pipe = Cosmos3OmniPipeline.from_pretrained( - "nvidia/Cosmos3-Nano", torch_dtype=torch.bfloat16, device_map="cuda" + "nvidia/Cosmos3-Nano", dtype=torch.bfloat16, device_map="cuda" ) pipe.scheduler = UniPCMultistepScheduler.from_config( pipe.scheduler.config, flow_shift=10.0, use_karras_sigmas=False @@ -341,7 +341,7 @@ json_prompt = json.load(open("assets/example_v2v_prompt.json")) negative_prompt = json.load(open("assets/negative_prompt_i2v.json")) pipe = Cosmos3OmniPipeline.from_pretrained( - "nvidia/Cosmos3-Super", torch_dtype=torch.bfloat16, device_map="cuda" + "nvidia/Cosmos3-Super", dtype=torch.bfloat16, device_map="cuda" ) pipe.scheduler = UniPCMultistepScheduler.from_config( pipe.scheduler.config, flow_shift=10.0, use_karras_sigmas=False @@ -390,7 +390,7 @@ json_prompt = json.load(open("assets/example_v2v_prompt.json")) negative_prompt = json.load(open("assets/negative_prompt_i2v.json")) pipe = Cosmos3OmniPipeline.from_pretrained( - "nvidia/Cosmos3-Nano", torch_dtype=torch.bfloat16, device_map="cuda" + "nvidia/Cosmos3-Nano", dtype=torch.bfloat16, device_map="cuda" ) pipe.scheduler = UniPCMultistepScheduler.from_config( pipe.scheduler.config, flow_shift=10.0, use_karras_sigmas=False @@ -437,7 +437,7 @@ json_prompt = json.load(open("assets/example_v2v_prompt.json")) negative_prompt = json.load(open("assets/negative_prompt_i2v.json")) pipe = Cosmos3OmniPipeline.from_pretrained( - "nvidia/Cosmos3-Super", torch_dtype=torch.bfloat16, device_map="cuda" + "nvidia/Cosmos3-Super", dtype=torch.bfloat16, device_map="cuda" ) pipe.scheduler = UniPCMultistepScheduler.from_config( pipe.scheduler.config, flow_shift=10.0, use_karras_sigmas=False @@ -492,7 +492,7 @@ json_prompt = json.load(open("assets/example_t2v_sound_prompt.json")) negative_prompt = json.load(open("assets/negative_prompt.json")) pipe = Cosmos3OmniPipeline.from_pretrained( - "nvidia/Cosmos3-Nano", torch_dtype=torch.bfloat16, device_map="cuda" + "nvidia/Cosmos3-Nano", dtype=torch.bfloat16, device_map="cuda" ) result = pipe( @@ -528,7 +528,7 @@ json_prompt = json.load(open("assets/example_t2v_sound_prompt.json")) negative_prompt = json.load(open("assets/negative_prompt.json")) pipe = Cosmos3OmniPipeline.from_pretrained( - "nvidia/Cosmos3-Super", torch_dtype=torch.bfloat16, device_map="cuda" + "nvidia/Cosmos3-Super", dtype=torch.bfloat16, device_map="cuda" ) result = pipe( @@ -575,7 +575,7 @@ from diffusers.schedulers.scheduling_unipc_multistep import UniPCMultistepSchedu from diffusers.utils import export_to_video, load_video pipe = Cosmos3OmniPipeline.from_pretrained( - "nvidia/Cosmos3-Nano", torch_dtype=torch.bfloat16, device_map="cuda" + "nvidia/Cosmos3-Nano", dtype=torch.bfloat16, device_map="cuda" ) pipe.scheduler = UniPCMultistepScheduler.from_config( pipe.scheduler.config, flow_shift=10.0, use_karras_sigmas=False @@ -622,7 +622,7 @@ from diffusers.schedulers.scheduling_unipc_multistep import UniPCMultistepSchedu from diffusers.utils import export_to_video, load_video pipe = Cosmos3OmniPipeline.from_pretrained( - "nvidia/Cosmos3-Super", torch_dtype=torch.bfloat16, device_map="cuda" + "nvidia/Cosmos3-Super", dtype=torch.bfloat16, device_map="cuda" ) pipe.scheduler = UniPCMultistepScheduler.from_config( pipe.scheduler.config, flow_shift=10.0, use_karras_sigmas=False @@ -739,7 +739,7 @@ mesh = init_device_mesh("cuda", (tp_degree, cp_degree), mesh_dim_names=("tp", "c # Load components on CPU first; a TP-sharded model may not fit one GPU. pipe = Cosmos3OmniModularPipeline.from_pretrained(model_id) -pipe.load_components(torch_dtype=torch.bfloat16) +pipe.load_components(dtype=torch.bfloat16) pipe.enable_safety_checker() if tp_degree > 1: @@ -820,7 +820,7 @@ from diffusers import Cosmos3OmniPipeline pipe = Cosmos3OmniPipeline.from_pretrained( "nvidia/Cosmos3-Nano", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, device_map="cuda", enable_safety_checker=False, ) @@ -844,7 +844,7 @@ To supply a custom checker (e.g., a no-op subclass for fast tests), pass it as ` ```python pipe = Cosmos3OmniPipeline.from_pretrained( "nvidia/Cosmos3-Nano", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, device_map="cuda", safety_checker=MyCustomSafetyChecker(), ) @@ -866,9 +866,9 @@ import torch from diffusers import Cosmos3OmniModularPipeline pipe = Cosmos3OmniModularPipeline.from_pretrained( - "nvidia/Cosmos3-Nano", torch_dtype=torch.bfloat16 + "nvidia/Cosmos3-Nano", dtype=torch.bfloat16 ) -pipe.load_components(torch_dtype=torch.bfloat16) +pipe.load_components(dtype=torch.bfloat16) pipe.enable_safety_checker() videos = pipe( @@ -890,8 +890,8 @@ You can also load through [`ModularPipeline`] and let the repository config sele import torch from diffusers import ModularPipeline -pipe = ModularPipeline.from_pretrained("nvidia/Cosmos3-Nano", torch_dtype=torch.bfloat16) -pipe.load_components(torch_dtype=torch.bfloat16) +pipe = ModularPipeline.from_pretrained("nvidia/Cosmos3-Nano", dtype=torch.bfloat16) +pipe.load_components(dtype=torch.bfloat16) pipe.enable_safety_checker() videos = pipe( prompt='{"scene":"A robot arm in a kitchen"}', num_frames=1, height=720, width=1280, output="videos" @@ -916,8 +916,8 @@ from diffusers import Cosmos3OmniModularPipeline, CosmosActionCondition from diffusers.schedulers.scheduling_unipc_multistep import UniPCMultistepScheduler from diffusers.utils import encode_video, export_to_video, load_image, load_video -pipe = Cosmos3OmniModularPipeline.from_pretrained("nvidia/Cosmos3-Nano", torch_dtype=torch.bfloat16) -pipe.load_components(torch_dtype=torch.bfloat16) +pipe = Cosmos3OmniModularPipeline.from_pretrained("nvidia/Cosmos3-Nano", dtype=torch.bfloat16) +pipe.load_components(dtype=torch.bfloat16) pipe.enable_safety_checker() pipe.to("cuda") pipe.scheduler = UniPCMultistepScheduler.from_config( @@ -1080,8 +1080,8 @@ from diffusers import Cosmos3OmniModularPipeline from diffusers.schedulers.scheduling_unipc_multistep import UniPCMultistepScheduler from diffusers.utils import export_to_video, load_video -pipe = Cosmos3OmniModularPipeline.from_pretrained("nvidia/Cosmos3-Nano", torch_dtype=torch.bfloat16) -pipe.load_components(torch_dtype=torch.bfloat16) +pipe = Cosmos3OmniModularPipeline.from_pretrained("nvidia/Cosmos3-Nano", dtype=torch.bfloat16) +pipe.load_components(dtype=torch.bfloat16) pipe.to("cuda") pipe.scheduler = UniPCMultistepScheduler.from_config( pipe.scheduler.config, flow_shift=10.0, use_karras_sigmas=False @@ -1131,8 +1131,8 @@ from diffusers.utils import export_to_video, load_image json_prompt = json.load(open("assets/example_t2i_prompt.json")) repo = "nvidia/Cosmos3-Super-Text2Image-4Step" -pipe = Cosmos3DistilledModularPipeline.from_pretrained(repo, torch_dtype=torch.bfloat16) -pipe.load_components(torch_dtype=torch.bfloat16) +pipe = Cosmos3DistilledModularPipeline.from_pretrained(repo, dtype=torch.bfloat16) +pipe.load_components(dtype=torch.bfloat16) pipe.to("cuda") # text-to-image (distilled) @@ -1151,8 +1151,8 @@ videos[0].save("cosmos3_distilled_t2i.jpg", format="JPEG", quality=85) json_prompt_i2v = json.load(open("assets/example_i2v_prompt.json")) repo_i2v = "nvidia/Cosmos3-Super-Image2Video-4Step" -pipe = Cosmos3DistilledModularPipeline.from_pretrained(repo_i2v, torch_dtype=torch.bfloat16) -pipe.load_components(torch_dtype=torch.bfloat16) +pipe = Cosmos3DistilledModularPipeline.from_pretrained(repo_i2v, dtype=torch.bfloat16) +pipe.load_components(dtype=torch.bfloat16) pipe.to("cuda") image = load_image( diff --git a/docs/source/en/api/pipelines/deepfloyd_if.md b/docs/source/en/api/pipelines/deepfloyd_if.md index d631aac72eb3..c6e336385352 100644 --- a/docs/source/en/api/pipelines/deepfloyd_if.md +++ b/docs/source/en/api/pipelines/deepfloyd_if.md @@ -91,12 +91,12 @@ from diffusers.utils import pt_to_pil, make_image_grid import torch # stage 1 -stage_1 = DiffusionPipeline.from_pretrained("DeepFloyd/IF-I-XL-v1.0", variant="fp16", torch_dtype=torch.float16) +stage_1 = DiffusionPipeline.from_pretrained("DeepFloyd/IF-I-XL-v1.0", variant="fp16", dtype=torch.float16) stage_1.enable_model_cpu_offload() # stage 2 stage_2 = DiffusionPipeline.from_pretrained( - "DeepFloyd/IF-II-L-v1.0", text_encoder=None, variant="fp16", torch_dtype=torch.float16 + "DeepFloyd/IF-II-L-v1.0", text_encoder=None, variant="fp16", dtype=torch.float16 ) stage_2.enable_model_cpu_offload() @@ -107,7 +107,7 @@ safety_modules = { "watermarker": stage_1.watermarker, } stage_3 = DiffusionPipeline.from_pretrained( - "stabilityai/stable-diffusion-x4-upscaler", **safety_modules, torch_dtype=torch.float16 + "stabilityai/stable-diffusion-x4-upscaler", **safety_modules, dtype=torch.float16 ) stage_3.enable_model_cpu_offload() @@ -158,12 +158,12 @@ original_image = load_image(url) original_image = original_image.resize((768, 512)) # stage 1 -stage_1 = IFImg2ImgPipeline.from_pretrained("DeepFloyd/IF-I-XL-v1.0", variant="fp16", torch_dtype=torch.float16) +stage_1 = IFImg2ImgPipeline.from_pretrained("DeepFloyd/IF-I-XL-v1.0", variant="fp16", dtype=torch.float16) stage_1.enable_model_cpu_offload() # stage 2 stage_2 = IFImg2ImgSuperResolutionPipeline.from_pretrained( - "DeepFloyd/IF-II-L-v1.0", text_encoder=None, variant="fp16", torch_dtype=torch.float16 + "DeepFloyd/IF-II-L-v1.0", text_encoder=None, variant="fp16", dtype=torch.float16 ) stage_2.enable_model_cpu_offload() @@ -174,7 +174,7 @@ safety_modules = { "watermarker": stage_1.watermarker, } stage_3 = DiffusionPipeline.from_pretrained( - "stabilityai/stable-diffusion-x4-upscaler", **safety_modules, torch_dtype=torch.float16 + "stabilityai/stable-diffusion-x4-upscaler", **safety_modules, dtype=torch.float16 ) stage_3.enable_model_cpu_offload() @@ -233,12 +233,12 @@ url = "https://huggingface.co/datasets/diffusers/docs-images/resolve/main/if/gla mask_image = load_image(url) # stage 1 -stage_1 = IFInpaintingPipeline.from_pretrained("DeepFloyd/IF-I-XL-v1.0", variant="fp16", torch_dtype=torch.float16) +stage_1 = IFInpaintingPipeline.from_pretrained("DeepFloyd/IF-I-XL-v1.0", variant="fp16", dtype=torch.float16) stage_1.enable_model_cpu_offload() # stage 2 stage_2 = IFInpaintingSuperResolutionPipeline.from_pretrained( - "DeepFloyd/IF-II-L-v1.0", text_encoder=None, variant="fp16", torch_dtype=torch.float16 + "DeepFloyd/IF-II-L-v1.0", text_encoder=None, variant="fp16", dtype=torch.float16 ) stage_2.enable_model_cpu_offload() @@ -249,7 +249,7 @@ safety_modules = { "watermarker": stage_1.watermarker, } stage_3 = DiffusionPipeline.from_pretrained( - "stabilityai/stable-diffusion-x4-upscaler", **safety_modules, torch_dtype=torch.float16 + "stabilityai/stable-diffusion-x4-upscaler", **safety_modules, dtype=torch.float16 ) stage_3.enable_model_cpu_offload() @@ -316,7 +316,7 @@ pipe_2 = IFInpaintingSuperResolutionPipeline(**pipe_2.components) The simplest optimization to run IF faster is to move all model components to the GPU. ```py -pipe = DiffusionPipeline.from_pretrained("DeepFloyd/IF-I-XL-v1.0", variant="fp16", torch_dtype=torch.float16) +pipe = DiffusionPipeline.from_pretrained("DeepFloyd/IF-I-XL-v1.0", variant="fp16", dtype=torch.float16) pipe.to("cuda") ``` @@ -341,7 +341,7 @@ with the strength argument. The strength argument is the amount of noise to add A smaller number will vary the image less but run faster. ```py -pipe = IFImg2ImgPipeline.from_pretrained("DeepFloyd/IF-I-XL-v1.0", variant="fp16", torch_dtype=torch.float16) +pipe = IFImg2ImgPipeline.from_pretrained("DeepFloyd/IF-I-XL-v1.0", variant="fp16", dtype=torch.float16) pipe.to("cuda") image = pipe(image=image, prompt="", strength=0.3).images @@ -354,7 +354,7 @@ with IF and it might not give expected results. from diffusers import DiffusionPipeline import torch -pipe = DiffusionPipeline.from_pretrained("DeepFloyd/IF-I-XL-v1.0", variant="fp16", torch_dtype=torch.float16) +pipe = DiffusionPipeline.from_pretrained("DeepFloyd/IF-I-XL-v1.0", variant="fp16", dtype=torch.float16) pipe.to("cuda") pipe.text_encoder = torch.compile(pipe.text_encoder, mode="reduce-overhead", fullgraph=True) @@ -368,14 +368,14 @@ When optimizing for GPU memory, we can use the standard diffusers CPU offloading Either the model based CPU offloading, ```py -pipe = DiffusionPipeline.from_pretrained("DeepFloyd/IF-I-XL-v1.0", variant="fp16", torch_dtype=torch.float16) +pipe = DiffusionPipeline.from_pretrained("DeepFloyd/IF-I-XL-v1.0", variant="fp16", dtype=torch.float16) pipe.enable_model_cpu_offload() ``` or the more aggressive layer based CPU offloading. ```py -pipe = DiffusionPipeline.from_pretrained("DeepFloyd/IF-I-XL-v1.0", variant="fp16", torch_dtype=torch.float16) +pipe = DiffusionPipeline.from_pretrained("DeepFloyd/IF-I-XL-v1.0", variant="fp16", dtype=torch.float16) pipe.enable_sequential_cpu_offload() ``` @@ -432,7 +432,7 @@ gc.collect() torch.cuda.empty_cache() pipe = IFPipeline.from_pretrained( - "DeepFloyd/IF-I-XL-v1.0", text_encoder=None, variant="fp16", torch_dtype=torch.float16, device_map="auto" + "DeepFloyd/IF-I-XL-v1.0", text_encoder=None, variant="fp16", dtype=torch.float16, device_map="auto" ) generator = torch.Generator().manual_seed(0) @@ -453,7 +453,7 @@ torch.cuda.empty_cache() # First super resolution pipe = IFSuperResolutionPipeline.from_pretrained( - "DeepFloyd/IF-II-L-v1.0", text_encoder=None, variant="fp16", torch_dtype=torch.float16, device_map="auto" + "DeepFloyd/IF-II-L-v1.0", text_encoder=None, variant="fp16", dtype=torch.float16, device_map="auto" ) generator = torch.Generator().manual_seed(0) diff --git a/docs/source/en/api/pipelines/dreamlite.md b/docs/source/en/api/pipelines/dreamlite.md index 3e5b5bd4da93..37cb298e17dc 100644 --- a/docs/source/en/api/pipelines/dreamlite.md +++ b/docs/source/en/api/pipelines/dreamlite.md @@ -44,7 +44,7 @@ Official checkpoints: import torch from diffusers import DreamLitePipeline -pipe = DreamLitePipeline.from_pretrained("carlofkl/DreamLite-base", revision="diffusers", torch_dtype=torch.bfloat16) +pipe = DreamLitePipeline.from_pretrained("carlofkl/DreamLite-base", revision="diffusers", dtype=torch.bfloat16) pipe = pipe.to("cuda") image = pipe( @@ -68,7 +68,7 @@ import torch from diffusers import DreamLitePipeline from diffusers.utils import load_image -pipe = DreamLitePipeline.from_pretrained("carlofkl/DreamLite-base", revision="diffusers", torch_dtype=torch.bfloat16) +pipe = DreamLitePipeline.from_pretrained("carlofkl/DreamLite-base", revision="diffusers", dtype=torch.bfloat16) pipe = pipe.to("cuda") source = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png") @@ -94,7 +94,7 @@ same `prompt` / `height` / `width` / `num_inference_steps` arguments, but **igno import torch from diffusers import DreamLiteMobilePipeline -pipe = DreamLiteMobilePipeline.from_pretrained("carlofkl/DreamLite-mobile", revision="diffusers", torch_dtype=torch.bfloat16) +pipe = DreamLiteMobilePipeline.from_pretrained("carlofkl/DreamLite-mobile", revision="diffusers", dtype=torch.bfloat16) pipe = pipe.to("cuda") image = pipe( @@ -114,7 +114,7 @@ import torch from diffusers import DreamLiteMobilePipeline from diffusers.utils import load_image -pipe = DreamLiteMobilePipeline.from_pretrained("carlofkl/DreamLite-mobile", revision="diffusers", torch_dtype=torch.bfloat16) +pipe = DreamLiteMobilePipeline.from_pretrained("carlofkl/DreamLite-mobile", revision="diffusers", dtype=torch.bfloat16) pipe = pipe.to("cuda") source = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png") diff --git a/docs/source/en/api/pipelines/easyanimate.md b/docs/source/en/api/pipelines/easyanimate.md index 15d44a12b1b6..c03878d20f44 100644 --- a/docs/source/en/api/pipelines/easyanimate.md +++ b/docs/source/en/api/pipelines/easyanimate.md @@ -61,13 +61,13 @@ transformer_8bit = EasyAnimateTransformer3DModel.from_pretrained( "alibaba-pai/EasyAnimateV5.1-12b-zh", subfolder="transformer", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) pipeline = EasyAnimatePipeline.from_pretrained( "alibaba-pai/EasyAnimateV5.1-12b-zh", transformer=transformer_8bit, - torch_dtype=torch.float16, + dtype=torch.float16, device_map="balanced", ) diff --git a/docs/source/en/api/pipelines/ernie_image.md b/docs/source/en/api/pipelines/ernie_image.md index 79f35cf93a2e..02fe166fab87 100644 --- a/docs/source/en/api/pipelines/ernie_image.md +++ b/docs/source/en/api/pipelines/ernie_image.md @@ -44,7 +44,7 @@ import torch from diffusers import ErnieImagePipeline from diffusers.utils import load_image -pipe = ErnieImagePipeline.from_pretrained("baidu/ERNIE-Image", torch_dtype=torch.bfloat16) +pipe = ErnieImagePipeline.from_pretrained("baidu/ERNIE-Image", dtype=torch.bfloat16) pipe.to("cuda") # If you are running low on GPU VRAM, you can enable offloading pipe.enable_model_cpu_offload() @@ -67,7 +67,7 @@ import torch from diffusers import ErnieImagePipeline from diffusers.utils import load_image -pipe = ErnieImagePipeline.from_pretrained("baidu/ERNIE-Image-Turbo", torch_dtype=torch.bfloat16) +pipe = ErnieImagePipeline.from_pretrained("baidu/ERNIE-Image-Turbo", dtype=torch.bfloat16) pipe.to("cuda") # If you are running low on GPU VRAM, you can enable offloading pipe.enable_model_cpu_offload() diff --git a/docs/source/en/api/pipelines/flux.md b/docs/source/en/api/pipelines/flux.md index 358b8139c73a..9e5c1df7a373 100644 --- a/docs/source/en/api/pipelines/flux.md +++ b/docs/source/en/api/pipelines/flux.md @@ -52,7 +52,7 @@ All checkpoints have different usage which we detail below. import torch from diffusers import FluxPipeline -pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-schnell", torch_dtype=torch.bfloat16) +pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-schnell", dtype=torch.bfloat16) pipe.enable_model_cpu_offload() prompt = "A cat holding a sign that says hello world" @@ -76,7 +76,7 @@ out.save("image.png") import torch from diffusers import FluxPipeline -pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16) +pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", dtype=torch.bfloat16) pipe.enable_model_cpu_offload() prompt = "a tiny astronaut hatching from an egg on the moon" @@ -104,7 +104,7 @@ image = load_image("https://huggingface.co/datasets/YiYiXu/testing-images/resolv mask = load_image("https://huggingface.co/datasets/YiYiXu/testing-images/resolve/main/cup_mask.png") repo_id = "black-forest-labs/FLUX.1-Fill-dev" -pipe = FluxFillPipeline.from_pretrained(repo_id, torch_dtype=torch.bfloat16).to("cuda") +pipe = FluxFillPipeline.from_pretrained(repo_id, dtype=torch.bfloat16).to("cuda") image = pipe( prompt="a white paper cup", @@ -129,7 +129,7 @@ from controlnet_aux import CannyDetector from diffusers import FluxControlPipeline from diffusers.utils import load_image -pipe = FluxControlPipeline.from_pretrained("black-forest-labs/FLUX.1-Canny-dev", torch_dtype=torch.bfloat16).to("cuda") +pipe = FluxControlPipeline.from_pretrained("black-forest-labs/FLUX.1-Canny-dev", dtype=torch.bfloat16).to("cuda") prompt = "A robot made of exotic candies and chocolates of different kinds. The background is filled with confetti and celebratory gifts." control_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/robot.png") @@ -157,7 +157,7 @@ from controlnet_aux import CannyDetector from diffusers import FluxControlPipeline from diffusers.utils import load_image -pipe = FluxControlPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16).to("cuda") +pipe = FluxControlPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", dtype=torch.bfloat16).to("cuda") pipe.load_lora_weights("black-forest-labs/FLUX.1-Canny-dev-lora") prompt = "A robot made of exotic candies and chocolates of different kinds. The background is filled with confetti and celebratory gifts." @@ -188,7 +188,7 @@ from diffusers import FluxControlPipeline, FluxTransformer2DModel from diffusers.utils import load_image from image_gen_aux import DepthPreprocessor -pipe = FluxControlPipeline.from_pretrained("black-forest-labs/FLUX.1-Depth-dev", torch_dtype=torch.bfloat16).to("cuda") +pipe = FluxControlPipeline.from_pretrained("black-forest-labs/FLUX.1-Depth-dev", dtype=torch.bfloat16).to("cuda") prompt = "A robot made of exotic candies and chocolates of different kinds. The background is filled with confetti and celebratory gifts." control_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/robot.png") @@ -217,7 +217,7 @@ from diffusers import FluxControlPipeline, FluxTransformer2DModel from diffusers.utils import load_image from image_gen_aux import DepthPreprocessor -pipe = FluxControlPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16).to("cuda") +pipe = FluxControlPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", dtype=torch.bfloat16).to("cuda") pipe.load_lora_weights("black-forest-labs/FLUX.1-Depth-dev-lora") prompt = "A robot made of exotic candies and chocolates of different kinds. The background is filled with confetti and celebratory gifts." @@ -254,12 +254,12 @@ dtype = torch.bfloat16 repo_redux = "black-forest-labs/FLUX.1-Redux-dev" repo_base = "black-forest-labs/FLUX.1-dev" -pipe_prior_redux = FluxPriorReduxPipeline.from_pretrained(repo_redux, torch_dtype=dtype).to(device) +pipe_prior_redux = FluxPriorReduxPipeline.from_pretrained(repo_redux, dtype=dtype).to(device) pipe = FluxPipeline.from_pretrained( repo_base, text_encoder=None, text_encoder_2=None, - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ).to(device) image = load_image("https://huggingface.co/datasets/YiYiXu/testing-images/resolve/main/style_ziggy/img5.png") @@ -283,7 +283,7 @@ from diffusers import FluxKontextPipeline from diffusers.utils import load_image pipe = FluxKontextPipeline.from_pretrained( - "black-forest-labs/FLUX.1-Kontext-dev", torch_dtype=torch.bfloat16 + "black-forest-labs/FLUX.1-Kontext-dev", dtype=torch.bfloat16 ) pipe.to("cuda") @@ -336,7 +336,7 @@ source = load_image(img_url) mask = load_image(mask_url) pipe = FluxKontextInpaintPipeline.from_pretrained( - "black-forest-labs/FLUX.1-Kontext-dev", torch_dtype=torch.bfloat16 + "black-forest-labs/FLUX.1-Kontext-dev", dtype=torch.bfloat16 ) pipe.to("cuda") @@ -352,7 +352,7 @@ from diffusers import FluxKontextInpaintPipeline from diffusers.utils import load_image pipe = FluxKontextInpaintPipeline.from_pretrained( - "black-forest-labs/FLUX.1-Kontext-dev", torch_dtype=torch.bfloat16 + "black-forest-labs/FLUX.1-Kontext-dev", dtype=torch.bfloat16 ) pipe.to("cuda") @@ -385,7 +385,7 @@ from diffusers.utils import load_image from huggingface_hub import hf_hub_download import torch -control_pipe = FluxControlPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16) +control_pipe = FluxControlPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", dtype=torch.bfloat16) control_pipe.load_lora_weights("black-forest-labs/FLUX.1-Depth-dev-lora", adapter_name="depth") control_pipe.load_lora_weights( hf_hub_download("ByteDance/Hyper-SD", "Hyper-FLUX.1-dev-8steps-lora.safetensors"), adapter_name="hyper-sd" @@ -428,7 +428,7 @@ from diffusers import FluxPipeline from diffusers.utils import load_image pipe = FluxPipeline.from_pretrained( - "black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16 + "black-forest-labs/FLUX.1-dev", dtype=torch.bfloat16 ).to("cuda") image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/flux_ip_adapter_input.jpg").resize((1024, 1024)) @@ -480,7 +480,7 @@ model_id = "black-forest-labs/FLUX.1-dev" dtype = torch.bfloat16 pipe = FluxPipeline.from_pretrained( model_id, - torch_dtype=dtype, + dtype=dtype, ) apply_group_offloading( @@ -534,7 +534,7 @@ FP16 inference code: import torch from diffusers import FluxPipeline -pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-schnell", torch_dtype=torch.bfloat16) # can replace schnell with dev +pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-schnell", dtype=torch.bfloat16) # can replace schnell with dev # to run on low vram GPUs (i.e. between 4 and 32 GB VRAM) pipe.enable_sequential_cpu_offload() pipe.vae.enable_slicing() @@ -570,7 +570,7 @@ text_encoder_8bit = T5EncoderModel.from_pretrained( "black-forest-labs/FLUX.1-dev", subfolder="text_encoder_2", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) quant_config = DiffusersBitsAndBytesConfig(load_in_8bit=True) @@ -578,14 +578,14 @@ transformer_8bit = FluxTransformer2DModel.from_pretrained( "black-forest-labs/FLUX.1-dev", subfolder="transformer", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) pipeline = FluxPipeline.from_pretrained( "black-forest-labs/FLUX.1-dev", text_encoder_2=text_encoder_8bit, transformer=transformer_8bit, - torch_dtype=torch.float16, + dtype=torch.float16, device_map="balanced", ) @@ -620,15 +620,15 @@ from optimum.quanto import freeze, qfloat8, quantize bfl_repo = "black-forest-labs/FLUX.1-dev" dtype = torch.bfloat16 -transformer = FluxTransformer2DModel.from_single_file("https://huggingface.co/Kijai/flux-fp8/blob/main/flux1-dev-fp8.safetensors", torch_dtype=dtype) +transformer = FluxTransformer2DModel.from_single_file("https://huggingface.co/Kijai/flux-fp8/blob/main/flux1-dev-fp8.safetensors", dtype=dtype) quantize(transformer, weights=qfloat8) freeze(transformer) -text_encoder_2 = T5EncoderModel.from_pretrained(bfl_repo, subfolder="text_encoder_2", torch_dtype=dtype) +text_encoder_2 = T5EncoderModel.from_pretrained(bfl_repo, subfolder="text_encoder_2", dtype=dtype) quantize(text_encoder_2, weights=qfloat8) freeze(text_encoder_2) -pipe = FluxPipeline.from_pretrained(bfl_repo, transformer=None, text_encoder_2=None, torch_dtype=dtype) +pipe = FluxPipeline.from_pretrained(bfl_repo, transformer=None, text_encoder_2=None, dtype=dtype) pipe.transformer = transformer pipe.text_encoder_2 = text_encoder_2 diff --git a/docs/source/en/api/pipelines/framepack.md b/docs/source/en/api/pipelines/framepack.md index a25cfe24a4ba..72f4c30630df 100644 --- a/docs/source/en/api/pipelines/framepack.md +++ b/docs/source/en/api/pipelines/framepack.md @@ -47,20 +47,20 @@ from diffusers.utils import export_to_video, load_image from transformers import SiglipImageProcessor, SiglipVisionModel transformer = HunyuanVideoFramepackTransformer3DModel.from_pretrained( - "lllyasviel/FramePackI2V_HY", torch_dtype=torch.bfloat16 + "lllyasviel/FramePackI2V_HY", dtype=torch.bfloat16 ) feature_extractor = SiglipImageProcessor.from_pretrained( "lllyasviel/flux_redux_bfl", subfolder="feature_extractor" ) image_encoder = SiglipVisionModel.from_pretrained( - "lllyasviel/flux_redux_bfl", subfolder="image_encoder", torch_dtype=torch.float16 + "lllyasviel/flux_redux_bfl", subfolder="image_encoder", dtype=torch.float16 ) pipe = HunyuanVideoFramepackPipeline.from_pretrained( "hunyuanvideo-community/HunyuanVideo", transformer=transformer, feature_extractor=feature_extractor, image_encoder=image_encoder, - torch_dtype=torch.float16, + dtype=torch.float16, ) # Enable memory optimizations @@ -100,20 +100,20 @@ from diffusers.utils import export_to_video, load_image from transformers import SiglipImageProcessor, SiglipVisionModel transformer = HunyuanVideoFramepackTransformer3DModel.from_pretrained( - "lllyasviel/FramePack_F1_I2V_HY_20250503", torch_dtype=torch.bfloat16 + "lllyasviel/FramePack_F1_I2V_HY_20250503", dtype=torch.bfloat16 ) feature_extractor = SiglipImageProcessor.from_pretrained( "lllyasviel/flux_redux_bfl", subfolder="feature_extractor" ) image_encoder = SiglipVisionModel.from_pretrained( - "lllyasviel/flux_redux_bfl", subfolder="image_encoder", torch_dtype=torch.float16 + "lllyasviel/flux_redux_bfl", subfolder="image_encoder", dtype=torch.float16 ) pipe = HunyuanVideoFramepackPipeline.from_pretrained( "hunyuanvideo-community/HunyuanVideo", transformer=transformer, feature_extractor=feature_extractor, image_encoder=image_encoder, - torch_dtype=torch.float16, + dtype=torch.float16, ) # Enable memory optimizations @@ -149,20 +149,20 @@ from diffusers.utils import export_to_video, load_image from transformers import SiglipImageProcessor, SiglipVisionModel transformer = HunyuanVideoFramepackTransformer3DModel.from_pretrained( - "lllyasviel/FramePack_F1_I2V_HY_20250503", torch_dtype=torch.bfloat16 + "lllyasviel/FramePack_F1_I2V_HY_20250503", dtype=torch.bfloat16 ) feature_extractor = SiglipImageProcessor.from_pretrained( "lllyasviel/flux_redux_bfl", subfolder="feature_extractor" ) image_encoder = SiglipVisionModel.from_pretrained( - "lllyasviel/flux_redux_bfl", subfolder="image_encoder", torch_dtype=torch.float16 + "lllyasviel/flux_redux_bfl", subfolder="image_encoder", dtype=torch.float16 ) pipe = HunyuanVideoFramepackPipeline.from_pretrained( "hunyuanvideo-community/HunyuanVideo", transformer=transformer, feature_extractor=feature_extractor, image_encoder=image_encoder, - torch_dtype=torch.float16, + dtype=torch.float16, ) # Enable group offloading diff --git a/docs/source/en/api/pipelines/glm_image.md b/docs/source/en/api/pipelines/glm_image.md index a99832787847..8f4d1ab3b50d 100644 --- a/docs/source/en/api/pipelines/glm_image.md +++ b/docs/source/en/api/pipelines/glm_image.md @@ -44,7 +44,7 @@ This pipeline was contributed by [zRzRzRzRzRzRzR](https://github.com/zRzRzRzRzRz import torch from diffusers.pipelines.glm_image import GlmImagePipeline -pipe = GlmImagePipeline.from_pretrained("zai-org/GLM-Image",torch_dtype=torch.bfloat16,device_map="cuda") +pipe = GlmImagePipeline.from_pretrained("zai-org/GLM-Image",dtype=torch.bfloat16,device_map="cuda") prompt = "A beautifully designed modern food magazine style dessert recipe illustration, themed around a raspberry mousse cake. The overall layout is clean and bright, divided into four main areas: the top left features a bold black title 'Raspberry Mousse Cake Recipe Guide', with a soft-lit close-up photo of the finished cake on the right, showcasing a light pink cake adorned with fresh raspberries and mint leaves; the bottom left contains an ingredient list section, titled 'Ingredients' in a simple font, listing 'Flour 150g', 'Eggs 3', 'Sugar 120g', 'Raspberry puree 200g', 'Gelatin sheets 10g', 'Whipping cream 300ml', and 'Fresh raspberries', each accompanied by minimalist line icons (like a flour bag, eggs, sugar jar, etc.); the bottom right displays four equally sized step boxes, each containing high-definition macro photos and corresponding instructions, arranged from top to bottom as follows: Step 1 shows a whisk whipping white foam (with the instruction 'Whip egg whites to stiff peaks'), Step 2 shows a red-and-white mixture being folded with a spatula (with the instruction 'Gently fold in the puree and batter'), Step 3 shows pink liquid being poured into a round mold (with the instruction 'Pour into mold and chill for 4 hours'), Step 4 shows the finished cake decorated with raspberries and mint leaves (with the instruction 'Decorate with raspberries and mint'); a light brown information bar runs along the bottom edge, with icons on the left representing 'Preparation time: 30 minutes', 'Cooking time: 20 minutes', and 'Servings: 8'. The overall color scheme is dominated by creamy white and light pink, with a subtle paper texture in the background, featuring compact and orderly text and image layout with clear information hierarchy." image = pipe( prompt=prompt, @@ -65,7 +65,7 @@ import torch from diffusers.pipelines.glm_image import GlmImagePipeline from PIL import Image -pipe = GlmImagePipeline.from_pretrained("zai-org/GLM-Image",torch_dtype=torch.bfloat16,device_map="cuda") +pipe = GlmImagePipeline.from_pretrained("zai-org/GLM-Image",dtype=torch.bfloat16,device_map="cuda") image_path = "cond.jpg" prompt = "Replace the background of the snow forest with an underground station featuring an automatic escalator." image = Image.open(image_path).convert("RGB") diff --git a/docs/source/en/api/pipelines/helios.md b/docs/source/en/api/pipelines/helios.md index 0b017bd7c2ab..b01fe88f4845 100644 --- a/docs/source/en/api/pipelines/helios.md +++ b/docs/source/en/api/pipelines/helios.md @@ -52,13 +52,13 @@ from diffusers import AutoModel, HeliosPipeline from diffusers.hooks.group_offloading import apply_group_offloading from diffusers.utils import export_to_video -vae = AutoModel.from_pretrained("BestWishYsh/Helios-Base", subfolder="vae", torch_dtype=torch.float32) +vae = AutoModel.from_pretrained("BestWishYsh/Helios-Base", subfolder="vae", dtype=torch.float32) # group-offloading pipeline = HeliosPipeline.from_pretrained( "BestWishYsh/Helios-Base", vae=vae, - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) pipeline.enable_group_offload( onload_device=torch.device("cuda"), @@ -103,12 +103,12 @@ import torch from diffusers import AutoModel, HeliosPipeline from diffusers.utils import export_to_video -vae = AutoModel.from_pretrained("BestWishYsh/Helios-Base", subfolder="vae", torch_dtype=torch.float32) +vae = AutoModel.from_pretrained("BestWishYsh/Helios-Base", subfolder="vae", dtype=torch.float32) pipeline = HeliosPipeline.from_pretrained( "BestWishYsh/Helios-Base", vae=vae, - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) pipeline.to("cuda") @@ -163,12 +163,12 @@ import torch from diffusers import AutoModel, HeliosPipeline from diffusers.utils import export_to_video, load_video, load_image -vae = AutoModel.from_pretrained("BestWishYsh/Helios-Base", subfolder="vae", torch_dtype=torch.float32) +vae = AutoModel.from_pretrained("BestWishYsh/Helios-Base", subfolder="vae", dtype=torch.float32) pipeline = HeliosPipeline.from_pretrained( "BestWishYsh/Helios-Base", vae=vae, - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) pipeline.to("cuda") @@ -258,12 +258,12 @@ import torch from diffusers import AutoModel, HeliosPyramidPipeline from diffusers.utils import export_to_video, load_video, load_image -vae = AutoModel.from_pretrained("BestWishYsh/Helios-Mid", subfolder="vae", torch_dtype=torch.float32) +vae = AutoModel.from_pretrained("BestWishYsh/Helios-Mid", subfolder="vae", dtype=torch.float32) pipeline = HeliosPyramidPipeline.from_pretrained( "BestWishYsh/Helios-Mid", vae=vae, - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) pipeline.to("cuda") @@ -359,12 +359,12 @@ import torch from diffusers import AutoModel, HeliosPyramidPipeline from diffusers.utils import export_to_video, load_video, load_image -vae = AutoModel.from_pretrained("BestWishYsh/Helios-Distilled", subfolder="vae", torch_dtype=torch.float32) +vae = AutoModel.from_pretrained("BestWishYsh/Helios-Distilled", subfolder="vae", dtype=torch.float32) pipeline = HeliosPyramidPipeline.from_pretrained( "BestWishYsh/Helios-Distilled", vae=vae, - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) pipeline.to("cuda") diff --git a/docs/source/en/api/pipelines/hunyuan_video.md b/docs/source/en/api/pipelines/hunyuan_video.md index cdd81495b621..462222f7aeaa 100644 --- a/docs/source/en/api/pipelines/hunyuan_video.md +++ b/docs/source/en/api/pipelines/hunyuan_video.md @@ -60,7 +60,7 @@ pipeline_quant_config = PipelineQuantizationConfig( pipeline = HunyuanVideoPipeline.from_pretrained( "hunyuanvideo-community/HunyuanVideo", quantization_config=pipeline_quant_config, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) # model-offloading and tiling @@ -97,7 +97,7 @@ pipeline_quant_config = PipelineQuantizationConfig( pipeline = HunyuanVideoPipeline.from_pretrained( "hunyuanvideo-community/HunyuanVideo", quantization_config=pipeline_quant_config, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) # model-offloading and tiling @@ -145,7 +145,7 @@ export_to_video(video, "output.mp4", fps=15) pipeline = HunyuanVideoPipeline.from_pretrained( "hunyuanvideo-community/HunyuanVideo", quantization_config=pipeline_quant_config, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) # load LoRA weights diff --git a/docs/source/en/api/pipelines/hunyuan_video15.md b/docs/source/en/api/pipelines/hunyuan_video15.md index dfaeab6528f9..f6e968ae753a 100644 --- a/docs/source/en/api/pipelines/hunyuan_video15.md +++ b/docs/source/en/api/pipelines/hunyuan_video15.md @@ -40,7 +40,7 @@ from diffusers.utils import export_to_video pipeline = HunyuanVideo15Pipeline.from_pretrained( "HunyuanVideo-1.5-Diffusers-480p_t2v", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) # model-offloading and tiling diff --git a/docs/source/en/api/pipelines/hunyuandit.md b/docs/source/en/api/pipelines/hunyuandit.md index 70989e26337d..e1aae6756fda 100644 --- a/docs/source/en/api/pipelines/hunyuandit.md +++ b/docs/source/en/api/pipelines/hunyuandit.md @@ -49,7 +49,7 @@ from diffusers import HunyuanDiTPipeline import torch pipeline = HunyuanDiTPipeline.from_pretrained( - "Tencent-Hunyuan/HunyuanDiT-Diffusers", torch_dtype=torch.float16 + "Tencent-Hunyuan/HunyuanDiT-Diffusers", dtype=torch.float16 ).to("cuda") ``` diff --git a/docs/source/en/api/pipelines/hunyuanimage21.md b/docs/source/en/api/pipelines/hunyuanimage21.md index 9e8ea2627e33..84dcf1c4b87d 100644 --- a/docs/source/en/api/pipelines/hunyuanimage21.md +++ b/docs/source/en/api/pipelines/hunyuanimage21.md @@ -38,7 +38,7 @@ from diffusers import HunyuanImagePipeline pipe = HunyuanImagePipeline.from_pretrained( "hunyuanvideo-community/HunyuanImage-2.1-Diffusers", - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) pipe = pipe.to("cuda") ``` @@ -83,7 +83,7 @@ from diffusers import HunyuanImagePipeline pipe = HunyuanImagePipeline.from_pretrained( "hunyuanvideo-community/HunyuanImage-2.1-Diffusers", - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) pipe = pipe.to("cuda") @@ -113,7 +113,7 @@ use `distilled_guidance_scale` with the guidance-distilled checkpoint, ```py import torch from diffusers import HunyuanImagePipeline -pipe = HunyuanImagePipeline.from_pretrained("hunyuanvideo-community/HunyuanImage-2.1-Distilled-Diffusers", torch_dtype=torch.bfloat16) +pipe = HunyuanImagePipeline.from_pretrained("hunyuanvideo-community/HunyuanImage-2.1-Distilled-Diffusers", dtype=torch.bfloat16) pipe = pipe.to("cuda") prompt = ( diff --git a/docs/source/en/api/pipelines/ideogram4.md b/docs/source/en/api/pipelines/ideogram4.md index 3c8553a9fdad..d9f3e341b169 100644 --- a/docs/source/en/api/pipelines/ideogram4.md +++ b/docs/source/en/api/pipelines/ideogram4.md @@ -31,7 +31,7 @@ Key inference-time knobs are exposed via the pipeline call: import torch from diffusers import Ideogram4Pipeline -pipe = Ideogram4Pipeline.from_pretrained("ideogram-ai/ideogram-v4", torch_dtype=torch.bfloat16) +pipe = Ideogram4Pipeline.from_pretrained("ideogram-ai/ideogram-v4", dtype=torch.bfloat16) pipe.to("cuda") prompt = "A photo of a cat holding a sign that says hello world" @@ -56,7 +56,7 @@ import requests import torch from diffusers import Ideogram4Pipeline -pipe = Ideogram4Pipeline.from_pretrained("ideogram-ai/ideogram-4-nf4", torch_dtype=torch.bfloat16) +pipe = Ideogram4Pipeline.from_pretrained("ideogram-ai/ideogram-4-nf4", dtype=torch.bfloat16) pipe.to("cuda") # Expand the prompt into a structured JSON caption with Ideogram's hosted magic-prompt API. @@ -84,10 +84,10 @@ import torch from diffusers import Ideogram4Pipeline, Ideogram4PromptEnhancerHead prompt_enhancer_head = Ideogram4PromptEnhancerHead.from_pretrained( - "diffusers/qwen3-vl-8b-instruct-lm-head", torch_dtype=torch.bfloat16 + "diffusers/qwen3-vl-8b-instruct-lm-head", dtype=torch.bfloat16 ) pipe = Ideogram4Pipeline.from_pretrained( - "ideogram-ai/ideogram-4-nf4", prompt_enhancer_head=prompt_enhancer_head, torch_dtype=torch.bfloat16 + "ideogram-ai/ideogram-4-nf4", prompt_enhancer_head=prompt_enhancer_head, dtype=torch.bfloat16 ) pipe.to("cuda") diff --git a/docs/source/en/api/pipelines/joyimage_edit.md b/docs/source/en/api/pipelines/joyimage_edit.md index cb8af3c76d4c..dc30aeedf6e1 100644 --- a/docs/source/en/api/pipelines/joyimage_edit.md +++ b/docs/source/en/api/pipelines/joyimage_edit.md @@ -26,7 +26,7 @@ from diffusers import JoyImageEditPipeline from diffusers.utils import load_image pipeline = JoyImageEditPipeline.from_pretrained( - "jdopensource/JoyAI-Image-Edit-Diffusers", torch_dtype=torch.bfloat16 + "jdopensource/JoyAI-Image-Edit-Diffusers", dtype=torch.bfloat16 ) pipeline.to("cuda") diff --git a/docs/source/en/api/pipelines/joyimage_edit_plus.md b/docs/source/en/api/pipelines/joyimage_edit_plus.md index 92609d166af0..c16856fd191c 100644 --- a/docs/source/en/api/pipelines/joyimage_edit_plus.md +++ b/docs/source/en/api/pipelines/joyimage_edit_plus.md @@ -26,7 +26,7 @@ from PIL import Image from diffusers import JoyImageEditPlusPipeline pipeline = JoyImageEditPlusPipeline.from_pretrained( - "jdopensource/JoyAI-Image-Edit-Plus-Diffusers", torch_dtype=torch.bfloat16 + "jdopensource/JoyAI-Image-Edit-Plus-Diffusers", dtype=torch.bfloat16 ) pipeline.to("cuda") diff --git a/docs/source/en/api/pipelines/kandinsky.md b/docs/source/en/api/pipelines/kandinsky.md index ba78740ac372..d56477a17591 100644 --- a/docs/source/en/api/pipelines/kandinsky.md +++ b/docs/source/en/api/pipelines/kandinsky.md @@ -48,8 +48,8 @@ To use the Kandinsky models for any task, you always start by setting up the pri from diffusers import KandinskyPriorPipeline, KandinskyPipeline import torch -prior_pipeline = KandinskyPriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-1-prior", torch_dtype=torch.float16).to("cuda") -pipeline = KandinskyPipeline.from_pretrained("kandinsky-community/kandinsky-2-1", torch_dtype=torch.float16).to("cuda") +prior_pipeline = KandinskyPriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-1-prior", dtype=torch.float16).to("cuda") +pipeline = KandinskyPipeline.from_pretrained("kandinsky-community/kandinsky-2-1", dtype=torch.float16).to("cuda") prompt = "A alien cheeseburger creature eating itself, claymation, cinematic, moody lighting" negative_prompt = "low quality, bad quality" # optional to include a negative prompt, but results are usually better @@ -74,8 +74,8 @@ image from diffusers import KandinskyV22PriorPipeline, KandinskyV22Pipeline import torch -prior_pipeline = KandinskyV22PriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-2-prior", torch_dtype=torch.float16).to("cuda") -pipeline = KandinskyV22Pipeline.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", torch_dtype=torch.float16).to("cuda") +prior_pipeline = KandinskyV22PriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-2-prior", dtype=torch.float16).to("cuda") +pipeline = KandinskyV22Pipeline.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", dtype=torch.float16).to("cuda") prompt = "A alien cheeseburger creature eating itself, claymation, cinematic, moody lighting" negative_prompt = "low quality, bad quality" # optional to include a negative prompt, but results are usually better @@ -102,7 +102,7 @@ Kandinsky 3 doesn't require a prior model so you can directly load the [`Kandins from diffusers import Kandinsky3Pipeline import torch -pipeline = Kandinsky3Pipeline.from_pretrained("kandinsky-community/kandinsky-3", variant="fp16", torch_dtype=torch.float16) +pipeline = Kandinsky3Pipeline.from_pretrained("kandinsky-community/kandinsky-3", variant="fp16", dtype=torch.float16) pipeline.enable_model_cpu_offload() prompt = "A alien cheeseburger creature eating itself, claymation, cinematic, moody lighting" @@ -124,7 +124,7 @@ Use the [`AutoPipelineForText2Image`] to automatically call the combined pipelin from diffusers import AutoPipelineForText2Image import torch -pipeline = AutoPipelineForText2Image.from_pretrained("kandinsky-community/kandinsky-2-1", torch_dtype=torch.float16) +pipeline = AutoPipelineForText2Image.from_pretrained("kandinsky-community/kandinsky-2-1", dtype=torch.float16) pipeline.enable_model_cpu_offload() prompt = "A alien cheeseburger creature eating itself, claymation, cinematic, moody lighting" @@ -141,7 +141,7 @@ image from diffusers import AutoPipelineForText2Image import torch -pipeline = AutoPipelineForText2Image.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", torch_dtype=torch.float16) +pipeline = AutoPipelineForText2Image.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", dtype=torch.float16) pipeline.enable_model_cpu_offload() prompt = "A alien cheeseburger creature eating itself, claymation, cinematic, moody lighting" @@ -165,8 +165,8 @@ For image-to-image, pass the initial image and text prompt to condition the imag import torch from diffusers import KandinskyImg2ImgPipeline, KandinskyPriorPipeline -prior_pipeline = KandinskyPriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-1-prior", torch_dtype=torch.float16, use_safetensors=True).to("cuda") -pipeline = KandinskyImg2ImgPipeline.from_pretrained("kandinsky-community/kandinsky-2-1", torch_dtype=torch.float16, use_safetensors=True).to("cuda") +prior_pipeline = KandinskyPriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-1-prior", dtype=torch.float16, use_safetensors=True).to("cuda") +pipeline = KandinskyImg2ImgPipeline.from_pretrained("kandinsky-community/kandinsky-2-1", dtype=torch.float16, use_safetensors=True).to("cuda") ``` @@ -176,8 +176,8 @@ pipeline = KandinskyImg2ImgPipeline.from_pretrained("kandinsky-community/kandins import torch from diffusers import KandinskyV22Img2ImgPipeline, KandinskyPriorPipeline -prior_pipeline = KandinskyPriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-2-prior", torch_dtype=torch.float16, use_safetensors=True).to("cuda") -pipeline = KandinskyV22Img2ImgPipeline.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", torch_dtype=torch.float16, use_safetensors=True).to("cuda") +prior_pipeline = KandinskyPriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-2-prior", dtype=torch.float16, use_safetensors=True).to("cuda") +pipeline = KandinskyV22Img2ImgPipeline.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", dtype=torch.float16, use_safetensors=True).to("cuda") ``` @@ -190,7 +190,7 @@ from diffusers import Kandinsky3Img2ImgPipeline from diffusers.utils import load_image import torch -pipeline = Kandinsky3Img2ImgPipeline.from_pretrained("kandinsky-community/kandinsky-3", variant="fp16", torch_dtype=torch.float16) +pipeline = Kandinsky3Img2ImgPipeline.from_pretrained("kandinsky-community/kandinsky-3", variant="fp16", dtype=torch.float16) pipeline.enable_model_cpu_offload() ``` @@ -274,7 +274,7 @@ from diffusers import AutoPipelineForImage2Image from diffusers.utils import make_image_grid, load_image import torch -pipeline = AutoPipelineForImage2Image.from_pretrained("kandinsky-community/kandinsky-2-1", torch_dtype=torch.float16, use_safetensors=True) +pipeline = AutoPipelineForImage2Image.from_pretrained("kandinsky-community/kandinsky-2-1", dtype=torch.float16, use_safetensors=True) pipeline.enable_model_cpu_offload() prompt = "A fantasy landscape, Cinematic lighting" @@ -297,7 +297,7 @@ from diffusers import AutoPipelineForImage2Image from diffusers.utils import make_image_grid, load_image import torch -pipeline = AutoPipelineForImage2Image.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", torch_dtype=torch.float16) +pipeline = AutoPipelineForImage2Image.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", dtype=torch.float16) pipeline.enable_model_cpu_offload() prompt = "A fantasy landscape, Cinematic lighting" @@ -341,8 +341,8 @@ import torch import numpy as np from PIL import Image -prior_pipeline = KandinskyPriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-1-prior", torch_dtype=torch.float16, use_safetensors=True).to("cuda") -pipeline = KandinskyInpaintPipeline.from_pretrained("kandinsky-community/kandinsky-2-1-inpaint", torch_dtype=torch.float16, use_safetensors=True).to("cuda") +prior_pipeline = KandinskyPriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-1-prior", dtype=torch.float16, use_safetensors=True).to("cuda") +pipeline = KandinskyInpaintPipeline.from_pretrained("kandinsky-community/kandinsky-2-1-inpaint", dtype=torch.float16, use_safetensors=True).to("cuda") ``` @@ -355,8 +355,8 @@ import torch import numpy as np from PIL import Image -prior_pipeline = KandinskyV22PriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-2-prior", torch_dtype=torch.float16, use_safetensors=True).to("cuda") -pipeline = KandinskyV22InpaintPipeline.from_pretrained("kandinsky-community/kandinsky-2-2-decoder-inpaint", torch_dtype=torch.float16, use_safetensors=True).to("cuda") +prior_pipeline = KandinskyV22PriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-2-prior", dtype=torch.float16, use_safetensors=True).to("cuda") +pipeline = KandinskyV22InpaintPipeline.from_pretrained("kandinsky-community/kandinsky-2-2-decoder-inpaint", dtype=torch.float16, use_safetensors=True).to("cuda") ``` @@ -421,7 +421,7 @@ from PIL import Image from diffusers import AutoPipelineForInpainting from diffusers.utils import load_image, make_image_grid -pipe = AutoPipelineForInpainting.from_pretrained("kandinsky-community/kandinsky-2-1-inpaint", torch_dtype=torch.float16) +pipe = AutoPipelineForInpainting.from_pretrained("kandinsky-community/kandinsky-2-1-inpaint", dtype=torch.float16) pipe.enable_model_cpu_offload() init_image = load_image("https://huggingface.co/datasets/hf-internal-testing/diffusers-images/resolve/main/kandinsky/cat.png") @@ -445,7 +445,7 @@ from PIL import Image from diffusers import AutoPipelineForInpainting from diffusers.utils import load_image, make_image_grid -pipe = AutoPipelineForInpainting.from_pretrained("kandinsky-community/kandinsky-2-2-decoder-inpaint", torch_dtype=torch.float16) +pipe = AutoPipelineForInpainting.from_pretrained("kandinsky-community/kandinsky-2-2-decoder-inpaint", dtype=torch.float16) pipe.enable_model_cpu_offload() init_image = load_image("https://huggingface.co/datasets/hf-internal-testing/diffusers-images/resolve/main/kandinsky/cat.png") @@ -474,7 +474,7 @@ from diffusers import KandinskyPriorPipeline, KandinskyPipeline from diffusers.utils import load_image, make_image_grid import torch -prior_pipeline = KandinskyPriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-1-prior", torch_dtype=torch.float16, use_safetensors=True).to("cuda") +prior_pipeline = KandinskyPriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-1-prior", dtype=torch.float16, use_safetensors=True).to("cuda") img_1 = load_image("https://huggingface.co/datasets/hf-internal-testing/diffusers-images/resolve/main/kandinsky/cat.png") img_2 = load_image("https://huggingface.co/datasets/hf-internal-testing/diffusers-images/resolve/main/kandinsky/starry_night.jpeg") make_image_grid([img_1.resize((512,512)), img_2.resize((512,512))], rows=1, cols=2) @@ -488,7 +488,7 @@ from diffusers import KandinskyV22PriorPipeline, KandinskyV22Pipeline from diffusers.utils import load_image, make_image_grid import torch -prior_pipeline = KandinskyV22PriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-2-prior", torch_dtype=torch.float16, use_safetensors=True).to("cuda") +prior_pipeline = KandinskyV22PriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-2-prior", dtype=torch.float16, use_safetensors=True).to("cuda") img_1 = load_image("https://huggingface.co/datasets/hf-internal-testing/diffusers-images/resolve/main/kandinsky/cat.png") img_2 = load_image("https://huggingface.co/datasets/hf-internal-testing/diffusers-images/resolve/main/kandinsky/starry_night.jpeg") make_image_grid([img_1.resize((512,512)), img_2.resize((512,512))], rows=1, cols=2) @@ -525,7 +525,7 @@ Call the `interpolate` function to generate the embeddings, and then pass them t prompt = "" prior_out = prior_pipeline.interpolate(images_texts, weights) -pipeline = KandinskyPipeline.from_pretrained("kandinsky-community/kandinsky-2-1", torch_dtype=torch.float16, use_safetensors=True).to("cuda") +pipeline = KandinskyPipeline.from_pretrained("kandinsky-community/kandinsky-2-1", dtype=torch.float16, use_safetensors=True).to("cuda") image = pipeline(prompt, **prior_out, height=768, width=768).images[0] image @@ -543,7 +543,7 @@ image prompt = "" prior_out = prior_pipeline.interpolate(images_texts, weights) -pipeline = KandinskyV22Pipeline.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", torch_dtype=torch.float16, use_safetensors=True).to("cuda") +pipeline = KandinskyV22Pipeline.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", dtype=torch.float16, use_safetensors=True).to("cuda") image = pipeline(prompt, **prior_out, height=768, width=768).images[0] image @@ -607,11 +607,11 @@ Load the prior pipeline and the [`KandinskyV22ControlnetPipeline`]: from diffusers import KandinskyV22PriorPipeline, KandinskyV22ControlnetPipeline prior_pipeline = KandinskyV22PriorPipeline.from_pretrained( - "kandinsky-community/kandinsky-2-2-prior", torch_dtype=torch.float16, use_safetensors=True + "kandinsky-community/kandinsky-2-2-prior", dtype=torch.float16, use_safetensors=True ).to("cuda") pipeline = KandinskyV22ControlnetPipeline.from_pretrained( - "kandinsky-community/kandinsky-2-2-controlnet-depth", torch_dtype=torch.float16 + "kandinsky-community/kandinsky-2-2-controlnet-depth", dtype=torch.float16 ).to("cuda") ``` @@ -677,11 +677,11 @@ Load the prior pipeline and the [`KandinskyV22ControlnetImg2ImgPipeline`]: ```py prior_pipeline = KandinskyV22PriorEmb2EmbPipeline.from_pretrained( - "kandinsky-community/kandinsky-2-2-prior", torch_dtype=torch.float16, use_safetensors=True + "kandinsky-community/kandinsky-2-2-prior", dtype=torch.float16, use_safetensors=True ).to("cuda") pipeline = KandinskyV22ControlnetImg2ImgPipeline.from_pretrained( - "kandinsky-community/kandinsky-2-2-controlnet-depth", torch_dtype=torch.float16 + "kandinsky-community/kandinsky-2-2-controlnet-depth", dtype=torch.float16 ).to("cuda") ``` @@ -718,7 +718,7 @@ Kandinsky is unique because it requires a prior pipeline to generate the mapping from diffusers import DiffusionPipeline import torch - pipe = DiffusionPipeline.from_pretrained("kandinsky-community/kandinsky-2-1", torch_dtype=torch.float16) + pipe = DiffusionPipeline.from_pretrained("kandinsky-community/kandinsky-2-1", dtype=torch.float16) + pipe.enable_xformers_memory_efficient_attention() ``` @@ -743,7 +743,7 @@ pipe.unet.set_attn_processor(AttnAddedKVProcessor2_0()) from diffusers import DiffusionPipeline import torch - pipe = DiffusionPipeline.from_pretrained("kandinsky-community/kandinsky-2-1", torch_dtype=torch.float16) + pipe = DiffusionPipeline.from_pretrained("kandinsky-community/kandinsky-2-1", dtype=torch.float16) + pipe.enable_model_cpu_offload() ``` @@ -754,7 +754,7 @@ from diffusers import DDPMScheduler from diffusers import DiffusionPipeline scheduler = DDPMScheduler.from_pretrained("kandinsky-community/kandinsky-2-1", subfolder="ddpm_scheduler") -pipe = DiffusionPipeline.from_pretrained("kandinsky-community/kandinsky-2-1", scheduler=scheduler, torch_dtype=torch.float16, use_safetensors=True).to("cuda") +pipe = DiffusionPipeline.from_pretrained("kandinsky-community/kandinsky-2-1", scheduler=scheduler, dtype=torch.float16, use_safetensors=True).to("cuda") ``` ## KandinskyPriorPipeline diff --git a/docs/source/en/api/pipelines/kandinsky5_video.md b/docs/source/en/api/pipelines/kandinsky5_video.md index 733e2481732a..78e421e23a70 100644 --- a/docs/source/en/api/pipelines/kandinsky5_video.md +++ b/docs/source/en/api/pipelines/kandinsky5_video.md @@ -62,7 +62,7 @@ from diffusers.utils import export_to_video # Load the pipeline model_id = "kandinskylab/Kandinsky-5.0-T2V-Pro-sft-5s-Diffusers" -pipe = Kandinsky5T2VPipeline.from_pretrained(model_id, torch_dtype=torch.bfloat16) +pipe = Kandinsky5T2VPipeline.from_pretrained(model_id, dtype=torch.bfloat16) pipe = pipe.to("cuda") pipeline.transformer.set_attention_backend("flex") # <--- Set attention bakend to Flex @@ -94,7 +94,7 @@ from diffusers.utils import export_to_video # Load the pipeline model_id = "kandinskylab/Kandinsky-5.0-T2V-Lite-sft-5s-Diffusers" -pipe = Kandinsky5T2VPipeline.from_pretrained(model_id, torch_dtype=torch.bfloat16) +pipe = Kandinsky5T2VPipeline.from_pretrained(model_id, dtype=torch.bfloat16) pipe = pipe.to("cuda") # Generate video @@ -120,7 +120,7 @@ export_to_video(output, "output.mp4", fps=24, quality=9) ```python pipe = Kandinsky5T2VPipeline.from_pretrained( "kandinskylab/Kandinsky-5.0-T2V-Lite-sft-10s-Diffusers", - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) pipe = pipe.to("cuda") @@ -153,7 +153,7 @@ export_to_video(output, "output.mp4", fps=24, quality=9) ```python model_id = "kandinskylab/Kandinsky-5.0-T2V-Lite-distilled16steps-5s-Diffusers" -pipe = Kandinsky5T2VPipeline.from_pretrained(model_id, torch_dtype=torch.bfloat16) +pipe = Kandinsky5T2VPipeline.from_pretrained(model_id, dtype=torch.bfloat16) pipe = pipe.to("cuda") output = pipe( @@ -175,7 +175,7 @@ from diffusers.utils import export_to_video # Load the pipeline model_id = "kandinskylab/Kandinsky-5.0-I2V-Pro-sft-5s-Diffusers" -pipe = Kandinsky5T2VPipeline.from_pretrained(model_id, torch_dtype=torch.bfloat16) +pipe = Kandinsky5T2VPipeline.from_pretrained(model_id, dtype=torch.bfloat16) pipe = pipe.to("cuda") pipeline.transformer.set_attention_backend("flex") # <--- Set attention bakend to Flex diff --git a/docs/source/en/api/pipelines/kolors.md b/docs/source/en/api/pipelines/kolors.md index b4c83fe134f5..c5e97ca955a1 100644 --- a/docs/source/en/api/pipelines/kolors.md +++ b/docs/source/en/api/pipelines/kolors.md @@ -32,7 +32,7 @@ import torch from diffusers import DPMSolverMultistepScheduler, KolorsPipeline -pipe = KolorsPipeline.from_pretrained("Kwai-Kolors/Kolors-diffusers", torch_dtype=torch.float16, variant="fp16") +pipe = KolorsPipeline.from_pretrained("Kwai-Kolors/Kolors-diffusers", dtype=torch.float16, variant="fp16") pipe.to("cuda") pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config, use_karras_sigmas=True) @@ -67,12 +67,12 @@ image_encoder = CLIPVisionModelWithProjection.from_pretrained( "Kwai-Kolors/Kolors-IP-Adapter-Plus", subfolder="image_encoder", low_cpu_mem_usage=True, - torch_dtype=torch.float16, + dtype=torch.float16, revision="refs/pr/4", ) pipe = KolorsPipeline.from_pretrained( - "Kwai-Kolors/Kolors-diffusers", image_encoder=image_encoder, torch_dtype=torch.float16, variant="fp16" + "Kwai-Kolors/Kolors-diffusers", image_encoder=image_encoder, dtype=torch.float16, variant="fp16" ) pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config, use_karras_sigmas=True) diff --git a/docs/source/en/api/pipelines/krea2.md b/docs/source/en/api/pipelines/krea2.md index 08e38870875c..b6e6fd5998c3 100644 --- a/docs/source/en/api/pipelines/krea2.md +++ b/docs/source/en/api/pipelines/krea2.md @@ -35,7 +35,7 @@ import torch from diffusers import Krea2Pipeline # Load from a local directory produced by the Krea 2 conversion (no hub repo yet). -pipe = Krea2Pipeline.from_pretrained("krea/Krea-2-Raw", torch_dtype=torch.bfloat16) +pipe = Krea2Pipeline.from_pretrained("krea/Krea-2-Raw", dtype=torch.bfloat16) pipe.to("cuda") prompt = "a fox in the snow" @@ -56,7 +56,7 @@ We additionally provide an example for using Krea2 Turbo : import torch from diffusers import Krea2Pipeline -pipe = Krea2Pipeline.from_pretrained("krea/Krea-2-Turbo", torch_dtype=torch.bfloat16) +pipe = Krea2Pipeline.from_pretrained("krea/Krea-2-Turbo", dtype=torch.bfloat16) pipe.to("cuda") image = pipe( @@ -92,7 +92,7 @@ import torch from diffusers import ClassifierFreeGuidance, ModularPipeline pipe = ModularPipeline.from_pretrained("krea/Krea-2-Raw") -pipe.load_components(torch_dtype=torch.bfloat16) +pipe.load_components(dtype=torch.bfloat16) pipe.to("cuda") @@ -116,7 +116,7 @@ import torch from diffusers import ModularPipeline pipe = ModularPipeline.from_pretrained("krea/Krea-2-Turbo") -pipe.load_components(torch_dtype=torch.bfloat16) +pipe.load_components(dtype=torch.bfloat16) pipe.to("cuda") image = pipe( diff --git a/docs/source/en/api/pipelines/latent_consistency_models.md b/docs/source/en/api/pipelines/latent_consistency_models.md index 14aee3de2eae..28ef207ff4b6 100644 --- a/docs/source/en/api/pipelines/latent_consistency_models.md +++ b/docs/source/en/api/pipelines/latent_consistency_models.md @@ -47,11 +47,11 @@ import torch unet = UNet2DConditionModel.from_pretrained( "latent-consistency/lcm-sdxl", - torch_dtype=torch.float16, + dtype=torch.float16, variant="fp16", ) pipe = StableDiffusionXLPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", unet=unet, torch_dtype=torch.float16, variant="fp16", + "stabilityai/stable-diffusion-xl-base-1.0", unet=unet, dtype=torch.float16, variant="fp16", ).to("cuda") pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config) @@ -85,7 +85,7 @@ from diffusers import DiffusionPipeline, LCMScheduler pipe = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", variant="fp16", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config) pipe.load_lora_weights("latent-consistency/lcm-lora-sdxl") @@ -123,13 +123,13 @@ from diffusers.utils import load_image unet = UNet2DConditionModel.from_pretrained( "SimianLuo/LCM_Dreamshaper_v7", subfolder="unet", - torch_dtype=torch.float16, + dtype=torch.float16, ) pipe = AutoPipelineForImage2Image.from_pretrained( "Lykon/dreamshaper-7", unet=unet, - torch_dtype=torch.float16, + dtype=torch.float16, variant="fp16", ).to("cuda") pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config) @@ -174,7 +174,7 @@ from diffusers.utils import make_image_grid, load_image pipe = AutoPipelineForImage2Image.from_pretrained( "Lykon/dreamshaper-7", - torch_dtype=torch.float16, + dtype=torch.float16, variant="fp16", ).to("cuda") @@ -222,7 +222,7 @@ from diffusers.utils import load_image, make_image_grid pipe = AutoPipelineForInpainting.from_pretrained( "stable-diffusion-v1-5/stable-diffusion-inpainting", - torch_dtype=torch.float16, + dtype=torch.float16, variant="fp16", ).to("cuda") @@ -276,11 +276,11 @@ import torch unet = UNet2DConditionModel.from_pretrained( "latent-consistency/lcm-sdxl", - torch_dtype=torch.float16, + dtype=torch.float16, variant="fp16", ) pipe = StableDiffusionXLPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", unet=unet, torch_dtype=torch.float16, variant="fp16", + "stabilityai/stable-diffusion-xl-base-1.0", unet=unet, dtype=torch.float16, variant="fp16", ).to("cuda") pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config) pipe.load_lora_weights("TheLastBen/Papercut_SDXL", weight_name="papercut.safetensors", adapter_name="papercut") @@ -309,7 +309,7 @@ from diffusers import DiffusionPipeline, LCMScheduler pipe = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", variant="fp16", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config) @@ -369,11 +369,11 @@ image = image[:, :, None] image = np.concatenate([image, image, image], axis=2) canny_image = Image.fromarray(image) -controlnet = ControlNetModel.from_pretrained("lllyasviel/sd-controlnet-canny", torch_dtype=torch.float16) +controlnet = ControlNetModel.from_pretrained("lllyasviel/sd-controlnet-canny", dtype=torch.float16) pipe = StableDiffusionControlNetPipeline.from_pretrained( "SimianLuo/LCM_Dreamshaper_v7", controlnet=controlnet, - torch_dtype=torch.float16, + dtype=torch.float16, safety_checker=None, ).to("cuda") pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config) @@ -423,11 +423,11 @@ image = image[:, :, None] image = np.concatenate([image, image, image], axis=2) canny_image = Image.fromarray(image) -controlnet = ControlNetModel.from_pretrained("lllyasviel/sd-controlnet-canny", torch_dtype=torch.float16) +controlnet = ControlNetModel.from_pretrained("lllyasviel/sd-controlnet-canny", dtype=torch.float16) pipe = StableDiffusionControlNetPipeline.from_pretrained( "stable-diffusion-v1-5/stable-diffusion-v1-5", controlnet=controlnet, - torch_dtype=torch.float16, + dtype=torch.float16, safety_checker=None, variant="fp16" ).to("cuda") @@ -491,18 +491,18 @@ image = image[:, :, None] image = np.concatenate([image, image, image], axis=2) canny_image = Image.fromarray(image).resize((1024, 1216)) -adapter = T2IAdapter.from_pretrained("TencentARC/t2i-adapter-canny-sdxl-1.0", torch_dtype=torch.float16, variant="fp16").to("cuda") +adapter = T2IAdapter.from_pretrained("TencentARC/t2i-adapter-canny-sdxl-1.0", dtype=torch.float16, variant="fp16").to("cuda") unet = UNet2DConditionModel.from_pretrained( "latent-consistency/lcm-sdxl", - torch_dtype=torch.float16, + dtype=torch.float16, variant="fp16", ) pipe = StableDiffusionXLAdapterPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", unet=unet, adapter=adapter, - torch_dtype=torch.float16, + dtype=torch.float16, variant="fp16", ).to("cuda") @@ -557,12 +557,12 @@ image = image[:, :, None] image = np.concatenate([image, image, image], axis=2) canny_image = Image.fromarray(image).resize((1024, 1024)) -adapter = T2IAdapter.from_pretrained("TencentARC/t2i-adapter-canny-sdxl-1.0", torch_dtype=torch.float16, variant="fp16").to("cuda") +adapter = T2IAdapter.from_pretrained("TencentARC/t2i-adapter-canny-sdxl-1.0", dtype=torch.float16, variant="fp16").to("cuda") pipe = StableDiffusionXLAdapterPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", adapter=adapter, - torch_dtype=torch.float16, + dtype=torch.float16, variant="fp16", ).to("cuda") diff --git a/docs/source/en/api/pipelines/latte.md b/docs/source/en/api/pipelines/latte.md index 4730b5088d17..ecb28b6f0872 100644 --- a/docs/source/en/api/pipelines/latte.md +++ b/docs/source/en/api/pipelines/latte.md @@ -40,7 +40,7 @@ import torch from diffusers import LattePipeline pipeline = LattePipeline.from_pretrained( - "maxin-cn/Latte-1", torch_dtype=torch.float16 + "maxin-cn/Latte-1", dtype=torch.float16 ).to("cuda") ``` @@ -84,7 +84,7 @@ text_encoder_8bit = T5EncoderModel.from_pretrained( "maxin-cn/Latte-1", subfolder="text_encoder", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) quant_config = DiffusersBitsAndBytesConfig(load_in_8bit=True) @@ -92,14 +92,14 @@ transformer_8bit = LatteTransformer3DModel.from_pretrained( "maxin-cn/Latte-1", subfolder="transformer", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) pipeline = LattePipeline.from_pretrained( "maxin-cn/Latte-1", text_encoder=text_encoder_8bit, transformer=transformer_8bit, - torch_dtype=torch.float16, + dtype=torch.float16, device_map="balanced", ) diff --git a/docs/source/en/api/pipelines/longcat_audio_dit.md b/docs/source/en/api/pipelines/longcat_audio_dit.md index 4ecdbd39d356..ce692d034aec 100644 --- a/docs/source/en/api/pipelines/longcat_audio_dit.md +++ b/docs/source/en/api/pipelines/longcat_audio_dit.md @@ -27,7 +27,7 @@ from diffusers import LongCatAudioDiTPipeline pipeline = LongCatAudioDiTPipeline.from_pretrained( "ruixiangma/LongCat-AudioDiT-1B-Diffusers", - torch_dtype=torch.float16, + dtype=torch.float16, ) pipeline = pipeline.to("cuda") diff --git a/docs/source/en/api/pipelines/longcat_image.md b/docs/source/en/api/pipelines/longcat_image.md index a7e8a7a3712e..30c5a76866e0 100644 --- a/docs/source/en/api/pipelines/longcat_image.md +++ b/docs/source/en/api/pipelines/longcat_image.md @@ -38,7 +38,7 @@ import diffusers from diffusers import LongCatImagePipeline weight_dtype = torch.bfloat16 -pipe = LongCatImagePipeline.from_pretrained("meituan-longcat/LongCat-Image", torch_dtype=torch.bfloat16 ) +pipe = LongCatImagePipeline.from_pretrained("meituan-longcat/LongCat-Image", dtype=torch.bfloat16 ) pipe.to('cuda') # pipe.enable_model_cpu_offload() diff --git a/docs/source/en/api/pipelines/ltx2.md b/docs/source/en/api/pipelines/ltx2.md index dab1b0e61db9..32aded9a3903 100644 --- a/docs/source/en/api/pipelines/ltx2.md +++ b/docs/source/en/api/pipelines/ltx2.md @@ -45,7 +45,7 @@ width = 768 height = 512 pipe = LTX2Pipeline.from_pretrained( - "Lightricks/LTX-2", torch_dtype=torch.bfloat16 + "Lightricks/LTX-2", dtype=torch.bfloat16 ) pipe.enable_sequential_cpu_offload(device=device) @@ -71,7 +71,7 @@ video_latent, audio_latent = pipe( latent_upsampler = LTX2LatentUpsamplerModel.from_pretrained( "Lightricks/LTX-2", subfolder="latent_upsampler", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) upsample_pipe = LTX2LatentUpsamplePipeline(vae=pipe.vae, latent_upsampler=latent_upsampler) upsample_pipe.enable_model_cpu_offload(device=device) @@ -134,7 +134,7 @@ generator = torch.Generator(device).manual_seed(random_seed) model_path = "rootonchair/LTX-2-19b-distilled" pipe = LTX2Pipeline.from_pretrained( - model_path, torch_dtype=torch.bfloat16 + model_path, dtype=torch.bfloat16 ) pipe.enable_sequential_cpu_offload(device=device) @@ -160,7 +160,7 @@ video_latent, audio_latent = pipe( latent_upsampler = LTX2LatentUpsamplerModel.from_pretrained( model_path, subfolder="latent_upsampler", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) upsample_pipe = LTX2LatentUpsamplePipeline(vae=pipe.vae, latent_upsampler=latent_upsampler) upsample_pipe.enable_model_cpu_offload(device=device) @@ -213,7 +213,7 @@ random_seed = 42 generator = torch.Generator(device).manual_seed(random_seed) model_path = "rootonchair/LTX-2-19b-distilled" -pipe = LTX2ConditionPipeline.from_pretrained(model_path, torch_dtype=torch.bfloat16) +pipe = LTX2ConditionPipeline.from_pretrained(model_path, dtype=torch.bfloat16) pipe.enable_sequential_cpu_offload(device=device) pipe.vae.enable_tiling() @@ -253,7 +253,7 @@ video_latent, audio_latent = pipe( latent_upsampler = LTX2LatentUpsamplerModel.from_pretrained( model_path, subfolder="latent_upsampler", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) upsample_pipe = LTX2LatentUpsamplePipeline(vae=pipe.vae, latent_upsampler=latent_upsampler) upsample_pipe.enable_model_cpu_offload(device=device) @@ -303,7 +303,7 @@ random_seed = 42 generator = torch.Generator(device).manual_seed(random_seed) model_path = "rootonchair/LTX-2-19b-distilled" -pipe = LTX2ConditionPipeline.from_pretrained(model_path, torch_dtype=torch.bfloat16) +pipe = LTX2ConditionPipeline.from_pretrained(model_path, dtype=torch.bfloat16) pipe.enable_sequential_cpu_offload(device=device) pipe.vae.enable_tiling() @@ -379,7 +379,7 @@ frame_rate = 24.0 generator = torch.Generator(device).manual_seed(random_seed) model_path = "diffusers/LTX-2.3-Diffusers" -pipe = LTX2ImageToVideoPipeline.from_pretrained(model_path, torch_dtype=torch.bfloat16) +pipe = LTX2ImageToVideoPipeline.from_pretrained(model_path, dtype=torch.bfloat16) pipe.enable_sequential_cpu_offload(device=device) pipe.vae.enable_tiling() @@ -451,7 +451,7 @@ frame_rate = 24.0 generator = torch.Generator(device).manual_seed(random_seed) model_path = "diffusers/LTX-2.3-Diffusers" -pipe = LTX2Pipeline.from_pretrained(model_path, torch_dtype=torch.bfloat16) +pipe = LTX2Pipeline.from_pretrained(model_path, dtype=torch.bfloat16) pipe.enable_model_cpu_offload(device=device) pipe.vae.enable_tiling() if getattr(pipe, "processor", None) is None: diff --git a/docs/source/en/api/pipelines/ltx_video.md b/docs/source/en/api/pipelines/ltx_video.md index 68658f41dabc..75abcf916d3a 100644 --- a/docs/source/en/api/pipelines/ltx_video.md +++ b/docs/source/en/api/pipelines/ltx_video.md @@ -49,13 +49,13 @@ from diffusers.utils import export_to_video transformer = AutoModel.from_pretrained( "Lightricks/LTX-Video", subfolder="transformer", - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) transformer.enable_layerwise_casting( storage_dtype=torch.float8_e4m3fn, compute_dtype=torch.bfloat16 ) -pipeline = LTXPipeline.from_pretrained("Lightricks/LTX-Video", transformer=transformer, torch_dtype=torch.bfloat16) +pipeline = LTXPipeline.from_pretrained("Lightricks/LTX-Video", transformer=transformer, dtype=torch.bfloat16) # group-offloading onload_device = torch.device("cuda") @@ -96,7 +96,7 @@ from diffusers import LTXPipeline from diffusers.utils import export_to_video pipeline = LTXPipeline.from_pretrained( - "Lightricks/LTX-Video", torch_dtype=torch.bfloat16 + "Lightricks/LTX-Video", dtype=torch.bfloat16 ) # torch.compile @@ -149,8 +149,8 @@ export_to_video(video, "output.mp4", fps=24) from diffusers.pipelines.ltx.pipeline_ltx_condition import LTXVideoCondition from diffusers.utils import export_to_video, load_video - pipeline = LTXConditionPipeline.from_pretrained("Lightricks/LTX-Video-0.9.7-dev", torch_dtype=torch.bfloat16) - pipeline_upsample = LTXLatentUpsamplePipeline.from_pretrained("Lightricks/ltxv-spatial-upscaler-0.9.7", vae=pipeline.vae, torch_dtype=torch.bfloat16) + pipeline = LTXConditionPipeline.from_pretrained("Lightricks/LTX-Video-0.9.7-dev", dtype=torch.bfloat16) + pipeline_upsample = LTXLatentUpsamplePipeline.from_pretrained("Lightricks/ltxv-spatial-upscaler-0.9.7", vae=pipeline.vae, dtype=torch.bfloat16) pipeline.to("cuda") pipe_upsample.to("cuda") pipeline.vae.enable_tiling() @@ -247,8 +247,8 @@ export_to_video(video, "output.mp4", fps=24) from diffusers.pipelines.ltx.pipeline_ltx_condition import LTXVideoCondition from diffusers.utils import export_to_video, load_video - pipeline = LTXConditionPipeline.from_pretrained("Lightricks/LTX-Video-0.9.7-distilled", torch_dtype=torch.bfloat16) - pipe_upsample = LTXLatentUpsamplePipeline.from_pretrained("Lightricks/ltxv-spatial-upscaler-0.9.7", vae=pipeline.vae, torch_dtype=torch.bfloat16) + pipeline = LTXConditionPipeline.from_pretrained("Lightricks/LTX-Video-0.9.7-distilled", dtype=torch.bfloat16) + pipe_upsample = LTXLatentUpsamplePipeline.from_pretrained("Lightricks/ltxv-spatial-upscaler-0.9.7", vae=pipeline.vae, dtype=torch.bfloat16) pipeline.to("cuda") pipe_upsample.to("cuda") pipeline.vae.enable_tiling() @@ -337,9 +337,9 @@ export_to_video(video, "output.mp4", fps=24) from diffusers.pipelines.ltx.modeling_latent_upsampler import LTXLatentUpsamplerModel from diffusers.utils import export_to_video, load_video - pipeline = LTXConditionPipeline.from_pretrained("Lightricks/LTX-Video-0.9.8-13B-distilled", torch_dtype=torch.bfloat16) + pipeline = LTXConditionPipeline.from_pretrained("Lightricks/LTX-Video-0.9.8-13B-distilled", dtype=torch.bfloat16) # TODO: Update the checkpoint here once updated in LTX org - upsampler = LTXLatentUpsamplerModel.from_pretrained("a-r-r-o-w/LTX-0.9.8-Latent-Upsampler", torch_dtype=torch.bfloat16) + upsampler = LTXLatentUpsamplerModel.from_pretrained("a-r-r-o-w/LTX-0.9.8-Latent-Upsampler", dtype=torch.bfloat16) pipe_upsample = LTXLatentUpsamplePipeline(vae=pipeline.vae, latent_upsampler=upsampler).to(torch.bfloat16) pipeline.to("cuda") pipe_upsample.to("cuda") @@ -425,7 +425,7 @@ export_to_video(video, "output.mp4", fps=24) from diffusers.utils import export_to_video, load_image pipeline = LTXConditionPipeline.from_pretrained( - "Lightricks/LTX-Video-0.9.5", torch_dtype=torch.bfloat16 + "Lightricks/LTX-Video-0.9.5", dtype=torch.bfloat16 ) pipeline.load_lora_weights("Lightricks/LTX-Video-Cakeify-LoRA", adapter_name="cakeify") @@ -463,12 +463,12 @@ export_to_video(video, "output.mp4", fps=24) transformer = AutoModel.from_single_file( "https://huggingface.co/city96/LTX-Video-gguf/blob/main/ltx-video-2b-v0.9-Q3_K_S.gguf" quantization_config=GGUFQuantizationConfig(compute_dtype=torch.bfloat16), - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) pipeline = LTXPipeline.from_pretrained( "Lightricks/LTX-Video", transformer=transformer, - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) ``` diff --git a/docs/source/en/api/pipelines/lumina.md b/docs/source/en/api/pipelines/lumina.md index 0a236d213d6c..cd6f1c220939 100644 --- a/docs/source/en/api/pipelines/lumina.md +++ b/docs/source/en/api/pipelines/lumina.md @@ -59,7 +59,7 @@ from diffusers import LuminaPipeline import torch pipeline = LuminaPipeline.from_pretrained( - "Alpha-VLLM/Lumina-Next-SFT-diffusers", torch_dtype=torch.bfloat16 + "Alpha-VLLM/Lumina-Next-SFT-diffusers", dtype=torch.bfloat16 ).to("cuda") ``` @@ -95,7 +95,7 @@ text_encoder_8bit = T5EncoderModel.from_pretrained( "Alpha-VLLM/Lumina-Next-SFT-diffusers", subfolder="text_encoder", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) quant_config = DiffusersBitsAndBytesConfig(load_in_8bit=True) @@ -103,14 +103,14 @@ transformer_8bit = Transformer2DModel.from_pretrained( "Alpha-VLLM/Lumina-Next-SFT-diffusers", subfolder="transformer", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) pipeline = LuminaPipeline.from_pretrained( "Alpha-VLLM/Lumina-Next-SFT-diffusers", text_encoder=text_encoder_8bit, transformer=transformer_8bit, - torch_dtype=torch.float16, + dtype=torch.float16, device_map="balanced", ) diff --git a/docs/source/en/api/pipelines/lumina2.md b/docs/source/en/api/pipelines/lumina2.md index 0c4e793404fe..43907b57f9b6 100644 --- a/docs/source/en/api/pipelines/lumina2.md +++ b/docs/source/en/api/pipelines/lumina2.md @@ -37,11 +37,11 @@ from diffusers import Lumina2Transformer2DModel, Lumina2Pipeline ckpt_path = "https://huggingface.co/Alpha-VLLM/Lumina-Image-2.0/blob/main/consolidated.00-of-01.pth" transformer = Lumina2Transformer2DModel.from_single_file( - ckpt_path, torch_dtype=torch.bfloat16 + ckpt_path, dtype=torch.bfloat16 ) pipe = Lumina2Pipeline.from_pretrained( - "Alpha-VLLM/Lumina-Image-2.0", transformer=transformer, torch_dtype=torch.bfloat16 + "Alpha-VLLM/Lumina-Image-2.0", transformer=transformer, dtype=torch.bfloat16 ) pipe.enable_model_cpu_offload() image = pipe( @@ -63,11 +63,11 @@ ckpt_path = "https://huggingface.co/calcuis/lumina-gguf/blob/main/lumina2-q4_0.g transformer = Lumina2Transformer2DModel.from_single_file( ckpt_path, quantization_config=GGUFQuantizationConfig(compute_dtype=torch.bfloat16), - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) pipe = Lumina2Pipeline.from_pretrained( - "Alpha-VLLM/Lumina-Image-2.0", transformer=transformer, torch_dtype=torch.bfloat16 + "Alpha-VLLM/Lumina-Image-2.0", transformer=transformer, dtype=torch.bfloat16 ) pipe.enable_model_cpu_offload() image = pipe( diff --git a/docs/source/en/api/pipelines/marigold.md b/docs/source/en/api/pipelines/marigold.md index 521afebf0ad5..45320fbd9c80 100644 --- a/docs/source/en/api/pipelines/marigold.md +++ b/docs/source/en/api/pipelines/marigold.md @@ -117,7 +117,7 @@ import diffusers import torch pipe = diffusers.MarigoldDepthPipeline.from_pretrained( - "prs-eth/marigold-depth-v1-1", variant="fp16", torch_dtype=torch.float16 + "prs-eth/marigold-depth-v1-1", variant="fp16", dtype=torch.float16 ).to("cuda") image = diffusers.utils.load_image("https://marigoldmonodepth.github.io/images/einstein.jpg") @@ -164,7 +164,7 @@ import diffusers import torch pipe = diffusers.MarigoldNormalsPipeline.from_pretrained( - "prs-eth/marigold-normals-v1-1", variant="fp16", torch_dtype=torch.float16 + "prs-eth/marigold-normals-v1-1", variant="fp16", dtype=torch.float16 ).to("cuda") image = diffusers.utils.load_image("https://marigoldmonodepth.github.io/images/einstein.jpg") @@ -214,7 +214,7 @@ import diffusers import torch pipe = diffusers.MarigoldIntrinsicsPipeline.from_pretrained( - "prs-eth/marigold-iid-appearance-v1-1", variant="fp16", torch_dtype=torch.float16 + "prs-eth/marigold-iid-appearance-v1-1", variant="fp16", dtype=torch.float16 ).to("cuda") image = diffusers.utils.load_image("https://marigoldmonodepth.github.io/images/einstein.jpg") @@ -234,7 +234,7 @@ import diffusers import torch pipe = diffusers.MarigoldIntrinsicsPipeline.from_pretrained( - "prs-eth/marigold-iid-lighting-v1-1", variant="fp16", torch_dtype=torch.float16 + "prs-eth/marigold-iid-lighting-v1-1", variant="fp16", dtype=torch.float16 ).to("cuda") image = diffusers.utils.load_image("https://marigoldmonodepth.github.io/images/einstein.jpg") @@ -283,7 +283,7 @@ steps to the minimum: import torch pipe = diffusers.MarigoldDepthPipeline.from_pretrained( - "prs-eth/marigold-depth-v1-1", variant="fp16", torch_dtype=torch.float16 + "prs-eth/marigold-depth-v1-1", variant="fp16", dtype=torch.float16 ).to("cuda") image = diffusers.utils.load_image("https://marigoldmonodepth.github.io/images/einstein.jpg") @@ -306,11 +306,11 @@ Note that using a lightweight VAE may slightly reduce the visual quality of the import torch pipe = diffusers.MarigoldDepthPipeline.from_pretrained( - "prs-eth/marigold-depth-v1-1", variant="fp16", torch_dtype=torch.float16 + "prs-eth/marigold-depth-v1-1", variant="fp16", dtype=torch.float16 ).to("cuda") + pipe.vae = diffusers.AutoencoderTiny.from_pretrained( -+ "madebyollin/taesd", torch_dtype=torch.float16 ++ "madebyollin/taesd", dtype=torch.float16 + ).cuda() image = diffusers.utils.load_image("https://marigoldmonodepth.github.io/images/einstein.jpg") @@ -328,7 +328,7 @@ Speeding them up can be achieved by using a more efficient attention processor: + from diffusers.models.attention_processor import AttnProcessor2_0 pipe = diffusers.MarigoldDepthPipeline.from_pretrained( - "prs-eth/marigold-depth-v1-1", variant="fp16", torch_dtype=torch.float16 + "prs-eth/marigold-depth-v1-1", variant="fp16", dtype=torch.float16 ).to("cuda") + pipe.vae.set_attn_processor(AttnProcessor2_0()) @@ -350,7 +350,7 @@ the same pipeline instance is called repeatedly, such as within a loop. from diffusers.models.attention_processor import AttnProcessor2_0 pipe = diffusers.MarigoldDepthPipeline.from_pretrained( - "prs-eth/marigold-depth-v1-1", variant="fp16", torch_dtype=torch.float16 + "prs-eth/marigold-depth-v1-1", variant="fp16", dtype=torch.float16 ).to("cuda") pipe.vae.set_attn_processor(AttnProcessor2_0()) @@ -442,10 +442,10 @@ path_in = "https://huggingface.co/spaces/prs-eth/marigold-lcm/resolve/c7adb54279 path_out = "obama_depth.gif" pipe = diffusers.MarigoldDepthPipeline.from_pretrained( - "prs-eth/marigold-depth-v1-1", variant="fp16", torch_dtype=torch.float16 + "prs-eth/marigold-depth-v1-1", variant="fp16", dtype=torch.float16 ).to(device) pipe.vae = diffusers.AutoencoderTiny.from_pretrained( - "madebyollin/taesd", torch_dtype=torch.float16 + "madebyollin/taesd", dtype=torch.float16 ).to(device) pipe.unet.set_attn_processor(AttnProcessor2_0()) pipe.vae = torch.compile(pipe.vae, mode="reduce-overhead", fullgraph=True) @@ -513,7 +513,7 @@ image = diffusers.utils.load_image( ) pipe = diffusers.MarigoldDepthPipeline.from_pretrained( - "prs-eth/marigold-depth-v1-1", torch_dtype=torch.float16, variant="fp16" + "prs-eth/marigold-depth-v1-1", dtype=torch.float16, variant="fp16" ).to(device) depth_image = pipe(image, generator=generator).prediction @@ -521,10 +521,10 @@ depth_image = pipe.image_processor.visualize_depth(depth_image, color_map="binar depth_image[0].save("motorcycle_controlnet_depth.png") controlnet = diffusers.ControlNetModel.from_pretrained( - "diffusers/controlnet-depth-sdxl-1.0", torch_dtype=torch.float16, variant="fp16" + "diffusers/controlnet-depth-sdxl-1.0", dtype=torch.float16, variant="fp16" ).to(device) pipe = diffusers.StableDiffusionXLControlNetPipeline.from_pretrained( - "SG161222/RealVisXL_V4.0", torch_dtype=torch.float16, variant="fp16", controlnet=controlnet + "SG161222/RealVisXL_V4.0", dtype=torch.float16, variant="fp16", controlnet=controlnet ).to(device) pipe.scheduler = diffusers.DPMSolverMultistepScheduler.from_config(pipe.scheduler.config, use_karras_sigmas=True) @@ -606,7 +606,7 @@ import diffusers import torch pipe = diffusers.MarigoldDepthPipeline.from_pretrained( - "prs-eth/marigold-depth-v1-1", variant="fp16", torch_dtype=torch.float16 + "prs-eth/marigold-depth-v1-1", variant="fp16", dtype=torch.float16 ).to("cuda") image = diffusers.utils.load_image("https://marigoldmonodepth.github.io/images/einstein.jpg") diff --git a/docs/source/en/api/pipelines/mochi.md b/docs/source/en/api/pipelines/mochi.md index f9daa3515774..50b182ce33c7 100644 --- a/docs/source/en/api/pipelines/mochi.md +++ b/docs/source/en/api/pipelines/mochi.md @@ -46,7 +46,7 @@ text_encoder_8bit = T5EncoderModel.from_pretrained( "genmo/mochi-1-preview", subfolder="text_encoder", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) quant_config = DiffusersBitsAndBytesConfig(load_in_8bit=True) @@ -54,14 +54,14 @@ transformer_8bit = MochiTransformer3DModel.from_pretrained( "genmo/mochi-1-preview", subfolder="transformer", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) pipeline = MochiPipeline.from_pretrained( "genmo/mochi-1-preview", text_encoder=text_encoder_8bit, transformer=transformer_8bit, - torch_dtype=torch.float16, + dtype=torch.float16, device_map="balanced", ) @@ -105,7 +105,7 @@ import torch from diffusers import MochiPipeline from diffusers.utils import export_to_video -pipe = MochiPipeline.from_pretrained("genmo/mochi-1-preview", variant="bf16", torch_dtype=torch.bfloat16) +pipe = MochiPipeline.from_pretrained("genmo/mochi-1-preview", variant="bf16", dtype=torch.bfloat16) # Enable memory savings pipe.enable_model_cpu_offload() @@ -241,7 +241,7 @@ model_id = "genmo/mochi-1-preview" ckpt_path = "https://huggingface.co/Comfy-Org/mochi_preview_repackaged/blob/main/split_files/diffusion_models/mochi_preview_bf16.safetensors" -transformer = MochiTransformer3DModel.from_pretrained(ckpt_path, torch_dtype=torch.bfloat16) +transformer = MochiTransformer3DModel.from_pretrained(ckpt_path, dtype=torch.bfloat16) pipe = MochiPipeline.from_pretrained(model_id, transformer=transformer) pipe.enable_model_cpu_offload() diff --git a/docs/source/en/api/pipelines/motif_video.md b/docs/source/en/api/pipelines/motif_video.md index 9e0929599ea2..a9343240c472 100644 --- a/docs/source/en/api/pipelines/motif_video.md +++ b/docs/source/en/api/pipelines/motif_video.md @@ -22,7 +22,7 @@ from diffusers.utils import export_to_video pipe = MotifVideoPipeline.from_pretrained( "Motif-Technologies/Motif-Video-2B", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) pipe.to("cuda") @@ -52,7 +52,7 @@ from diffusers.utils import export_to_video, load_image pipe = MotifVideoImage2VideoPipeline.from_pretrained( "Motif-Technologies/Motif-Video-2B", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) pipe.to("cuda") @@ -88,7 +88,7 @@ from diffusers.utils import export_to_video pipe = MotifVideoPipeline.from_pretrained( "Motif-Technologies/Motif-Video-2B", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) pipe.enable_model_cpu_offload() diff --git a/docs/source/en/api/pipelines/omnigen.md b/docs/source/en/api/pipelines/omnigen.md index 8ff30c7ab6f8..7952aca584dc 100644 --- a/docs/source/en/api/pipelines/omnigen.md +++ b/docs/source/en/api/pipelines/omnigen.md @@ -34,7 +34,7 @@ Model weights may be stored in separate subfolders on the Hub or locally, in whi import torch from diffusers import OmniGenPipeline -pipe = OmniGenPipeline.from_pretrained("Shitao/OmniGen-v1-diffusers", torch_dtype=torch.bfloat16) +pipe = OmniGenPipeline.from_pretrained("Shitao/OmniGen-v1-diffusers", dtype=torch.bfloat16) ``` ## Text-to-image @@ -48,7 +48,7 @@ from diffusers import OmniGenPipeline pipe = OmniGenPipeline.from_pretrained( "Shitao/OmniGen-v1-diffusers", - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) pipe.to("cuda") @@ -80,7 +80,7 @@ from diffusers.utils import load_image pipe = OmniGenPipeline.from_pretrained( "Shitao/OmniGen-v1-diffusers", - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) pipe.to("cuda") @@ -139,7 +139,7 @@ from diffusers.utils import load_image pipe = OmniGenPipeline.from_pretrained( "Shitao/OmniGen-v1-diffusers", - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) pipe.to("cuda") @@ -193,7 +193,7 @@ from diffusers.utils import load_image pipe = OmniGenPipeline.from_pretrained( "Shitao/OmniGen-v1-diffusers", - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) pipe.to("cuda") @@ -229,7 +229,7 @@ from diffusers.utils import load_image pipe = OmniGenPipeline.from_pretrained( "Shitao/OmniGen-v1-diffusers", - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) pipe.to("cuda") @@ -271,7 +271,7 @@ from diffusers.utils import load_image pipe = OmniGenPipeline.from_pretrained( "Shitao/OmniGen-v1-diffusers", - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) pipe.to("cuda") diff --git a/docs/source/en/api/pipelines/pag.md b/docs/source/en/api/pipelines/pag.md index 72d9a773ff79..b670b5ff50f8 100644 --- a/docs/source/en/api/pipelines/pag.md +++ b/docs/source/en/api/pipelines/pag.md @@ -53,7 +53,7 @@ pipeline = AutoPipelineForText2Image.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", enable_pag=True, pag_applied_layers=["mid"], - torch_dtype=torch.float16 + dtype=torch.float16 ) pipeline.enable_model_cpu_offload() ``` @@ -64,7 +64,7 @@ pipeline.enable_model_cpu_offload() If you already have a pipeline created and loaded, you can enable PAG on it using the `from_pipe` API with the `enable_pag` flag. Internally, a PAG pipeline is created based on the pipeline and task you specified. In the example below, since we used `AutoPipelineForText2Image` and passed a `StableDiffusionXLPipeline`, a `StableDiffusionXLPAGPipeline` is created accordingly. Note that this does not require additional memory, and you will have both `StableDiffusionXLPipeline` and `StableDiffusionXLPAGPipeline` loaded and ready to use. You can read more about the `from_pipe` API and how to reuse pipelines in diffuser [here](https://huggingface.co/docs/diffusers/using-diffusers/loading#reuse-a-pipeline). ```py -pipeline_sdxl = AutoPipelineForText2Image.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16) +pipeline_sdxl = AutoPipelineForText2Image.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16) pipeline = AutoPipelineForText2Image.from_pipe(pipeline_sdxl, enable_pag=True) ``` @@ -109,7 +109,7 @@ pipeline = AutoPipelineForImage2Image.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", enable_pag=True, pag_applied_layers=["mid"], - torch_dtype=torch.float16 + dtype=torch.float16 ) pipeline.enable_model_cpu_offload() ``` @@ -117,21 +117,21 @@ pipeline.enable_model_cpu_offload() If you already have a image-to-image pipeline and would like enable PAG on it, you can run this ```py -pipeline_t2i = AutoPipelineForImage2Image.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16) +pipeline_t2i = AutoPipelineForImage2Image.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16) pipeline = AutoPipelineForImage2Image.from_pipe(pipeline_t2i, enable_pag=True) ``` It is also very easy to directly switch from a text-to-image pipeline to PAG enabled image-to-image pipeline ```py -pipeline_pag = AutoPipelineForText2Image.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16) +pipeline_pag = AutoPipelineForText2Image.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16) pipeline = AutoPipelineForImage2Image.from_pipe(pipeline_t2i, enable_pag=True) ``` If you have a PAG enabled text-to-image pipeline, you can directly switch to a image-to-image pipeline with PAG still enabled ```py -pipeline_pag = AutoPipelineForText2Image.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", enable_pag=True, torch_dtype=torch.float16) +pipeline_pag = AutoPipelineForText2Image.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", enable_pag=True, dtype=torch.float16) pipeline = AutoPipelineForImage2Image.from_pipe(pipeline_t2i) ``` @@ -166,7 +166,7 @@ import torch pipeline = AutoPipelineForInpainting.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", enable_pag=True, - torch_dtype=torch.float16 + dtype=torch.float16 ) pipeline.enable_model_cpu_offload() ``` @@ -174,14 +174,14 @@ pipeline.enable_model_cpu_offload() You can enable PAG on an existing inpainting pipeline like this ```py -pipeline_inpaint = AutoPipelineForInpainting.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16) +pipeline_inpaint = AutoPipelineForInpainting.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16) pipeline = AutoPipelineForInpainting.from_pipe(pipeline_inpaint, enable_pag=True) ``` This still works when your pipeline has a different task: ```py -pipeline_t2i = AutoPipelineForText2Image.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16) +pipeline_t2i = AutoPipelineForText2Image.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16) pipeline = AutoPipelineForInpaiting.from_pipe(pipeline_t2i, enable_pag=True) ``` @@ -223,7 +223,7 @@ from diffusers import AutoPipelineForText2Image, ControlNetModel import torch controlnet = ControlNetModel.from_pretrained( - "diffusers/controlnet-canny-sdxl-1.0", torch_dtype=torch.float16 + "diffusers/controlnet-canny-sdxl-1.0", dtype=torch.float16 ) pipeline = AutoPipelineForText2Image.from_pretrained( @@ -231,7 +231,7 @@ pipeline = AutoPipelineForText2Image.from_pretrained( controlnet=controlnet, enable_pag=True, pag_applied_layers="mid", - torch_dtype=torch.float16 + dtype=torch.float16 ) pipeline.enable_model_cpu_offload() ``` @@ -285,14 +285,14 @@ import torch image_encoder = CLIPVisionModelWithProjection.from_pretrained( "h94/IP-Adapter", subfolder="models/image_encoder", - torch_dtype=torch.float16 + dtype=torch.float16 ) pipeline = AutoPipelineForText2Image.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", image_encoder=image_encoder, enable_pag=True, - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.load_ip_adapter("h94/IP-Adapter", subfolder="sdxl_models", weight_name="ip-adapter-plus_sdxl_vit-h.bin") diff --git a/docs/source/en/api/pipelines/pixart.md b/docs/source/en/api/pipelines/pixart.md index dbdc89857e5e..843b629d4a24 100644 --- a/docs/source/en/api/pipelines/pixart.md +++ b/docs/source/en/api/pipelines/pixart.md @@ -92,7 +92,7 @@ Then compute the latents with the prompt embeddings as inputs: pipe = PixArtAlphaPipeline.from_pretrained( "PixArt-alpha/PixArt-XL-2-1024-MS", text_encoder=None, - torch_dtype=torch.float16, + dtype=torch.float16, ).to("cuda") latents = pipe( diff --git a/docs/source/en/api/pipelines/pixart_sigma.md b/docs/source/en/api/pipelines/pixart_sigma.md index 43546daae1f9..a2d378d0586a 100644 --- a/docs/source/en/api/pipelines/pixart_sigma.md +++ b/docs/source/en/api/pipelines/pixart_sigma.md @@ -96,7 +96,7 @@ Then compute the latents with the prompt embeddings as inputs: pipe = PixArtSigmaPipeline.from_pretrained( "PixArt-alpha/PixArt-Sigma-XL-2-1024-MS", text_encoder=None, - torch_dtype=torch.float16, + dtype=torch.float16, ).to("cuda") latents = pipe( diff --git a/docs/source/en/api/pipelines/prx.md b/docs/source/en/api/pipelines/prx.md index 16670f4bfc86..90a4eae0d123 100644 --- a/docs/source/en/api/pipelines/prx.md +++ b/docs/source/en/api/pipelines/prx.md @@ -43,7 +43,7 @@ Load the pipeline with [`~DiffusionPipeline.from_pretrained`]. from diffusers.pipelines.prx import PRXPipeline # Load pipeline - VAE and text encoder will be loaded from HuggingFace -pipe = PRXPipeline.from_pretrained("Photoroom/prx-512-t2i-sft", torch_dtype=torch.bfloat16) +pipe = PRXPipeline.from_pretrained("Photoroom/prx-512-t2i-sft", dtype=torch.bfloat16) pipe.to("cuda") prompt = "A front-facing portrait of a lion the golden savanna at sunset." @@ -71,7 +71,7 @@ transformer = PRXTransformer2DModel.from_pretrained( "checkpoints/prx-512-t2i-sft", subfolder="transformer", quantization_config=quant_config, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) # Load scheduler @@ -82,7 +82,7 @@ scheduler = FlowMatchEulerDiscreteScheduler.from_pretrained( # Load T5Gemma text encoder t5gemma_model = T5GemmaModel.from_pretrained("google/t5gemma-2b-2b-ul2", quantization_config=quant_config, - torch_dtype=torch.bfloat16) + dtype=torch.bfloat16) text_encoder = t5gemma_model.encoder.to(dtype=torch.bfloat16) tokenizer = GemmaTokenizerFast.from_pretrained("google/t5gemma-2b-2b-ul2") tokenizer.model_max_length = 256 @@ -92,7 +92,7 @@ tokenizer.model_max_length = 256 vae = AutoencoderKL.from_pretrained("black-forest-labs/FLUX.1-dev", subfolder="vae", quantization_config=quant_config, - torch_dtype=torch.bfloat16) + dtype=torch.bfloat16) pipe = PRXPipeline( transformer=transformer, @@ -113,7 +113,7 @@ For memory-constrained environments: import torch from diffusers.pipelines.prx import PRXPipeline -pipe = PRXPipeline.from_pretrained("Photoroom/prx-512-t2i-sft", torch_dtype=torch.bfloat16) +pipe = PRXPipeline.from_pretrained("Photoroom/prx-512-t2i-sft", dtype=torch.bfloat16) pipe.enable_model_cpu_offload() # Offload components to CPU when not in use # Or use sequential CPU offload for even lower memory diff --git a/docs/source/en/api/pipelines/prx_pixel.md b/docs/source/en/api/pipelines/prx_pixel.md index 6342726efd1c..3bb5d38d72f1 100644 --- a/docs/source/en/api/pipelines/prx_pixel.md +++ b/docs/source/en/api/pipelines/prx_pixel.md @@ -33,7 +33,7 @@ generation resolution is fed into the timestep modulation so the model is aware import torch from diffusers import PRXPixelPipeline -pipe = PRXPixelPipeline.from_pretrained("Photoroom/prxpixel-t2i", torch_dtype=torch.bfloat16) +pipe = PRXPixelPipeline.from_pretrained("Photoroom/prxpixel-t2i", dtype=torch.bfloat16) pipe.to("cuda") prompt = "A front-facing portrait of a lion in the golden savanna at sunset." @@ -49,7 +49,7 @@ For memory-constrained environments: import torch from diffusers import PRXPixelPipeline -pipe = PRXPixelPipeline.from_pretrained("Photoroom/prxpixel-t2i", torch_dtype=torch.bfloat16) +pipe = PRXPixelPipeline.from_pretrained("Photoroom/prxpixel-t2i", dtype=torch.bfloat16) pipe.enable_model_cpu_offload() # Or use sequential CPU offload for even lower memory diff --git a/docs/source/en/api/pipelines/qwenimage.md b/docs/source/en/api/pipelines/qwenimage.md index c0994c8685d0..d0b78ae2658e 100644 --- a/docs/source/en/api/pipelines/qwenimage.md +++ b/docs/source/en/api/pipelines/qwenimage.md @@ -66,7 +66,7 @@ scheduler_config = { } scheduler = FlowMatchEulerDiscreteScheduler.from_config(scheduler_config) pipe = DiffusionPipeline.from_pretrained( - ckpt_id, scheduler=scheduler, torch_dtype=torch.bfloat16 + ckpt_id, scheduler=scheduler, dtype=torch.bfloat16 ).to("cuda") pipe.load_lora_weights( "lightx2v/Qwen-Image-Lightning", weight_name="Qwen-Image-Lightning-8steps-V1.0.safetensors" @@ -102,7 +102,7 @@ from diffusers import QwenImageEditPlusPipeline from diffusers.utils import load_image pipe = QwenImageEditPlusPipeline.from_pretrained( - "Qwen/Qwen-Image-Edit-2509", torch_dtype=torch.bfloat16 + "Qwen/Qwen-Image-Edit-2509", dtype=torch.bfloat16 ).to("cuda") image_1 = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/grumpy.jpg") @@ -124,7 +124,7 @@ Using `torch.compile` on the transformer provides ~2.4x speedup (A100 80GB: 4.70 import torch from diffusers import QwenImagePipeline -pipe = QwenImagePipeline.from_pretrained("Qwen/Qwen-Image", torch_dtype=torch.bfloat16).to("cuda") +pipe = QwenImagePipeline.from_pretrained("Qwen/Qwen-Image", dtype=torch.bfloat16).to("cuda") pipe.transformer = torch.compile(pipe.transformer) # First call triggers compilation (~7s overhead) diff --git a/docs/source/en/api/pipelines/sana.md b/docs/source/en/api/pipelines/sana.md index a948620f96cb..61c7aebe515e 100644 --- a/docs/source/en/api/pipelines/sana.md +++ b/docs/source/en/api/pipelines/sana.md @@ -47,7 +47,7 @@ Refer to [this](https://huggingface.co/collections/Efficient-Large-Model/sana-67 Note: The recommended dtype mentioned is for the transformer weights. The text encoder and VAE weights must stay in `torch.bfloat16` or `torch.float32` for the model to work correctly. Please refer to the inference example below to see how to load the model with the recommended dtype. > [!TIP] -> Make sure to pass the `variant` argument for downloaded checkpoints to use lower disk space. Set it to `"fp16"` for models with recommended dtype as `torch.float16`, and `"bf16"` for models with recommended dtype as `torch.bfloat16`. By default, `torch.float32` weights are downloaded, which use twice the amount of disk storage. Additionally, `torch.float32` weights can be downcasted on-the-fly by specifying the `torch_dtype` argument. Read about it in the [docs](https://huggingface.co/docs/diffusers/v0.31.0/en/api/pipelines/overview#diffusers.DiffusionPipeline.from_pretrained). +> Make sure to pass the `variant` argument for downloaded checkpoints to use lower disk space. Set it to `"fp16"` for models with recommended dtype as `torch.float16`, and `"bf16"` for models with recommended dtype as `torch.bfloat16`. By default, `torch.float32` weights are downloaded, which use twice the amount of disk storage. Additionally, `torch.float32` weights can be downcasted on-the-fly by specifying the `dtype` argument. Read about it in the [docs](https://huggingface.co/docs/diffusers/v0.31.0/en/api/pipelines/overview#diffusers.DiffusionPipeline.from_pretrained). ## Quantization @@ -65,7 +65,7 @@ text_encoder_8bit = AutoModel.from_pretrained( "Efficient-Large-Model/Sana_1600M_1024px_diffusers", subfolder="text_encoder", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) quant_config = DiffusersBitsAndBytesConfig(load_in_8bit=True) @@ -73,14 +73,14 @@ transformer_8bit = SanaTransformer2DModel.from_pretrained( "Efficient-Large-Model/Sana_1600M_1024px_diffusers", subfolder="transformer", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) pipeline = SanaPipeline.from_pretrained( "Efficient-Large-Model/Sana_1600M_1024px_diffusers", text_encoder=text_encoder_8bit, transformer=transformer_8bit, - torch_dtype=torch.float16, + dtype=torch.float16, device_map="balanced", ) diff --git a/docs/source/en/api/pipelines/sana_sprint.md b/docs/source/en/api/pipelines/sana_sprint.md index 46cdc13302ec..8cf29df9c294 100644 --- a/docs/source/en/api/pipelines/sana_sprint.md +++ b/docs/source/en/api/pipelines/sana_sprint.md @@ -54,7 +54,7 @@ text_encoder_8bit = AutoModel.from_pretrained( "Efficient-Large-Model/Sana_Sprint_1.6B_1024px_diffusers", subfolder="text_encoder", quantization_config=quant_config, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) quant_config = DiffusersBitsAndBytesConfig(load_in_8bit=True) @@ -62,14 +62,14 @@ transformer_8bit = SanaTransformer2DModel.from_pretrained( "Efficient-Large-Model/Sana_Sprint_1.6B_1024px_diffusers", subfolder="transformer", quantization_config=quant_config, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) pipeline = SanaSprintPipeline.from_pretrained( "Efficient-Large-Model/Sana_Sprint_1.6B_1024px_diffusers", text_encoder=text_encoder_8bit, transformer=transformer_8bit, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, device_map="balanced", ) @@ -97,7 +97,7 @@ image = load_image( pipe = SanaSprintImg2ImgPipeline.from_pretrained( "Efficient-Large-Model/Sana_Sprint_1.6B_1024px_diffusers", - torch_dtype=torch.bfloat16) + dtype=torch.bfloat16) pipe.to("cuda") image = pipe( diff --git a/docs/source/en/api/pipelines/sana_video.md b/docs/source/en/api/pipelines/sana_video.md index 9e330c758318..f710736787d1 100644 --- a/docs/source/en/api/pipelines/sana_video.md +++ b/docs/source/en/api/pipelines/sana_video.md @@ -48,7 +48,7 @@ The example below demonstrates how to use the text-to-video pipeline to generate ```python pipe = SanaVideoPipeline.from_pretrained( "Efficient-Large-Model/SANA-Video_2B_480p_diffusers", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) pipe.text_encoder.to(torch.bfloat16) pipe.vae.to(torch.float32) @@ -82,7 +82,7 @@ The example below demonstrates how to use the image-to-video pipeline to generat ```python pipe = SanaImageToVideoPipeline.from_pretrained( "Efficient-Large-Model/SANA-Video_2B_480p_diffusers", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) pipe.scheduler = FlowMatchEulerDiscreteScheduler.from_config(pipe.scheduler.config, flow_shift=8.0) pipe.vae.to(torch.float32) @@ -133,7 +133,7 @@ text_encoder_8bit = AutoModel.from_pretrained( "Efficient-Large-Model/SANA-Video_2B_480p_diffusers", subfolder="text_encoder", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) quant_config = DiffusersBitsAndBytesConfig(load_in_8bit=True) @@ -141,14 +141,14 @@ transformer_8bit = SanaVideoTransformer3DModel.from_pretrained( "Efficient-Large-Model/SANA-Video_2B_480p_diffusers", subfolder="transformer", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) pipeline = SanaVideoPipeline.from_pretrained( "Efficient-Large-Model/SANA-Video_2B_480p_diffusers", text_encoder=text_encoder_8bit, transformer=transformer_8bit, - torch_dtype=torch.float16, + dtype=torch.float16, device_map="balanced", ) diff --git a/docs/source/en/api/pipelines/shap_e.md b/docs/source/en/api/pipelines/shap_e.md index cb9e4353b131..2572c268b446 100644 --- a/docs/source/en/api/pipelines/shap_e.md +++ b/docs/source/en/api/pipelines/shap_e.md @@ -37,7 +37,7 @@ from diffusers import ShapEPipeline device = torch.device("cuda" if torch.cuda.is_available() else "cpu") -pipe = ShapEPipeline.from_pretrained("openai/shap-e", torch_dtype=torch.float16, variant="fp16") +pipe = ShapEPipeline.from_pretrained("openai/shap-e", dtype=torch.float16, variant="fp16") pipe = pipe.to(device) guidance_scale = 15.0 @@ -79,8 +79,8 @@ To generate a 3D object from another image, use the [`ShapEImg2ImgPipeline`]. Yo from diffusers import DiffusionPipeline import torch -prior_pipeline = DiffusionPipeline.from_pretrained("kandinsky-community/kandinsky-2-1-prior", torch_dtype=torch.float16, use_safetensors=True).to("cuda") -pipeline = DiffusionPipeline.from_pretrained("kandinsky-community/kandinsky-2-1", torch_dtype=torch.float16, use_safetensors=True).to("cuda") +prior_pipeline = DiffusionPipeline.from_pretrained("kandinsky-community/kandinsky-2-1-prior", dtype=torch.float16, use_safetensors=True).to("cuda") +pipeline = DiffusionPipeline.from_pretrained("kandinsky-community/kandinsky-2-1", dtype=torch.float16, use_safetensors=True).to("cuda") prompt = "A cheeseburger, white background" @@ -101,7 +101,7 @@ from PIL import Image from diffusers import ShapEImg2ImgPipeline from diffusers.utils import export_to_gif -pipe = ShapEImg2ImgPipeline.from_pretrained("openai/shap-e-img2img", torch_dtype=torch.float16, variant="fp16").to("cuda") +pipe = ShapEImg2ImgPipeline.from_pretrained("openai/shap-e-img2img", dtype=torch.float16, variant="fp16").to("cuda") guidance_scale = 3.0 image = Image.open("burger.png").resize((256, 256)) @@ -139,7 +139,7 @@ from diffusers import ShapEPipeline device = torch.device("cuda" if torch.cuda.is_available() else "cpu") -pipe = ShapEPipeline.from_pretrained("openai/shap-e", torch_dtype=torch.float16, variant="fp16") +pipe = ShapEPipeline.from_pretrained("openai/shap-e", dtype=torch.float16, variant="fp16") pipe = pipe.to(device) guidance_scale = 15.0 diff --git a/docs/source/en/api/pipelines/skyreels_v2.md b/docs/source/en/api/pipelines/skyreels_v2.md index e1829bc409eb..d05b1689ede2 100644 --- a/docs/source/en/api/pipelines/skyreels_v2.md +++ b/docs/source/en/api/pipelines/skyreels_v2.md @@ -172,12 +172,12 @@ from diffusers.utils import export_to_video model_id = "Skywork/SkyReels-V2-DF-1.3B-540P-Diffusers" -vae = AutoModel.from_pretrained(model_id, subfolder="vae", torch_dtype=torch.float32) +vae = AutoModel.from_pretrained(model_id, subfolder="vae", dtype=torch.float32) pipeline = SkyReelsV2DiffusionForcingPipeline.from_pretrained( model_id, vae=vae, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) pipeline.to("cuda") flow_shift = 8.0 # 8.0 for T2V, 5.0 for I2V @@ -219,9 +219,9 @@ from diffusers.utils import export_to_video, load_image model_id = "Skywork/SkyReels-V2-DF-1.3B-720P-Diffusers" -vae = AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", torch_dtype=torch.float32) +vae = AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", dtype=torch.float32) pipeline = SkyReelsV2DiffusionForcingImageToVideoPipeline.from_pretrained( - model_id, vae=vae, torch_dtype=torch.bfloat16 + model_id, vae=vae, dtype=torch.bfloat16 ) pipeline.to("cuda") flow_shift = 5.0 # 8.0 for T2V, 5.0 for I2V @@ -282,9 +282,9 @@ from diffusers.utils import export_to_video, load_video model_id = "Skywork/SkyReels-V2-DF-1.3B-720P-Diffusers" -vae = AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", torch_dtype=torch.float32) +vae = AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", dtype=torch.float32) pipeline = SkyReelsV2DiffusionForcingVideoToVideoPipeline.from_pretrained( - model_id, vae=vae, torch_dtype=torch.bfloat16 + model_id, vae=vae, dtype=torch.bfloat16 ) pipeline.to("cuda") flow_shift = 5.0 # 8.0 for T2V, 5.0 for I2V diff --git a/docs/source/en/api/pipelines/stable_audio.md b/docs/source/en/api/pipelines/stable_audio.md index 82763a52a942..aec3fd6e6707 100644 --- a/docs/source/en/api/pipelines/stable_audio.md +++ b/docs/source/en/api/pipelines/stable_audio.md @@ -52,7 +52,7 @@ text_encoder_8bit = T5EncoderModel.from_pretrained( "stabilityai/stable-audio-open-1.0", subfolder="text_encoder", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) quant_config = DiffusersBitsAndBytesConfig(load_in_8bit=True) @@ -60,14 +60,14 @@ transformer_8bit = StableAudioDiTModel.from_pretrained( "stabilityai/stable-audio-open-1.0", subfolder="transformer", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) pipeline = StableAudioPipeline.from_pretrained( "stabilityai/stable-audio-open-1.0", text_encoder=text_encoder_8bit, transformer=transformer_8bit, - torch_dtype=torch.float16, + dtype=torch.float16, device_map="balanced", ) diff --git a/docs/source/en/api/pipelines/stable_cascade.md b/docs/source/en/api/pipelines/stable_cascade.md index 70de6776e98f..bbe7d452819b 100644 --- a/docs/source/en/api/pipelines/stable_cascade.md +++ b/docs/source/en/api/pipelines/stable_cascade.md @@ -57,8 +57,8 @@ from diffusers import StableCascadeDecoderPipeline, StableCascadePriorPipeline prompt = "an image of a shiba inu, donning a spacesuit and helmet" negative_prompt = "" -prior = StableCascadePriorPipeline.from_pretrained("stabilityai/stable-cascade-prior", variant="bf16", torch_dtype=torch.bfloat16) -decoder = StableCascadeDecoderPipeline.from_pretrained("stabilityai/stable-cascade", variant="bf16", torch_dtype=torch.float16) +prior = StableCascadePriorPipeline.from_pretrained("stabilityai/stable-cascade-prior", variant="bf16", dtype=torch.bfloat16) +decoder = StableCascadeDecoderPipeline.from_pretrained("stabilityai/stable-cascade", variant="bf16", dtype=torch.float16) prior.enable_model_cpu_offload() prior_output = prior( @@ -142,15 +142,15 @@ negative_prompt = "" prior_unet = StableCascadeUNet.from_single_file( "https://huggingface.co/stabilityai/stable-cascade/resolve/main/stage_c_bf16.safetensors", - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) decoder_unet = StableCascadeUNet.from_single_file( "https://huggingface.co/stabilityai/stable-cascade/blob/main/stage_b_bf16.safetensors", - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) -prior = StableCascadePriorPipeline.from_pretrained("stabilityai/stable-cascade-prior", prior=prior_unet, torch_dtype=torch.bfloat16) -decoder = StableCascadeDecoderPipeline.from_pretrained("stabilityai/stable-cascade", decoder=decoder_unet, torch_dtype=torch.bfloat16) +prior = StableCascadePriorPipeline.from_pretrained("stabilityai/stable-cascade-prior", prior=prior_unet, dtype=torch.bfloat16) +decoder = StableCascadeDecoderPipeline.from_pretrained("stabilityai/stable-cascade", decoder=decoder_unet, dtype=torch.bfloat16) prior.enable_model_cpu_offload() prior_output = prior( diff --git a/docs/source/en/api/pipelines/stable_diffusion/sdxl_turbo.md b/docs/source/en/api/pipelines/stable_diffusion/sdxl_turbo.md index fb4f7dbbc18c..c42faf122bc1 100644 --- a/docs/source/en/api/pipelines/stable_diffusion/sdxl_turbo.md +++ b/docs/source/en/api/pipelines/stable_diffusion/sdxl_turbo.md @@ -44,7 +44,7 @@ Model weights may be stored in separate subfolders on the Hub or locally, in whi from diffusers import AutoPipelineForText2Image import torch -pipeline = AutoPipelineForText2Image.from_pretrained("stabilityai/sdxl-turbo", torch_dtype=torch.float16, variant="fp16") +pipeline = AutoPipelineForText2Image.from_pretrained("stabilityai/sdxl-turbo", dtype=torch.float16, variant="fp16") pipeline = pipeline.to("cuda") ``` @@ -56,7 +56,7 @@ import torch pipeline = StableDiffusionXLPipeline.from_single_file( "https://huggingface.co/stabilityai/sdxl-turbo/blob/main/sd_xl_turbo_1.0_fp16.safetensors", - torch_dtype=torch.float16, variant="fp16") + dtype=torch.float16, variant="fp16") pipeline = pipeline.to("cuda") pipeline.scheduler = EulerAncestralDiscreteScheduler.from_config(pipeline.scheduler.config, timestep_spacing="trailing") ``` @@ -72,7 +72,7 @@ Increasing the number of steps to 2, 3 or 4 should improve image quality. from diffusers import AutoPipelineForText2Image import torch -pipeline_text2image = AutoPipelineForText2Image.from_pretrained("stabilityai/sdxl-turbo", torch_dtype=torch.float16, variant="fp16") +pipeline_text2image = AutoPipelineForText2Image.from_pretrained("stabilityai/sdxl-turbo", dtype=torch.float16, variant="fp16") pipeline_text2image = pipeline_text2image.to("cuda") prompt = "A cinematic shot of a baby racoon wearing an intricate italian priest robe." diff --git a/docs/source/en/api/pipelines/stable_diffusion/stable_diffusion_2.md b/docs/source/en/api/pipelines/stable_diffusion/stable_diffusion_2.md index 31164fb9ae8f..590f8ef09275 100644 --- a/docs/source/en/api/pipelines/stable_diffusion/stable_diffusion_2.md +++ b/docs/source/en/api/pipelines/stable_diffusion/stable_diffusion_2.md @@ -45,7 +45,7 @@ from diffusers import DiffusionPipeline, DPMSolverMultistepScheduler import torch repo_id = "stabilityai/stable-diffusion-2-base" -pipe = DiffusionPipeline.from_pretrained(repo_id, torch_dtype=torch.float16, variant="fp16") +pipe = DiffusionPipeline.from_pretrained(repo_id, dtype=torch.float16, variant="fp16") pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config) pipe = pipe.to("cuda") @@ -69,7 +69,7 @@ init_image = load_image(img_url).resize((512, 512)) mask_image = load_image(mask_url).resize((512, 512)) repo_id = "stabilityai/stable-diffusion-2-inpainting" -pipe = DiffusionPipeline.from_pretrained(repo_id, torch_dtype=torch.float16, variant="fp16") +pipe = DiffusionPipeline.from_pretrained(repo_id, dtype=torch.float16, variant="fp16") pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config) pipe = pipe.to("cuda") @@ -88,7 +88,7 @@ import torch # load model and scheduler model_id = "stabilityai/stable-diffusion-x4-upscaler" -pipeline = StableDiffusionUpscalePipeline.from_pretrained(model_id, torch_dtype=torch.float16) +pipeline = StableDiffusionUpscalePipeline.from_pretrained(model_id, dtype=torch.float16) pipeline = pipeline.to("cuda") # let's download an image @@ -109,7 +109,7 @@ from diffusers.utils import load_image, make_image_grid pipe = StableDiffusionDepth2ImgPipeline.from_pretrained( "stabilityai/stable-diffusion-2-depth", - torch_dtype=torch.float16, + dtype=torch.float16, ).to("cuda") diff --git a/docs/source/en/api/pipelines/stable_diffusion/stable_diffusion_3.md b/docs/source/en/api/pipelines/stable_diffusion/stable_diffusion_3.md index 4c7e5b107316..c2c62716b182 100644 --- a/docs/source/en/api/pipelines/stable_diffusion/stable_diffusion_3.md +++ b/docs/source/en/api/pipelines/stable_diffusion/stable_diffusion_3.md @@ -41,7 +41,7 @@ hf auth login import torch from diffusers import StableDiffusion3Pipeline -pipe = StableDiffusion3Pipeline.from_pretrained("stabilityai/stable-diffusion-3-medium-diffusers", torch_dtype=torch.float16) +pipe = StableDiffusion3Pipeline.from_pretrained("stabilityai/stable-diffusion-3-medium-diffusers", dtype=torch.float16) pipe.to("cuda") image = pipe( @@ -83,16 +83,16 @@ ip_adapter_id = "InstantX/SD3.5-Large-IP-Adapter" feature_extractor = SiglipImageProcessor.from_pretrained( image_encoder_id, - torch_dtype=torch.float16 + dtype=torch.float16 ) image_encoder = SiglipVisionModel.from_pretrained( image_encoder_id, - torch_dtype=torch.float16 + dtype=torch.float16 ).to( "cuda") pipe = StableDiffusion3Pipeline.from_pretrained( "stabilityai/stable-diffusion-3.5-large", - torch_dtype=torch.float16, + dtype=torch.float16, feature_extractor=feature_extractor, image_encoder=image_encoder, ).to("cuda") @@ -137,7 +137,7 @@ The most basic memory optimization available in Diffusers allows you to offload import torch from diffusers import StableDiffusion3Pipeline -pipe = StableDiffusion3Pipeline.from_pretrained("stabilityai/stable-diffusion-3-medium-diffusers", torch_dtype=torch.float16) +pipe = StableDiffusion3Pipeline.from_pretrained("stabilityai/stable-diffusion-3-medium-diffusers", dtype=torch.float16) pipe.enable_model_cpu_offload() image = pipe( @@ -164,7 +164,7 @@ pipe = StableDiffusion3Pipeline.from_pretrained( "stabilityai/stable-diffusion-3-medium-diffusers", text_encoder_3=None, tokenizer_3=None, - torch_dtype=torch.float16 + dtype=torch.float16 ) pipe.to("cuda") @@ -209,7 +209,7 @@ pipe = StableDiffusion3Pipeline.from_pretrained( model_id, text_encoder_3=text_encoder, device_map="balanced", - torch_dtype=torch.float16 + dtype=torch.float16 ) image = pipe( @@ -245,7 +245,7 @@ torch._inductor.config.coordinate_descent_check_all_directions = True pipe = StableDiffusion3Pipeline.from_pretrained( "stabilityai/stable-diffusion-3-medium-diffusers", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipe.set_progress_bar_config(disable=True) @@ -283,7 +283,7 @@ text_encoder_8bit = T5EncoderModel.from_pretrained( "stabilityai/stable-diffusion-3.5-large", subfolder="text_encoder_3", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) quant_config = DiffusersBitsAndBytesConfig(load_in_8bit=True) @@ -291,14 +291,14 @@ transformer_8bit = SD3Transformer2DModel.from_pretrained( "stabilityai/stable-diffusion-3.5-large", subfolder="transformer", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) pipeline = StableDiffusion3Pipeline.from_pretrained( "stabilityai/stable-diffusion-3.5-large", text_encoder=text_encoder_8bit, transformer=transformer_8bit, - torch_dtype=torch.float16, + dtype=torch.float16, device_map="balanced", ) @@ -356,9 +356,9 @@ import torch from diffusers import StableDiffusion3Pipeline, AutoencoderTiny pipe = StableDiffusion3Pipeline.from_pretrained( - "stabilityai/stable-diffusion-3-medium-diffusers", torch_dtype=torch.float16 + "stabilityai/stable-diffusion-3-medium-diffusers", dtype=torch.float16 ) -pipe.vae = AutoencoderTiny.from_pretrained("madebyollin/taesd3", torch_dtype=torch.float16) +pipe.vae = AutoencoderTiny.from_pretrained("madebyollin/taesd3", dtype=torch.float16) pipe = pipe.to("cuda") prompt = "slice of delicious New York-style berry cheesecake" @@ -388,7 +388,7 @@ from diffusers import StableDiffusion3Pipeline pipe = StableDiffusion3Pipeline.from_single_file( "https://huggingface.co/stabilityai/stable-diffusion-3-medium/blob/main/sd3_medium_incl_clips.safetensors", - torch_dtype=torch.float16, + dtype=torch.float16, text_encoder_3=None ) pipe.enable_model_cpu_offload() @@ -408,7 +408,7 @@ from diffusers import StableDiffusion3Pipeline pipe = StableDiffusion3Pipeline.from_single_file( "https://huggingface.co/stabilityai/stable-diffusion-3-medium/blob/main/sd3_medium_incl_clips_t5xxlfp8.safetensors", - torch_dtype=torch.float16, + dtype=torch.float16, ) pipe.enable_model_cpu_offload() @@ -424,12 +424,12 @@ from diffusers import SD3Transformer2DModel, StableDiffusion3Pipeline transformer = SD3Transformer2DModel.from_single_file( "https://huggingface.co/stabilityai/stable-diffusion-3.5-large-turbo/blob/main/sd3.5_large.safetensors", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) pipe = StableDiffusion3Pipeline.from_pretrained( "stabilityai/stable-diffusion-3.5-large", transformer=transformer, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) pipe.enable_model_cpu_offload() image = pipe("a cat holding a sign that says hello world").images[0] diff --git a/docs/source/en/api/pipelines/stable_diffusion/stable_diffusion_xl.md b/docs/source/en/api/pipelines/stable_diffusion/stable_diffusion_xl.md index d65f78f799e5..e37dfbe666ba 100644 --- a/docs/source/en/api/pipelines/stable_diffusion/stable_diffusion_xl.md +++ b/docs/source/en/api/pipelines/stable_diffusion/stable_diffusion_xl.md @@ -59,11 +59,11 @@ from diffusers import StableDiffusionXLPipeline, StableDiffusionXLImg2ImgPipelin import torch pipeline = StableDiffusionXLPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16, variant="fp16", use_safetensors=True + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16, variant="fp16", use_safetensors=True ).to("cuda") refiner = StableDiffusionXLImg2ImgPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-refiner-1.0", torch_dtype=torch.float16, use_safetensors=True, variant="fp16" + "stabilityai/stable-diffusion-xl-refiner-1.0", dtype=torch.float16, use_safetensors=True, variant="fp16" ).to("cuda") ``` @@ -75,11 +75,11 @@ import torch pipeline = StableDiffusionXLPipeline.from_single_file( "https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/blob/main/sd_xl_base_1.0.safetensors", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") refiner = StableDiffusionXLImg2ImgPipeline.from_single_file( - "https://huggingface.co/stabilityai/stable-diffusion-xl-refiner-1.0/blob/main/sd_xl_refiner_1.0.safetensors", torch_dtype=torch.float16 + "https://huggingface.co/stabilityai/stable-diffusion-xl-refiner-1.0/blob/main/sd_xl_refiner_1.0.safetensors", dtype=torch.float16 ).to("cuda") ``` @@ -92,7 +92,7 @@ from diffusers import AutoPipelineForText2Image import torch pipeline_text2image = AutoPipelineForText2Image.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16, variant="fp16", use_safetensors=True + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16, variant="fp16", use_safetensors=True ).to("cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" @@ -170,14 +170,14 @@ from diffusers import DiffusionPipeline import torch base = DiffusionPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16, variant="fp16", use_safetensors=True + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16, variant="fp16", use_safetensors=True ).to("cuda") refiner = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-refiner-1.0", text_encoder_2=base.text_encoder_2, vae=base.vae, - torch_dtype=torch.float16, + dtype=torch.float16, use_safetensors=True, variant="fp16", ).to("cuda") @@ -227,14 +227,14 @@ from diffusers.utils import load_image, make_image_grid import torch base = StableDiffusionXLInpaintPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16, variant="fp16", use_safetensors=True + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16, variant="fp16", use_safetensors=True ).to("cuda") refiner = StableDiffusionXLInpaintPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-refiner-1.0", text_encoder_2=base.text_encoder_2, vae=base.vae, - torch_dtype=torch.float16, + dtype=torch.float16, use_safetensors=True, variant="fp16", ).to("cuda") @@ -280,14 +280,14 @@ from diffusers import DiffusionPipeline import torch base = DiffusionPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16, variant="fp16", use_safetensors=True + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16, variant="fp16", use_safetensors=True ).to("cuda") refiner = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-refiner-1.0", text_encoder_2=base.text_encoder_2, vae=base.vae, - torch_dtype=torch.float16, + dtype=torch.float16, use_safetensors=True, variant="fp16", ).to("cuda") @@ -345,7 +345,7 @@ from diffusers import StableDiffusionXLPipeline import torch pipe = StableDiffusionXLPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16, variant="fp16", use_safetensors=True + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16, variant="fp16", use_safetensors=True ).to("cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" @@ -370,7 +370,7 @@ from diffusers import StableDiffusionXLPipeline import torch pipeline = StableDiffusionXLPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16, variant="fp16", use_safetensors=True + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16, variant="fp16", use_safetensors=True ).to("cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" @@ -389,7 +389,7 @@ from diffusers import StableDiffusionXLPipeline import torch pipe = StableDiffusionXLPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16, variant="fp16", use_safetensors=True + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16, variant="fp16", use_safetensors=True ).to("cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" @@ -411,7 +411,7 @@ from diffusers import StableDiffusionXLPipeline import torch pipeline = StableDiffusionXLPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16, variant="fp16", use_safetensors=True + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16, variant="fp16", use_safetensors=True ).to("cuda") # prompt is passed to OAI CLIP-ViT/L-14 diff --git a/docs/source/en/api/pipelines/stable_diffusion/svd.md b/docs/source/en/api/pipelines/stable_diffusion/svd.md index 086ef96d690d..97392adc3bad 100644 --- a/docs/source/en/api/pipelines/stable_diffusion/svd.md +++ b/docs/source/en/api/pipelines/stable_diffusion/svd.md @@ -39,7 +39,7 @@ from diffusers import StableVideoDiffusionPipeline from diffusers.utils import load_image, export_to_video pipe = StableVideoDiffusionPipeline.from_pretrained( - "stabilityai/stable-video-diffusion-img2vid-xt", torch_dtype=torch.float16, variant="fp16" + "stabilityai/stable-video-diffusion-img2vid-xt", dtype=torch.float16, variant="fp16" ) pipe.enable_model_cpu_offload() @@ -109,7 +109,7 @@ from diffusers import StableVideoDiffusionPipeline from diffusers.utils import load_image, export_to_video pipe = StableVideoDiffusionPipeline.from_pretrained( - "stabilityai/stable-video-diffusion-img2vid-xt", torch_dtype=torch.float16, variant="fp16" + "stabilityai/stable-video-diffusion-img2vid-xt", dtype=torch.float16, variant="fp16" ) pipe.enable_model_cpu_offload() diff --git a/docs/source/en/api/pipelines/stable_unclip.md b/docs/source/en/api/pipelines/stable_unclip.md index 2284ee144d58..a843afb16b59 100644 --- a/docs/source/en/api/pipelines/stable_unclip.md +++ b/docs/source/en/api/pipelines/stable_unclip.md @@ -39,11 +39,11 @@ from transformers import CLIPTokenizer, CLIPTextModelWithProjection prior_model_id = "kakaobrain/karlo-v1-alpha" data_type = torch.float16 -prior = PriorTransformer.from_pretrained(prior_model_id, subfolder="prior", torch_dtype=data_type) +prior = PriorTransformer.from_pretrained(prior_model_id, subfolder="prior", dtype=data_type) prior_text_model_id = "openai/clip-vit-large-patch14" prior_tokenizer = CLIPTokenizer.from_pretrained(prior_text_model_id) -prior_text_model = CLIPTextModelWithProjection.from_pretrained(prior_text_model_id, torch_dtype=data_type) +prior_text_model = CLIPTextModelWithProjection.from_pretrained(prior_text_model_id, dtype=data_type) prior_scheduler = UnCLIPScheduler.from_pretrained(prior_model_id, subfolder="prior_scheduler") prior_scheduler = DDPMScheduler.from_config(prior_scheduler.config) @@ -51,7 +51,7 @@ stable_unclip_model_id = "stabilityai/stable-diffusion-2-1-unclip-small" pipe = StableUnCLIPPipeline.from_pretrained( stable_unclip_model_id, - torch_dtype=data_type, + dtype=data_type, variant="fp16", prior_tokenizer=prior_tokenizer, prior_text_encoder=prior_text_model, @@ -76,7 +76,7 @@ from diffusers.utils import load_image import torch pipe = StableUnCLIPImg2ImgPipeline.from_pretrained( - "stabilityai/stable-diffusion-2-1-unclip", torch_dtype=torch.float16, variation="fp16" + "stabilityai/stable-diffusion-2-1-unclip", dtype=torch.float16, variation="fp16" ) pipe = pipe.to("cuda") diff --git a/docs/source/en/api/pipelines/visualcloze.md b/docs/source/en/api/pipelines/visualcloze.md index 1a4f96a50d63..83b6930094cb 100644 --- a/docs/source/en/api/pipelines/visualcloze.md +++ b/docs/source/en/api/pipelines/visualcloze.md @@ -69,7 +69,7 @@ import torch from diffusers import VisualClozePipeline from diffusers.utils import load_image -pipe = VisualClozePipeline.from_pretrained("VisualCloze/VisualClozePipeline-384", resolution=384, torch_dtype=torch.bfloat16) +pipe = VisualClozePipeline.from_pretrained("VisualCloze/VisualClozePipeline-384", resolution=384, dtype=torch.bfloat16) pipe.to("cuda") # Load in-context images (make sure the paths are correct and accessible) @@ -121,7 +121,7 @@ import torch from diffusers import VisualClozePipeline from diffusers.utils import load_image -pipe = VisualClozePipeline.from_pretrained("VisualCloze/VisualClozePipeline-384", resolution=384, torch_dtype=torch.bfloat16) +pipe = VisualClozePipeline.from_pretrained("VisualCloze/VisualClozePipeline-384", resolution=384, dtype=torch.bfloat16) pipe.to("cuda") # Load in-context images (make sure the paths are correct and accessible) @@ -171,7 +171,7 @@ import torch from diffusers import VisualClozePipeline from diffusers.utils import load_image -pipe = VisualClozePipeline.from_pretrained("VisualCloze/VisualClozePipeline-384", resolution=384, torch_dtype=torch.bfloat16) +pipe = VisualClozePipeline.from_pretrained("VisualCloze/VisualClozePipeline-384", resolution=384, dtype=torch.bfloat16) pipe.to("cuda") # Load in-context images (make sure the paths are correct and accessible) @@ -229,7 +229,7 @@ from diffusers.utils import load_image from PIL import Image pipe = VisualClozeGenerationPipeline.from_pretrained( - "VisualCloze/VisualClozePipeline-384", resolution=384, torch_dtype=torch.bfloat16 + "VisualCloze/VisualClozePipeline-384", resolution=384, dtype=torch.bfloat16 ) pipe.to("cuda") diff --git a/docs/source/en/api/pipelines/wan.md b/docs/source/en/api/pipelines/wan.md index d5fdbbfe0f95..4da629777606 100644 --- a/docs/source/en/api/pipelines/wan.md +++ b/docs/source/en/api/pipelines/wan.md @@ -66,9 +66,9 @@ from diffusers.hooks.group_offloading import apply_group_offloading from diffusers.utils import export_to_video, load_image from transformers import UMT5EncoderModel -text_encoder = UMT5EncoderModel.from_pretrained("Wan-AI/Wan2.1-T2V-14B-Diffusers", subfolder="text_encoder", torch_dtype=torch.bfloat16) -vae = AutoModel.from_pretrained("Wan-AI/Wan2.1-T2V-14B-Diffusers", subfolder="vae", torch_dtype=torch.float32) -transformer = AutoModel.from_pretrained("Wan-AI/Wan2.1-T2V-14B-Diffusers", subfolder="transformer", torch_dtype=torch.bfloat16) +text_encoder = UMT5EncoderModel.from_pretrained("Wan-AI/Wan2.1-T2V-14B-Diffusers", subfolder="text_encoder", dtype=torch.bfloat16) +vae = AutoModel.from_pretrained("Wan-AI/Wan2.1-T2V-14B-Diffusers", subfolder="vae", dtype=torch.float32) +transformer = AutoModel.from_pretrained("Wan-AI/Wan2.1-T2V-14B-Diffusers", subfolder="transformer", dtype=torch.bfloat16) # group-offloading onload_device = torch.device("cuda") @@ -91,7 +91,7 @@ pipeline = WanPipeline.from_pretrained( vae=vae, transformer=transformer, text_encoder=text_encoder, - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) pipeline.to("cuda") @@ -131,16 +131,16 @@ from diffusers.hooks.group_offloading import apply_group_offloading from diffusers.utils import export_to_video, load_image from transformers import UMT5EncoderModel -text_encoder = UMT5EncoderModel.from_pretrained("Wan-AI/Wan2.1-T2V-14B-Diffusers", subfolder="text_encoder", torch_dtype=torch.bfloat16) -vae = AutoModel.from_pretrained("Wan-AI/Wan2.1-T2V-14B-Diffusers", subfolder="vae", torch_dtype=torch.float32) -transformer = AutoModel.from_pretrained("Wan-AI/Wan2.1-T2V-14B-Diffusers", subfolder="transformer", torch_dtype=torch.bfloat16) +text_encoder = UMT5EncoderModel.from_pretrained("Wan-AI/Wan2.1-T2V-14B-Diffusers", subfolder="text_encoder", dtype=torch.bfloat16) +vae = AutoModel.from_pretrained("Wan-AI/Wan2.1-T2V-14B-Diffusers", subfolder="vae", dtype=torch.float32) +transformer = AutoModel.from_pretrained("Wan-AI/Wan2.1-T2V-14B-Diffusers", subfolder="transformer", dtype=torch.bfloat16) pipeline = WanPipeline.from_pretrained( "Wan-AI/Wan2.1-T2V-14B-Diffusers", vae=vae, transformer=transformer, text_encoder=text_encoder, - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) pipeline.to("cuda") @@ -192,10 +192,10 @@ from transformers import CLIPVisionModel model_id = "Wan-AI/Wan2.1-FLF2V-14B-720P-diffusers" -image_encoder = CLIPVisionModel.from_pretrained(model_id, subfolder="image_encoder", torch_dtype=torch.float32) -vae = AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", torch_dtype=torch.float32) +image_encoder = CLIPVisionModel.from_pretrained(model_id, subfolder="image_encoder", dtype=torch.float32) +vae = AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", dtype=torch.float32) pipe = WanImageToVideoPipeline.from_pretrained( - model_id, vae=vae, image_encoder=image_encoder, torch_dtype=torch.bfloat16 + model_id, vae=vae, image_encoder=image_encoder, dtype=torch.bfloat16 ) pipe.to("cuda") @@ -292,8 +292,8 @@ from diffusers import AutoencoderKLWan, WanAnimatePipeline from diffusers.utils import export_to_video, load_image, load_video model_id = "Wan-AI/Wan2.2-Animate-14B-Diffusers" -vae = AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", torch_dtype=torch.float32) -pipe = WanAnimatePipeline.from_pretrained(model_id, vae=vae, torch_dtype=torch.bfloat16) +vae = AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", dtype=torch.float32) +pipe = WanAnimatePipeline.from_pretrained(model_id, vae=vae, dtype=torch.bfloat16) pipe.to("cuda") # Load character image and preprocessed videos @@ -341,8 +341,8 @@ from diffusers import AutoencoderKLWan, WanAnimatePipeline from diffusers.utils import export_to_video, load_image, load_video model_id = "Wan-AI/Wan2.2-Animate-14B-Diffusers" -vae = AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", torch_dtype=torch.float32) -pipe = WanAnimatePipeline.from_pretrained(model_id, vae=vae, torch_dtype=torch.bfloat16) +vae = AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", dtype=torch.float32) +pipe = WanAnimatePipeline.from_pretrained(model_id, vae=vae, dtype=torch.bfloat16) pipe.to("cuda") # Load all required inputs for replacement mode @@ -394,8 +394,8 @@ from diffusers import AutoencoderKLWan, WanAnimatePipeline from diffusers.utils import export_to_video, load_image, load_video model_id = "Wan-AI/Wan2.2-Animate-14B-Diffusers" -vae = AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", torch_dtype=torch.float32) -pipe = WanAnimatePipeline.from_pretrained(model_id, vae=vae, torch_dtype=torch.bfloat16) +vae = AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", dtype=torch.float32) +pipe = WanAnimatePipeline.from_pretrained(model_id, vae=vae, dtype=torch.bfloat16) pipe.to("cuda") image = load_image("path/to/character.jpg") @@ -464,10 +464,10 @@ export_to_video(output, "animated_advanced.mp4", fps=30) from diffusers.utils import export_to_video vae = AutoModel.from_pretrained( - "Wan-AI/Wan2.1-T2V-1.3B-Diffusers", subfolder="vae", torch_dtype=torch.float32 + "Wan-AI/Wan2.1-T2V-1.3B-Diffusers", subfolder="vae", dtype=torch.float32 ) pipeline = WanPipeline.from_pretrained( - "Wan-AI/Wan2.1-T2V-1.3B-Diffusers", vae=vae, torch_dtype=torch.bfloat16 + "Wan-AI/Wan2.1-T2V-1.3B-Diffusers", vae=vae, dtype=torch.bfloat16 ) pipeline.scheduler = UniPCMultistepScheduler.from_config( pipeline.scheduler.config, flow_shift=5.0 @@ -513,13 +513,13 @@ export_to_video(output, "animated_advanced.mp4", fps=30) ) transformer = WanTransformer3DModel.from_single_file( "https://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/blob/main/split_files/diffusion_models/wan2.1_t2v_1.3B_bf16.safetensors", - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) pipeline = WanPipeline.from_pretrained( "Wan-AI/Wan2.1-T2V-1.3B-Diffusers", vae=vae, transformer=transformer, - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) ``` diff --git a/docs/source/en/api/pipelines/z_image.md b/docs/source/en/api/pipelines/z_image.md index 85d926406368..ea7f010b51d6 100644 --- a/docs/source/en/api/pipelines/z_image.md +++ b/docs/source/en/api/pipelines/z_image.md @@ -35,7 +35,7 @@ import torch from diffusers import ZImageImg2ImgPipeline from diffusers.utils import load_image -pipe = ZImageImg2ImgPipeline.from_pretrained("Tongyi-MAI/Z-Image-Turbo", torch_dtype=torch.bfloat16) +pipe = ZImageImg2ImgPipeline.from_pretrained("Tongyi-MAI/Z-Image-Turbo", dtype=torch.bfloat16) pipe.to("cuda") url = "https://raw.githubusercontent.com/CompVis/stable-diffusion/main/assets/stable-samples/img2img/sketch-mountains-input.jpg" @@ -64,7 +64,7 @@ from PIL import Image from diffusers import ZImageInpaintPipeline from diffusers.utils import load_image -pipe = ZImageInpaintPipeline.from_pretrained("Tongyi-MAI/Z-Image-Turbo", torch_dtype=torch.bfloat16) +pipe = ZImageInpaintPipeline.from_pretrained("Tongyi-MAI/Z-Image-Turbo", dtype=torch.bfloat16) pipe.to("cuda") url = "https://raw.githubusercontent.com/CompVis/stable-diffusion/main/assets/stable-samples/img2img/sketch-mountains-input.jpg" diff --git a/docs/source/en/api/schedulers/ddim.md b/docs/source/en/api/schedulers/ddim.md index 61ef30c786f9..99ab1a9ac02b 100644 --- a/docs/source/en/api/schedulers/ddim.md +++ b/docs/source/en/api/schedulers/ddim.md @@ -61,7 +61,7 @@ For example: from diffusers import DiffusionPipeline, DDIMScheduler import torch -pipe = DiffusionPipeline.from_pretrained("ptx0/pseudo-journey-v2", torch_dtype=torch.float16) +pipe = DiffusionPipeline.from_pretrained("ptx0/pseudo-journey-v2", dtype=torch.float16) pipe.scheduler = DDIMScheduler.from_config( pipe.scheduler.config, rescale_betas_zero_snr=True, timestep_spacing="trailing" ) diff --git a/docs/source/en/conceptual/evaluation.md b/docs/source/en/conceptual/evaluation.md index 4af38254bea6..168aaf29889c 100644 --- a/docs/source/en/conceptual/evaluation.md +++ b/docs/source/en/conceptual/evaluation.md @@ -128,7 +128,7 @@ from diffusers import StableDiffusionPipeline import torch model_ckpt = "CompVis/stable-diffusion-v1-4" -sd_pipeline = StableDiffusionPipeline.from_pretrained(model_ckpt, torch_dtype=torch.float16).to("cuda") +sd_pipeline = StableDiffusionPipeline.from_pretrained(model_ckpt, dtype=torch.float16).to("cuda") ``` Generate some images with multiple prompts: @@ -183,7 +183,7 @@ Then we load the [v1-5 checkpoint](https://huggingface.co/stable-diffusion-v1-5/ ```python model_ckpt_1_5 = "stable-diffusion-v1-5/stable-diffusion-v1-5" -sd_pipeline_1_5 = StableDiffusionPipeline.from_pretrained(model_ckpt_1_5, torch_dtype=torch.float16).to("cuda") +sd_pipeline_1_5 = StableDiffusionPipeline.from_pretrained(model_ckpt_1_5, dtype=torch.float16).to("cuda") images_1_5 = sd_pipeline_1_5(prompts, num_images_per_prompt=1, generator=generator, output_type="np").images ``` @@ -278,7 +278,7 @@ Let's first load the [`StableDiffusionInstructPix2PixPipeline`]: from diffusers import StableDiffusionInstructPix2PixPipeline instruct_pix2pix_pipeline = StableDiffusionInstructPix2PixPipeline.from_pretrained( - "timbrooks/instruct-pix2pix", torch_dtype=torch.float16 + "timbrooks/instruct-pix2pix", dtype=torch.float16 ).to("cuda") ``` @@ -492,7 +492,7 @@ We now load the [`DiTPipeline`](https://huggingface.co/docs/diffusers/api/pipel ```python from diffusers import DiTPipeline, DPMSolverMultistepScheduler -dit_pipeline = DiTPipeline.from_pretrained("facebook/DiT-XL-2-256", torch_dtype=torch.float16) +dit_pipeline = DiTPipeline.from_pretrained("facebook/DiT-XL-2-256", dtype=torch.float16) dit_pipeline.scheduler = DPMSolverMultistepScheduler.from_config(dit_pipeline.scheduler.config) dit_pipeline = dit_pipeline.to("cuda") diff --git a/docs/source/en/hybrid_inference/overview.md b/docs/source/en/hybrid_inference/overview.md index 1384be9b7348..b7ebcb206850 100644 --- a/docs/source/en/hybrid_inference/overview.md +++ b/docs/source/en/hybrid_inference/overview.md @@ -40,7 +40,7 @@ from diffusers.utils.remote_utils import remote_encode pipeline = FluxPipeline.from_pretrained( "black-forest-labs/FLUX.1-schnell", - torch_dtype=torch.float16, + dtype=torch.float16, vae=None, device_map="cuda" ) @@ -72,7 +72,7 @@ from diffusers import FluxPipeline pipeline = FluxPipeline.from_pretrained( "black-forest-labs/FLUX.1-schnell", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, vae=None, device_map="cuda" ) @@ -106,10 +106,10 @@ import torch from diffusers import HunyuanVideoPipeline, HunyuanVideoTransformer3DModel transformer = HunyuanVideoTransformer3DModel.from_pretrained( - "hunyuanvideo-community/HunyuanVideo", subfolder="transformer", torch_dtype=torch.bfloat16 + "hunyuanvideo-community/HunyuanVideo", subfolder="transformer", dtype=torch.bfloat16 ) pipeline = HunyuanVideoPipeline.from_pretrained( - model_id, transformer=transformer, vae=None, torch_dtype=torch.float16, device_map="cuda" + model_id, transformer=transformer, vae=None, dtype=torch.float16, device_map="cuda" ) latent = pipeline( @@ -176,7 +176,7 @@ prompts = [ pipeline = StableDiffusionXLPipeline.from_pretrained( "https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16, + dtype=torch.float16, vae=None, device_map="cuda" ) diff --git a/docs/source/en/modular_diffusers/components_manager.md b/docs/source/en/modular_diffusers/components_manager.md index 426739347f27..98c8e4d7e7f3 100644 --- a/docs/source/en/modular_diffusers/components_manager.md +++ b/docs/source/en/modular_diffusers/components_manager.md @@ -30,7 +30,7 @@ import torch manager = ComponentsManager() pipe = ModularPipeline.from_pretrained("Tongyi-MAI/Z-Image-Turbo", components_manager=manager) -pipe.load_components(torch_dtype=torch.bfloat16) +pipe.load_components(dtype=torch.bfloat16) ``` @@ -42,7 +42,7 @@ import torch manager = ComponentsManager() blocks = ModularPipelineBlocks.from_pretrained("diffusers/Florence2-image-Annotator", trust_remote_code=True) pipe= blocks.init_pipeline(components_manager=manager) -pipe.load_components(torch_dtype=torch.bfloat16) +pipe.load_components(dtype=torch.bfloat16) ``` diff --git a/docs/source/en/modular_diffusers/custom_blocks.md b/docs/source/en/modular_diffusers/custom_blocks.md index 66e1de172b34..f5e957a08530 100644 --- a/docs/source/en/modular_diffusers/custom_blocks.md +++ b/docs/source/en/modular_diffusers/custom_blocks.md @@ -238,7 +238,7 @@ print(image_annotator.blocks.doc) Use the block to generate a mask: ```python -image_annotator.load_components(torch_dtype=torch.bfloat16) +image_annotator.load_components(dtype=torch.bfloat16) image_annotator.to("cuda") image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/car.jpg") @@ -269,7 +269,7 @@ inpaint_blocks.sub_blocks.insert("image_annotator", annotator_block, 0) # Initialize the combined pipeline pipe = inpaint_blocks.init_pipeline() -pipe.load_components(torch_dtype=torch.float16, device="cuda") +pipe.load_components(dtype=torch.float16, device="cuda") # Now the pipeline automatically generates masks from prompts output = pipe( diff --git a/docs/source/en/modular_diffusers/modular_pipeline.md b/docs/source/en/modular_diffusers/modular_pipeline.md index 27bc61634805..caa3b2e5a361 100644 --- a/docs/source/en/modular_diffusers/modular_pipeline.md +++ b/docs/source/en/modular_diffusers/modular_pipeline.md @@ -14,7 +14,7 @@ specific language governing permissions and limitations under the License. [`ModularPipeline`] converts [`~modular_pipelines.ModularPipelineBlocks`] into an executable pipeline that loads models and performs the computation steps defined in the blocks. It is the main interface for running a pipeline and the API is very similar to [`DiffusionPipeline`] but with a few key differences. -- **Loading is lazy.** With [`DiffusionPipeline`], [`~DiffusionPipeline.from_pretrained`] creates the pipeline and loads all models at the same time. With [`ModularPipeline`], creating and loading are two separate steps: [`~ModularPipeline.from_pretrained`] reads the configuration and knows where to load each component from, but doesn't actually load the model weights. You load the models later with [`~ModularPipeline.load_components`], which is where you pass loading arguments like `torch_dtype` and `quantization_config`. +- **Loading is lazy.** With [`DiffusionPipeline`], [`~DiffusionPipeline.from_pretrained`] creates the pipeline and loads all models at the same time. With [`ModularPipeline`], creating and loading are two separate steps: [`~ModularPipeline.from_pretrained`] reads the configuration and knows where to load each component from, but doesn't actually load the model weights. You load the models later with [`~ModularPipeline.load_components`], which is where you pass loading arguments like `dtype` and `quantization_config`. - **Two ways to create a pipeline.** You can use [`~ModularPipeline.from_pretrained`] with an existing diffusers model repository — it automatically maps to the default pipeline blocks and then converts to a [`ModularPipeline`] with no extra setup. You can check the [modular_pipelines_directory](https://github.com/huggingface/diffusers/tree/main/src/diffusers/modular_pipelines) to see which models are currently supported. You can also assemble your own pipeline from [`ModularPipelineBlocks`] and convert it with the [`~ModularPipelineBlocks.init_pipeline`] method (see [Creating a pipeline](#creating-a-pipeline) for more details). @@ -30,7 +30,7 @@ import torch from diffusers import ModularPipeline pipeline = ModularPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0") -pipeline.load_components(torch_dtype=torch.float16) +pipeline.load_components(dtype=torch.float16) pipeline.to("cuda") image = pipeline(prompt="Astronaut in a jungle, cold color palette, muted colors, detailed, 8k").images[0] @@ -46,7 +46,7 @@ from diffusers import ModularPipeline from diffusers.utils import load_image pipeline = ModularPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0") -pipeline.load_components(torch_dtype=torch.float16) +pipeline.load_components(dtype=torch.float16) pipeline.to("cuda") url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/sdxl-text2img.png" @@ -65,7 +65,7 @@ from diffusers import ModularPipeline from diffusers.utils import load_image pipeline = ModularPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0") -pipeline.load_components(torch_dtype=torch.float16) +pipeline.load_components(dtype=torch.float16) pipeline.to("cuda") img_url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/sdxl-text2img.png" @@ -243,13 +243,13 @@ This will load all the components that have a valid loading spec. ```py import torch -pipeline.load_components(torch_dtype=torch.float16) +pipeline.load_components(dtype=torch.float16) ``` You can also load specific components by name. The example below only loads the `text_encoder`. ```py -pipeline.load_components(names=["text_encoder"], torch_dtype=torch.float16) +pipeline.load_components(names=["text_encoder"], dtype=torch.float16) ``` After loading, printing the pipeline shows which components are loaded — the first two fields change from `null` to the component's library and class. @@ -274,14 +274,14 @@ pipeline ] ``` -Loading keyword arguments like `torch_dtype`, `variant`, `revision`, and `quantization_config` are passed through to `from_pretrained()` for each component. You can pass a single value to apply to all components, or a dict to set per-component values. +Loading keyword arguments like `dtype`, `variant`, `revision`, and `quantization_config` are passed through to `from_pretrained()` for each component. You can pass a single value to apply to all components, or a dict to set per-component values. ```py # apply bfloat16 to all components -pipeline.load_components(torch_dtype=torch.bfloat16) +pipeline.load_components(dtype=torch.bfloat16) # different dtypes per component -pipeline.load_components(torch_dtype={"transformer": torch.bfloat16, "default": torch.float32}) +pipeline.load_components(dtype={"transformer": torch.bfloat16, "default": torch.float32}) ``` [`~ModularPipeline.load_components`] only loads components that haven't been loaded yet and have a valid loading spec. This means if you've already set a component on the pipeline, calling [`~ModularPipeline.load_components`] again won't reload it. @@ -298,7 +298,7 @@ You can pass a model object loaded with `AutoModel.from_pretrained()`. Models lo from diffusers import AutoModel unet = AutoModel.from_pretrained( - "RunDiffusion/Juggernaut-XL-v9", subfolder="unet", variant="fp16", torch_dtype=torch.float16 + "RunDiffusion/Juggernaut-XL-v9", subfolder="unet", variant="fp16", dtype=torch.float16 ) pipeline.update_components(unet=unet) ``` @@ -314,7 +314,7 @@ unet_spec = pipeline.get_component_spec("unet") unet_spec.pretrained_model_name_or_path = "RunDiffusion/Juggernaut-XL-v9" # load and update -unet = unet_spec.load(torch_dtype=torch.float16) +unet = unet_spec.load(dtype=torch.float16) pipeline.update_components(unet=unet) ``` @@ -365,11 +365,11 @@ text_encoder_pipeline = text_block.init_pipeline(repo_id, components_manager=man pipeline = blocks.init_pipeline(repo_id, components_manager=manager) # encode text -text_encoder_pipeline.load_components(torch_dtype=dtype) +text_encoder_pipeline.load_components(dtype=dtype) text_embeddings = text_encoder_pipeline(prompt="a cat").get_by_kwargs("denoiser_input_fields") # denoise and decode -pipeline.load_components(torch_dtype=dtype) +pipeline.load_components(dtype=dtype) output = pipeline( **text_embeddings, num_inference_steps=4, diff --git a/docs/source/en/modular_diffusers/quickstart.md b/docs/source/en/modular_diffusers/quickstart.md index 884495e107b4..2961d6c6ecbc 100644 --- a/docs/source/en/modular_diffusers/quickstart.md +++ b/docs/source/en/modular_diffusers/quickstart.md @@ -28,7 +28,7 @@ manager = ComponentsManager() manager.enable_auto_cpu_offload(device="cuda:0") pipe = ModularPipeline.from_pretrained("Qwen/Qwen-Image", components_manager=manager) -pipe.load_components(torch_dtype=torch.bfloat16) +pipe.load_components(dtype=torch.bfloat16) image = pipe( prompt="cat wizard with red hat, gandalf, lord of the rings, detailed, fantasy, cute, adorable, Pixar, Disney", @@ -243,12 +243,12 @@ Create a pipeline from the modified blocks and load a ControlNet model. The Cont ```py pipeline = blocks.init_pipeline("Qwen/Qwen-Image", components_manager=manager) -pipeline.load_components(torch_dtype=torch.bfloat16) +pipeline.load_components(dtype=torch.bfloat16) # Load the ControlNet model controlnet_spec = pipeline.get_component_spec("controlnet") controlnet_spec.pretrained_model_name_or_path = "InstantX/Qwen-Image-ControlNet-Union" -controlnet = controlnet_spec.load(torch_dtype=torch.bfloat16) +controlnet = controlnet_spec.load(dtype=torch.bfloat16) pipeline.update_components(controlnet=controlnet) ``` diff --git a/docs/source/en/optimization/attention_backends.md b/docs/source/en/optimization/attention_backends.md index 0c67bc9e5dbf..022eb0a8830a 100644 --- a/docs/source/en/optimization/attention_backends.md +++ b/docs/source/en/optimization/attention_backends.md @@ -42,7 +42,7 @@ import torch from diffusers import QwenImagePipeline pipeline = QwenImagePipeline.from_pretrained( - "Qwen/Qwen-Image", torch_dtype=torch.bfloat16, device_map="cuda" + "Qwen/Qwen-Image", dtype=torch.bfloat16, device_map="cuda" ) pipeline.transformer.set_attention_backend("_flash_3_hub") @@ -68,7 +68,7 @@ import torch from diffusers import QwenImagePipeline pipeline = QwenImagePipeline.from_pretrained( - "Qwen/Qwen-Image", torch_dtype=torch.bfloat16, device_map="cuda" + "Qwen/Qwen-Image", dtype=torch.bfloat16, device_map="cuda" ) prompt = """ cinematic film still of a cat sipping a margarita in a pool in Palm Springs, California diff --git a/docs/source/en/optimization/cache.md b/docs/source/en/optimization/cache.md index 07db3d84b489..04f23758b88b 100644 --- a/docs/source/en/optimization/cache.md +++ b/docs/source/en/optimization/cache.md @@ -30,7 +30,7 @@ Set up and pass a [`PyramidAttentionBroadcastConfig`] to a pipeline's transforme import torch from diffusers import CogVideoXPipeline, PyramidAttentionBroadcastConfig -pipeline = CogVideoXPipeline.from_pretrained("THUDM/CogVideoX-5b", torch_dtype=torch.bfloat16) +pipeline = CogVideoXPipeline.from_pretrained("THUDM/CogVideoX-5b", dtype=torch.bfloat16) pipeline.to("cuda") config = PyramidAttentionBroadcastConfig( @@ -53,7 +53,7 @@ Set up and pass a [`FasterCacheConfig`] to a pipeline's transformer to enable it import torch from diffusers import CogVideoXPipeline, FasterCacheConfig -pipe line= CogVideoXPipeline.from_pretrained("THUDM/CogVideoX-5b", torch_dtype=torch.bfloat16) +pipe line= CogVideoXPipeline.from_pretrained("THUDM/CogVideoX-5b", dtype=torch.bfloat16) pipeline.to("cuda") config = FasterCacheConfig( @@ -78,7 +78,7 @@ from diffusers import DiffusionPipeline from diffusers.hooks import apply_first_block_cache, FirstBlockCacheConfig pipeline = DiffusionPipeline.from_pretrained( - "Qwen/Qwen-Image", torch_dtype=torch.bfloat16 + "Qwen/Qwen-Image", dtype=torch.bfloat16 ) apply_first_block_cache(pipeline.transformer, FirstBlockCacheConfig(threshold=0.2)) ``` @@ -100,7 +100,7 @@ from diffusers import FluxPipeline, TaylorSeerCacheConfig pipe = FluxPipeline.from_pretrained( "black-forest-labs/FLUX.1-dev", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ).to("cuda") config = TaylorSeerCacheConfig( @@ -129,7 +129,7 @@ from diffusers import FluxPipeline, MagCacheConfig pipe = FluxPipeline.from_pretrained( "black-forest-labs/FLUX.1-schnell", - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ).to("cuda") # 1. Calibration Step diff --git a/docs/source/en/optimization/cache_dit.md b/docs/source/en/optimization/cache_dit.md index 126142321249..52ae2cb9ad47 100644 --- a/docs/source/en/optimization/cache_dit.md +++ b/docs/source/en/optimization/cache_dit.md @@ -177,7 +177,7 @@ from diffusers import FluxPipeline pipe_or_adapter = FluxPipeline.from_pretrained( "black-forest-labs/FLUX.1-dev", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ).to("cuda") # Default options, F8B0, 8 warmup steps, and unlimited cached diff --git a/docs/source/en/optimization/deepcache.md b/docs/source/en/optimization/deepcache.md index bba22d72635a..c3b99ae44bed 100644 --- a/docs/source/en/optimization/deepcache.md +++ b/docs/source/en/optimization/deepcache.md @@ -23,7 +23,7 @@ Then load and enable the [`DeepCacheSDHelper`](https://github.com/horseee/DeepCa ```diff import torch from diffusers import StableDiffusionPipeline - pipe = StableDiffusionPipeline.from_pretrained('stable-diffusion-v1-5/stable-diffusion-v1-5', torch_dtype=torch.float16).to("cuda") + pipe = StableDiffusionPipeline.from_pretrained('stable-diffusion-v1-5/stable-diffusion-v1-5', dtype=torch.float16).to("cuda") + from DeepCache import DeepCacheSDHelper + helper = DeepCacheSDHelper(pipe=pipe) diff --git a/docs/source/en/optimization/fp16.md b/docs/source/en/optimization/fp16.md index 0e427d3a0afb..ecf0ba90ff45 100644 --- a/docs/source/en/optimization/fp16.md +++ b/docs/source/en/optimization/fp16.md @@ -32,7 +32,7 @@ import torch from diffusers import StableDiffusionXLPipeline pipeline = StableDiffusionXLPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.bfloat16 + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.bfloat16 ).to("cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" @@ -49,7 +49,7 @@ import torch from diffusers import StableDiffusionXLPipeline pipeline = StableDiffusionXLPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16 + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16 ).to("cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" @@ -70,7 +70,7 @@ from diffusers import StableDiffusionXLPipeline torch.backends.cuda.matmul.allow_tf32 = True pipeline = StableDiffusionXLPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.bfloat16 + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.bfloat16 ).to("cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" @@ -97,7 +97,7 @@ import torch from diffusers import StableDiffusionXLPipeline pipeline = StableDiffusionXLPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.bfloat16 + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.bfloat16 ).to("cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" @@ -131,7 +131,7 @@ Changing the memory layout to [channels_last](./memory#torchchannels_last) also ```py pipeline = StableDiffusionXLPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16 + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16 ).to("cuda") pipeline.unet.to(memory_format=torch.channels_last) pipeline.vae.to(memory_format=torch.channels_last) @@ -186,7 +186,7 @@ from diffusers import StableDiffusionXLPipeline pipeline = StableDiffusionXLPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16, + dtype=torch.float16, ).to("cuda") # compile only the repeated transformer layers inside the UNet @@ -212,7 +212,7 @@ from diffusers import StableDiffusionXLPipeline from accelerate.utils import compile_regions pipeline = StableDiffusionXLPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16 + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16 ).to("cuda") pipeline.unet = compile_regions(pipeline.unet, mode="reduce-overhead", fullgraph=True) ``` @@ -294,7 +294,7 @@ Filter out some linear layers in the UNet and VAE which don't benefit from dynam ```py pipeline = StableDiffusionXLPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.bfloat16 + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.bfloat16 ).to("cuda") apply_dynamic_quant(pipeline.unet, dynamic_quant_filter_fn) diff --git a/docs/source/en/optimization/habana.md b/docs/source/en/optimization/habana.md index 1e5563ae101e..da3fd55f2907 100644 --- a/docs/source/en/optimization/habana.md +++ b/docs/source/en/optimization/habana.md @@ -21,7 +21,7 @@ Call `.to("hpu")` on your pipeline to move it to a HPU device as shown below for import torch from diffusers import DiffusionPipeline -pipeline = DiffusionPipeline.from_pretrained("black-forest-labs/FLUX.1-schnell", torch_dtype=torch.bfloat16) +pipeline = DiffusionPipeline.from_pretrained("black-forest-labs/FLUX.1-schnell", dtype=torch.bfloat16) pipeline.to("hpu") image = pipeline("An image of a squirrel in Picasso style").images[0] diff --git a/docs/source/en/optimization/memory.md b/docs/source/en/optimization/memory.md index e95fdffb9e28..b335fb856c6d 100644 --- a/docs/source/en/optimization/memory.md +++ b/docs/source/en/optimization/memory.md @@ -49,12 +49,12 @@ import torch from diffusers import AutoModel, StableDiffusionXLPipeline unet = AutoModel.from_pretrained( - "username/sdxl-unet-sharded", torch_dtype=torch.float16 + "username/sdxl-unet-sharded", dtype=torch.float16 ) pipeline = StableDiffusionXLPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", unet=unet, - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") ``` @@ -76,7 +76,7 @@ from diffusers import AutoModel, StableDiffusionXLPipeline pipeline = StableDiffusionXLPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16, + dtype=torch.float16, device_map="balanced" ) ``` @@ -101,7 +101,7 @@ transformer = AutoModel.from_pretrained( "black-forest-labs/FLUX.1-dev", subfolder="transformer", device_map="auto", - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) ``` @@ -137,7 +137,7 @@ transformer = AutoModel.from_pretrained( "black-forest-labs/FLUX.1-dev", subfolder="transformer", device_map=device_map, - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) ``` @@ -150,7 +150,7 @@ from diffusers import AutoModel, StableDiffusionXLPipeline max_memory = {0:"1GB", 1:"1GB"} pipeline = StableDiffusionXLPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16, + dtype=torch.float16, device_map="balanced", max_memory=max_memory ) @@ -181,7 +181,7 @@ from diffusers import AutoModel, StableDiffusionXLPipeline pipeline = StableDiffusionXLPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16, + dtype=torch.float16, ).to("cuda") pipeline.vae.enable_slicing() pipeline(["An astronaut riding a horse on Mars"]*32).images[0] @@ -203,7 +203,7 @@ from diffusers import AutoPipelineForImage2Image from diffusers.utils import load_image pipeline = AutoPipelineForImage2Image.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16 + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16 ).to("cuda") pipeline.vae.enable_tiling() @@ -238,7 +238,7 @@ import torch from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( - "black-forest-labs/FLUX.1-schnell", torch_dtype=torch.bfloat16 + "black-forest-labs/FLUX.1-schnell", dtype=torch.bfloat16 ) pipeline.enable_sequential_cpu_offload() @@ -267,7 +267,7 @@ import torch from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( - "black-forest-labs/FLUX.1-schnell", torch_dtype=torch.bfloat16 + "black-forest-labs/FLUX.1-schnell", dtype=torch.bfloat16 ) pipeline.enable_model_cpu_offload() @@ -312,7 +312,7 @@ from diffusers.utils import export_to_video onload_device = torch.device("cuda") offload_device = torch.device("cpu") -pipeline = CogVideoXPipeline.from_pretrained("THUDM/CogVideoX-5b", torch_dtype=torch.bfloat16) +pipeline = CogVideoXPipeline.from_pretrained("THUDM/CogVideoX-5b", dtype=torch.bfloat16) pipeline.enable_group_offload( onload_device=onload_device, offload_device=offload_device, @@ -346,7 +346,7 @@ from diffusers.utils import export_to_video onload_device = torch.device("cuda") offload_device = torch.device("cpu") -pipeline = CogVideoXPipeline.from_pretrained("THUDM/CogVideoX-5b", torch_dtype=torch.bfloat16) +pipeline = CogVideoXPipeline.from_pretrained("THUDM/CogVideoX-5b", dtype=torch.bfloat16) # Use the enable_group_offload method for Diffusers model implementations pipeline.transformer.enable_group_offload(onload_device=onload_device, offload_device=offload_device, offload_type="leaf_level") @@ -424,13 +424,13 @@ from diffusers.utils import export_to_video transformer = CogVideoXTransformer3DModel.from_pretrained( "THUDM/CogVideoX-5b", subfolder="transformer", - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) transformer.enable_layerwise_casting(storage_dtype=torch.float8_e4m3fn, compute_dtype=torch.bfloat16) pipeline = CogVideoXPipeline.from_pretrained("THUDM/CogVideoX-5b", transformer=transformer, - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ).to("cuda") prompt = ( "A panda, dressed in a small, red jacket and a tiny hat, sits on a wooden stool in a serene bamboo forest. " @@ -455,7 +455,7 @@ from diffusers.hooks import apply_layerwise_casting transformer = CogVideoXTransformer3DModel.from_pretrained( "THUDM/CogVideoX-5b", subfolder="transformer", - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) # skip the normalization layer diff --git a/docs/source/en/optimization/mps.md b/docs/source/en/optimization/mps.md index b5afa25b2fda..7d975ee4999c 100644 --- a/docs/source/en/optimization/mps.md +++ b/docs/source/en/optimization/mps.md @@ -71,7 +71,7 @@ To prevent this from happening, we recommend *attention slicing* to reduce memor from diffusers import DiffusionPipeline import torch -pipeline = DiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16, variant="fp16", use_safetensors=True).to("mps") +pipeline = DiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16, variant="fp16", use_safetensors=True).to("mps") pipeline.enable_attention_slicing() ``` diff --git a/docs/source/en/optimization/para_attn.md b/docs/source/en/optimization/para_attn.md index 94b0d5ce3af4..d7f257008173 100644 --- a/docs/source/en/optimization/para_attn.md +++ b/docs/source/en/optimization/para_attn.md @@ -45,7 +45,7 @@ from diffusers import FluxPipeline pipe = FluxPipeline.from_pretrained( "black-forest-labs/FLUX.1-dev", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ).to("cuda") from para_attn.first_block_cache.diffusers_adapters import apply_cache_on_pipe @@ -90,13 +90,13 @@ model_id = "tencent/HunyuanVideo" transformer = HunyuanVideoTransformer3DModel.from_pretrained( model_id, subfolder="transformer", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, revision="refs/pr/18", ) pipe = HunyuanVideoPipeline.from_pretrained( model_id, transformer=transformer, - torch_dtype=torch.float16, + dtype=torch.float16, revision="refs/pr/18", ).to("cuda") @@ -171,7 +171,7 @@ from diffusers import FluxPipeline pipe = FluxPipeline.from_pretrained( "black-forest-labs/FLUX.1-dev", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ).to("cuda") from para_attn.first_block_cache.diffusers_adapters import apply_cache_on_pipe @@ -224,13 +224,13 @@ model_id = "tencent/HunyuanVideo" transformer = HunyuanVideoTransformer3DModel.from_pretrained( model_id, subfolder="transformer", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, revision="refs/pr/18", ) pipe = HunyuanVideoPipeline.from_pretrained( model_id, transformer=transformer, - torch_dtype=torch.float16, + dtype=torch.float16, revision="refs/pr/18", ).to("cuda") @@ -303,7 +303,7 @@ torch.cuda.set_device(dist.get_rank()) pipe = FluxPipeline.from_pretrained( "black-forest-labs/FLUX.1-dev", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ).to("cuda") from para_attn.context_parallel import init_context_parallel_mesh @@ -390,13 +390,13 @@ model_id = "tencent/HunyuanVideo" transformer = HunyuanVideoTransformer3DModel.from_pretrained( model_id, subfolder="transformer", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, revision="refs/pr/18", ) pipe = HunyuanVideoPipeline.from_pretrained( model_id, transformer=transformer, - torch_dtype=torch.float16, + dtype=torch.float16, revision="refs/pr/18", ).to("cuda") diff --git a/docs/source/en/optimization/pruna.md b/docs/source/en/optimization/pruna.md index 56c1f3af5957..694abee03e42 100644 --- a/docs/source/en/optimization/pruna.md +++ b/docs/source/en/optimization/pruna.md @@ -58,7 +58,7 @@ from pruna import PrunaModel, SmashConfig, smash # Try segmind/Segmind-Vega or black-forest-labs/FLUX.1-schnell with a small GPU memory pipe = FluxPipeline.from_pretrained( "black-forest-labs/FLUX.1-dev", - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ).to("cuda") # define the configuration @@ -165,7 +165,7 @@ from pruna import PrunaModel # Try PrunaAI/Segmind-Vega-smashed or PrunaAI/FLUX.1-dev-smashed with a small GPU memory pipe = FluxPipeline.from_pretrained( "black-forest-labs/FLUX.1-dev", - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ).to("cpu") wrapped_pipe = PrunaModel(model=pipe) ``` diff --git a/docs/source/en/optimization/speed-memory-optims.md b/docs/source/en/optimization/speed-memory-optims.md index 08cf933494a5..63b552a81faa 100644 --- a/docs/source/en/optimization/speed-memory-optims.md +++ b/docs/source/en/optimization/speed-memory-optims.md @@ -61,7 +61,7 @@ pipeline_quant_config = PipelineQuantizationConfig( pipeline = DiffusionPipeline.from_pretrained( "black-forest-labs/FLUX.1-dev", quantization_config=pipeline_quant_config, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ).to("cuda") # compile @@ -102,7 +102,7 @@ pipeline_quant_config = PipelineQuantizationConfig( pipeline = DiffusionPipeline.from_pretrained( "black-forest-labs/FLUX.1-dev", quantization_config=pipeline_quant_config, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ).to("cuda") # model CPU offloading @@ -142,12 +142,12 @@ pipeline_quant_config = PipelineQuantizationConfig( ) text_encoder = UMT5EncoderModel.from_pretrained( - "Wan-AI/Wan2.1-T2V-14B-Diffusers", subfolder="text_encoder", torch_dtype=torch.bfloat16 + "Wan-AI/Wan2.1-T2V-14B-Diffusers", subfolder="text_encoder", dtype=torch.bfloat16 ) pipeline = DiffusionPipeline.from_pretrained( "Wan-AI/Wan2.1-T2V-14B-Diffusers", quantization_config=pipeline_quant_config, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ).to("cuda") # group offloading diff --git a/docs/source/en/optimization/tgate.md b/docs/source/en/optimization/tgate.md index 90e0bc32f71b..9962e956cd56 100644 --- a/docs/source/en/optimization/tgate.md +++ b/docs/source/en/optimization/tgate.md @@ -34,7 +34,7 @@ import torch from diffusers import PixArtAlphaPipeline from tgate import TgatePixArtLoader -pipe = PixArtAlphaPipeline.from_pretrained("PixArt-alpha/PixArt-XL-2-1024-MS", torch_dtype=torch.float16) +pipe = PixArtAlphaPipeline.from_pretrained("PixArt-alpha/PixArt-XL-2-1024-MS", dtype=torch.float16) gate_step = 8 inference_step = 25 @@ -63,7 +63,7 @@ from tgate import TgateSDXLLoader pipe = StableDiffusionXLPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16, + dtype=torch.float16, variant="fp16", use_safetensors=True, ) @@ -96,7 +96,7 @@ from tgate import TgateSDXLDeepCacheLoader pipe = StableDiffusionXLPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16, + dtype=torch.float16, variant="fp16", use_safetensors=True, ) @@ -130,13 +130,13 @@ from tgate import TgateSDXLLoader unet = UNet2DConditionModel.from_pretrained( "latent-consistency/lcm-sdxl", - torch_dtype=torch.float16, + dtype=torch.float16, variant="fp16", ) pipe = StableDiffusionXLPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", unet=unet, - torch_dtype=torch.float16, + dtype=torch.float16, variant="fp16", ) pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config) diff --git a/docs/source/en/optimization/tome.md b/docs/source/en/optimization/tome.md index ab368c9ccbb9..74dbf7e6c961 100644 --- a/docs/source/en/optimization/tome.md +++ b/docs/source/en/optimization/tome.md @@ -28,7 +28,7 @@ You can use ToMe from the [`tomesd`](https://github.com/dbolya/tomesd) library w import tomesd pipeline = StableDiffusionPipeline.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16, use_safetensors=True, + "stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16, use_safetensors=True, ).to("cuda") + tomesd.apply_patch(pipeline, ratio=0.5) diff --git a/docs/source/en/optimization/xdit.md b/docs/source/en/optimization/xdit.md index ecf45635684a..1c66b34da69c 100644 --- a/docs/source/en/optimization/xdit.md +++ b/docs/source/en/optimization/xdit.md @@ -37,7 +37,7 @@ Here's an example of using xDiT to accelerate inference of a Diffusers model. local_rank = get_world_group().local_rank pipe = StableDiffusion3Pipeline.from_pretrained( pretrained_model_name_or_path=engine_config.model_config.model, - torch_dtype=torch.float16, + dtype=torch.float16, ).to(f"cuda:{local_rank}") # do anything you want with pipeline here diff --git a/docs/source/en/quantization/autoround.md b/docs/source/en/quantization/autoround.md index f4fcf1a780c3..6dc4c514141c 100644 --- a/docs/source/en/quantization/autoround.md +++ b/docs/source/en/quantization/autoround.md @@ -42,7 +42,7 @@ pipeline_quant_config = PipelineQuantizationConfig( pipe = DiffusionPipeline.from_pretrained( "INCModel/Z-Image-W4A16-AutoRound", quantization_config=pipeline_quant_config, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, device_map="cuda", ) @@ -63,14 +63,14 @@ transformer = ZImageTransformer2DModel.from_pretrained( model_id, subfolder="transformer", quantization_config=quantization_config, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, device_map="cuda", ) pipe = ZImagePipeline.from_pretrained( model_id, transformer=transformer, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, device_map="cuda", ) @@ -95,7 +95,7 @@ pipeline_quant_config = PipelineQuantizationConfig( pipe = DiffusionPipeline.from_pretrained( "INCModel/Z-Image-W4A16-AutoRound", quantization_config=pipeline_quant_config, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, device_map="cuda", ) @@ -171,7 +171,7 @@ model_id = "INCModel/Z-Image-W4A16-AutoRound" # The inference backend will be automatically selected. pipe = ZImagePipeline.from_pretrained( model_id, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, device_map="cuda", ) diff --git a/docs/source/en/quantization/bitsandbytes.md b/docs/source/en/quantization/bitsandbytes.md index 072947274463..bda70115d3fc 100644 --- a/docs/source/en/quantization/bitsandbytes.md +++ b/docs/source/en/quantization/bitsandbytes.md @@ -40,7 +40,7 @@ Quantizing a model in 8-bit halves the memory-usage: bitsandbytes is supported in both Transformers and Diffusers, so you can quantize both the [`FluxTransformer2DModel`] and [`~transformers.T5EncoderModel`]. -For Ada and higher-series GPUs. we recommend changing `torch_dtype` to `torch.bfloat16`. +For Ada and higher-series GPUs. we recommend changing `dtype` to `torch.bfloat16`. > [!TIP] > The [`CLIPTextModel`] and [`AutoencoderKL`] aren't quantized because they're already small in size and because [`AutoencoderKL`] only has a few `torch.nn.Linear` layers. @@ -58,7 +58,7 @@ text_encoder_2_8bit = T5EncoderModel.from_pretrained( "black-forest-labs/FLUX.1-dev", subfolder="text_encoder_2", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) quant_config = DiffusersBitsAndBytesConfig(load_in_8bit=True,) @@ -67,18 +67,18 @@ transformer_8bit = AutoModel.from_pretrained( "black-forest-labs/FLUX.1-dev", subfolder="transformer", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) ``` -By default, all the other modules such as `torch.nn.LayerNorm` are converted to `torch.float16`. You can change the data type of these modules with the `torch_dtype` parameter. +By default, all the other modules such as `torch.nn.LayerNorm` are converted to `torch.float16`. You can change the data type of these modules with the `dtype` parameter. ```diff transformer_8bit = AutoModel.from_pretrained( "black-forest-labs/FLUX.1-dev", subfolder="transformer", quantization_config=quant_config, -+ torch_dtype=torch.float32, ++ dtype=torch.float32, ) ``` @@ -94,7 +94,7 @@ pipe = FluxPipeline.from_pretrained( "black-forest-labs/FLUX.1-dev", transformer=transformer_8bit, text_encoder_2=text_encoder_2_8bit, - torch_dtype=torch.float16, + dtype=torch.float16, device_map="auto", ) @@ -126,7 +126,7 @@ Quantizing a model in 4-bit reduces your memory-usage by 4x: bitsandbytes is supported in both Transformers and Diffusers, so you can can quantize both the [`FluxTransformer2DModel`] and [`~transformers.T5EncoderModel`]. -For Ada and higher-series GPUs. we recommend changing `torch_dtype` to `torch.bfloat16`. +For Ada and higher-series GPUs. we recommend changing `dtype` to `torch.bfloat16`. > [!TIP] > The [`CLIPTextModel`] and [`AutoencoderKL`] aren't quantized because they're already small in size and because [`AutoencoderKL`] only has a few `torch.nn.Linear` layers. @@ -144,7 +144,7 @@ text_encoder_2_4bit = T5EncoderModel.from_pretrained( "black-forest-labs/FLUX.1-dev", subfolder="text_encoder_2", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) quant_config = DiffusersBitsAndBytesConfig(load_in_4bit=True,) @@ -153,18 +153,18 @@ transformer_4bit = AutoModel.from_pretrained( "black-forest-labs/FLUX.1-dev", subfolder="transformer", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) ``` -By default, all the other modules such as `torch.nn.LayerNorm` are converted to `torch.float16`. You can change the data type of these modules with the `torch_dtype` parameter. +By default, all the other modules such as `torch.nn.LayerNorm` are converted to `torch.float16`. You can change the data type of these modules with the `dtype` parameter. ```diff transformer_4bit = AutoModel.from_pretrained( "black-forest-labs/FLUX.1-dev", subfolder="transformer", quantization_config=quant_config, -+ torch_dtype=torch.float32, ++ dtype=torch.float32, ) ``` @@ -179,7 +179,7 @@ pipe = FluxPipeline.from_pretrained( "black-forest-labs/FLUX.1-dev", transformer=transformer_4bit, text_encoder_2=text_encoder_2_4bit, - torch_dtype=torch.float16, + dtype=torch.float16, device_map="auto", ) @@ -314,7 +314,7 @@ text_encoder_2_4bit = T5EncoderModel.from_pretrained( "black-forest-labs/FLUX.1-dev", subfolder="text_encoder_2", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) quant_config = DiffusersBitsAndBytesConfig( @@ -326,11 +326,11 @@ transformer_4bit = AutoModel.from_pretrained( "black-forest-labs/FLUX.1-dev", subfolder="transformer", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) ``` -For inference, the `bnb_4bit_quant_type` does not have a huge impact on performance. However, to remain consistent with the model weights, you should use the `bnb_4bit_compute_dtype` and `torch_dtype` values. +For inference, the `bnb_4bit_quant_type` does not have a huge impact on performance. However, to remain consistent with the model weights, you should use the `bnb_4bit_compute_dtype` and `dtype` values. ### Nested quantization @@ -352,7 +352,7 @@ text_encoder_2_4bit = T5EncoderModel.from_pretrained( "black-forest-labs/FLUX.1-dev", subfolder="text_encoder_2", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) quant_config = DiffusersBitsAndBytesConfig( @@ -364,7 +364,7 @@ transformer_4bit = AutoModel.from_pretrained( "black-forest-labs/FLUX.1-dev", subfolder="transformer", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) ``` @@ -388,7 +388,7 @@ text_encoder_2_4bit = T5EncoderModel.from_pretrained( "black-forest-labs/FLUX.1-dev", subfolder="text_encoder_2", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) quant_config = DiffusersBitsAndBytesConfig( @@ -400,7 +400,7 @@ transformer_4bit = AutoModel.from_pretrained( "black-forest-labs/FLUX.1-dev", subfolder="transformer", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) text_encoder_2_4bit.dequantize() @@ -421,7 +421,7 @@ transformer_4bit = AutoModel.from_pretrained( "black-forest-labs/FLUX.1-dev", subfolder="transformer", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) transformer_4bit.compile(fullgraph=True) ``` @@ -435,7 +435,7 @@ transformer_4bit = AutoModel.from_pretrained( "black-forest-labs/FLUX.1-dev", subfolder="transformer", quantization_config=quant_config, - torch_dtype=torch.float16, + dtype=torch.float16, ) transformer_4bit.compile(fullgraph=True) ``` diff --git a/docs/source/en/quantization/gguf.md b/docs/source/en/quantization/gguf.md index 47804c102da2..6ee91e2b272f 100644 --- a/docs/source/en/quantization/gguf.md +++ b/docs/source/en/quantization/gguf.md @@ -40,12 +40,12 @@ ckpt_path = ( transformer = FluxTransformer2DModel.from_single_file( ckpt_path, quantization_config=GGUFQuantizationConfig(compute_dtype=torch.bfloat16), - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) pipe = FluxPipeline.from_pretrained( "black-forest-labs/FLUX.1-dev", transformer=transformer, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) pipe.enable_model_cpu_offload() prompt = "A cat holding a sign that says hello world" @@ -103,12 +103,12 @@ transformer = FluxTransformer2DModel.from_single_file( quantization_config=GGUFQuantizationConfig(compute_dtype=torch.bfloat16), config="black-forest-labs/FLUX.1-dev", subfolder="transformer", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) pipe = FluxPipeline.from_pretrained( "black-forest-labs/FLUX.1-dev", transformer=transformer, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) pipe.enable_model_cpu_offload() prompt = "A cat holding a sign that says hello world" diff --git a/docs/source/en/quantization/modelopt.md b/docs/source/en/quantization/modelopt.md index c7fca9d44491..d6eae0bef4c7 100644 --- a/docs/source/en/quantization/modelopt.md +++ b/docs/source/en/quantization/modelopt.md @@ -35,12 +35,12 @@ transformer = AutoModel.from_pretrained( model_id, subfolder="transformer", quantization_config=quantization_config, - torch_dtype=dtype, + dtype=dtype, ) pipe = SanaPipeline.from_pretrained( model_id, transformer=transformer, - torch_dtype=dtype, + dtype=dtype, ) pipe.to("cuda") @@ -106,7 +106,7 @@ model = AutoModel.from_pretrained( model_id, subfolder="transformer", quantization_config=quant_config_fp8, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) model.save_pretrained('path/to/sana_fp8', safe_serialization=False) ``` @@ -125,12 +125,12 @@ transformer = AutoModel.from_pretrained( "path/to/sana_fp8", subfolder="transformer", quantization_config=quantization_config, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) pipe = SanaPipeline.from_pretrained( "Efficient-Large-Model/Sana_600M_1024px_diffusers", transformer=transformer, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) pipe.to("cuda") prompt = "A cat holding a sign that says hello world" diff --git a/docs/source/en/quantization/nunchaku.md b/docs/source/en/quantization/nunchaku.md index 22653b888326..1cdad66917b5 100644 --- a/docs/source/en/quantization/nunchaku.md +++ b/docs/source/en/quantization/nunchaku.md @@ -39,7 +39,7 @@ from diffusers import DiffusionPipeline model_id = "rootonchair/ERNIE-Image-Turbo-nunchaku-lite-nvfp4" pipe = DiffusionPipeline.from_pretrained( - model_id, torch_dtype=torch.bfloat16, + model_id, dtype=torch.bfloat16, ).to("cuda") prompt = "A modern red armchair in a quiet studio, soft window light, realistic product photography" diff --git a/docs/source/en/quantization/overview.md b/docs/source/en/quantization/overview.md index ff5657917e78..33d75ab6342e 100644 --- a/docs/source/en/quantization/overview.md +++ b/docs/source/en/quantization/overview.md @@ -58,7 +58,7 @@ Pass the `pipeline_quant_config` to [`~DiffusionPipeline.from_pretrained`] to qu pipe = DiffusionPipeline.from_pretrained( "black-forest-labs/FLUX.1-dev", quantization_config=pipeline_quant_config, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ).to("cuda") image = pipe("photo of a cute dog").images[0] @@ -119,7 +119,7 @@ Pass the `pipeline_quant_config` to [`~DiffusionPipeline.from_pretrained`] to qu pipe = DiffusionPipeline.from_pretrained( "black-forest-labs/FLUX.1-dev", quantization_config=pipeline_quant_config, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ).to("cuda") image = pipe("photo of a cute dog").images[0] @@ -135,7 +135,7 @@ from diffusers import DiffusionPipeline pipe = DiffusionPipeline.from_pretrained( "rootonchair/ERNIE-Image-Turbo-nunchaku-lite-nvfp4", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ).to("cuda") ``` diff --git a/docs/source/en/quantization/quanto.md b/docs/source/en/quantization/quanto.md index d322d76be267..d41b238961d4 100644 --- a/docs/source/en/quantization/quanto.md +++ b/docs/source/en/quantization/quanto.md @@ -38,10 +38,10 @@ transformer = FluxTransformer2DModel.from_pretrained( model_id, subfolder="transformer", quantization_config=quantization_config, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) -pipe = FluxPipeline.from_pretrained(model_id, transformer=transformer, torch_dtype=torch_dtype) +pipe = FluxPipeline.from_pretrained(model_id, transformer=transformer, dtype=dtype) pipe.to("cuda") prompt = "A cat holding a sign that says hello world" @@ -65,7 +65,7 @@ transformer = FluxTransformer2DModel.from_pretrained( model_id, subfolder="transformer", quantization_config=quantization_config, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) ``` @@ -79,7 +79,7 @@ from diffusers import FluxTransformer2DModel, QuantoConfig ckpt_path = "https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/flux1-dev.safetensors" quantization_config = QuantoConfig(weights_dtype="float8") -transformer = FluxTransformer2DModel.from_single_file(ckpt_path, quantization_config=quantization_config, torch_dtype=torch.bfloat16) +transformer = FluxTransformer2DModel.from_single_file(ckpt_path, quantization_config=quantization_config, dtype=torch.bfloat16) ``` ## Saving Quantized models @@ -99,7 +99,7 @@ transformer = FluxTransformer2DModel.from_pretrained( model_id, subfolder="transformer", quantization_config=quantization_config, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) # save quantized model to reuse transformer.save_pretrained("") @@ -124,12 +124,12 @@ transformer = FluxTransformer2DModel.from_pretrained( model_id, subfolder="transformer", quantization_config=quantization_config, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) transformer = torch.compile(transformer, mode="max-autotune", fullgraph=True) pipe = FluxPipeline.from_pretrained( - model_id, transformer=transformer, torch_dtype=torch_dtype + model_id, transformer=transformer, dtype=dtype ) pipe.to("cuda") images = pipe("A cat holding a sign that says hello").images[0] diff --git a/docs/source/en/quantization/torchao.md b/docs/source/en/quantization/torchao.md index 1fdcb7879aaa..8e647c73242d 100644 --- a/docs/source/en/quantization/torchao.md +++ b/docs/source/en/quantization/torchao.md @@ -34,7 +34,7 @@ pipeline_quant_config = PipelineQuantizationConfig( pipeline = DiffusionPipeline.from_pretrained( "black-forest-labs/FLUX.1-dev", quantization_config=pipeline_quant_config, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, device_map="cuda" ) ``` @@ -54,7 +54,7 @@ pipeline_quant_config = PipelineQuantizationConfig( pipeline = DiffusionPipeline.from_pretrained( "black-forest-labs/FLUX.1-dev", quantization_config=pipeline_quant_config, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, device_map="cuda" ) @@ -98,7 +98,7 @@ transformer = AutoModel.from_pretrained( "black-forest-labs/Flux.1-Dev", subfolder="transformer", quantization_config=quantization_config, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) transformer.save_pretrained("/path/to/flux_int8wo", safe_serialization=False) ``` @@ -109,8 +109,8 @@ To load a serialized quantized model, use the [`~ModelMixin.from_pretrained`] me import torch from diffusers import FluxPipeline, AutoModel -transformer = AutoModel.from_pretrained("/path/to/flux_int8wo", torch_dtype=torch.bfloat16, use_safetensors=False) -pipe = FluxPipeline.from_pretrained("black-forest-labs/Flux.1-Dev", transformer=transformer, torch_dtype=torch.bfloat16) +transformer = AutoModel.from_pretrained("/path/to/flux_int8wo", dtype=torch.bfloat16, use_safetensors=False) +pipe = FluxPipeline.from_pretrained("black-forest-labs/Flux.1-Dev", transformer=transformer, dtype=torch.bfloat16) pipe.to("cuda") prompt = "A cat holding a sign that says hello world" @@ -131,7 +131,7 @@ transformer = AutoModel.from_pretrained( "black-forest-labs/Flux.1-Dev", subfolder="transformer", quantization_config=TorchAoConfig(IntxWeightOnlyConfig(dtype=torch.uint4)), - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) transformer.save_pretrained("/path/to/flux_uint4wo", safe_serialization=False, max_shard_size="50GB") # ... diff --git a/docs/source/en/quicktour.md b/docs/source/en/quicktour.md index 897120aa2f87..341608852300 100644 --- a/docs/source/en/quicktour.md +++ b/docs/source/en/quicktour.md @@ -53,7 +53,7 @@ import torch from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( - "Qwen/Qwen-Image", torch_dtype=torch.bfloat16, device_map="cuda" + "Qwen/Qwen-Image", dtype=torch.bfloat16, device_map="cuda" ) prompt = """ @@ -77,12 +77,12 @@ from diffusers.utils import export_to_video vae = AutoencoderKLWan.from_pretrained( "Wan-AI/Wan2.2-T2V-A14B-Diffusers", subfolder="vae", - torch_dtype=torch.float32 + dtype=torch.float32 ) pipeline = DiffusionPipeline.from_pretrained( "Wan-AI/Wan2.2-T2V-A14B-Diffusers", vae=vae - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, device_map="cuda" ) @@ -110,7 +110,7 @@ import torch from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( - "Qwen/Qwen-Image", torch_dtype=torch.bfloat16, device_map="cuda" + "Qwen/Qwen-Image", dtype=torch.bfloat16, device_map="cuda" ) pipeline.load_lora_weights( "flymy-ai/qwen-image-realism-lora", @@ -145,7 +145,7 @@ quant_config = PipelineQuantizationConfig( ) pipeline = DiffusionPipeline.from_pretrained( "Qwen/Qwen-Image", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, quantization_config=quant_config, device_map="cuda" ) @@ -187,7 +187,7 @@ quant_config = PipelineQuantizationConfig( ) pipeline = DiffusionPipeline.from_pretrained( "Qwen/Qwen-Image", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, quantization_config=quant_config, device_map="cuda" ) @@ -216,7 +216,7 @@ import torch from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( - "Qwen/Qwen-Image", torch_dtype=torch.bfloat16, device_map="cuda" + "Qwen/Qwen-Image", dtype=torch.bfloat16, device_map="cuda" ) pipeline.transformer.compile_repeated_blocks( diff --git a/docs/source/en/stable_diffusion.md b/docs/source/en/stable_diffusion.md index 93e399d3db88..876526105fc7 100644 --- a/docs/source/en/stable_diffusion.md +++ b/docs/source/en/stable_diffusion.md @@ -30,7 +30,7 @@ from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, device_map="cuda" ) pipeline.enable_model_cpu_offload() @@ -48,7 +48,7 @@ print(f"Max memory reserved: {torch.cuda.max_memory_allocated() / 1024**3:.2f} G Denoising is the most computationally demanding process during diffusion. Methods that optimizes this process accelerates inference speed. Try the following methods for a speed up. - Add `device_map="cuda"` to place the pipeline on a GPU. Placing a model on an accelerator, like a GPU, increases speed because it performs computations in parallel. -- Set `torch_dtype=torch.bfloat16` to execute the pipeline in half-precision. Reducing the data type precision increases speed because it takes less time to perform computations in a lower precision. +- Set `dtype=torch.bfloat16` to execute the pipeline in half-precision. Reducing the data type precision increases speed because it takes less time to perform computations in a lower precision. ```py import torch @@ -57,7 +57,7 @@ from diffusers import DiffusionPipeline, DPMSolverMultistepScheduler pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, device_map="cuda ) ``` @@ -92,7 +92,7 @@ Many modern diffusion models deliver high-quality images out-of-the-box. However pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, device_map="cuda" ) @@ -114,7 +114,7 @@ Many modern diffusion models deliver high-quality images out-of-the-box. However pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, device_map="cuda" ) pipeline.scheduler = HeunDiscreteScheduler.from_config(pipeline.scheduler.config) diff --git a/docs/source/en/training/cogvideox.md b/docs/source/en/training/cogvideox.md index d0700c4da763..7de41be0aa95 100644 --- a/docs/source/en/training/cogvideox.md +++ b/docs/source/en/training/cogvideox.md @@ -232,7 +232,7 @@ import torch from diffusers import CogVideoXPipeline from diffusers.utils import export_to_video -pipe = CogVideoXPipeline.from_pretrained("THUDM/CogVideoX-2b", torch_dtype=torch.float16) +pipe = CogVideoXPipeline.from_pretrained("THUDM/CogVideoX-2b", dtype=torch.float16) # pipe.load_lora_weights("/path/to/lora/weights", adapter_name="cogvideox-lora") # Or, pipe.load_lora_weights("my-awesome-hf-username/my-awesome-lora-name", adapter_name="cogvideox-lora") # If loading from the HF Hub pipe.to("cuda") diff --git a/docs/source/en/training/controlnet.md b/docs/source/en/training/controlnet.md index 840130d2b43c..eeb1ecdd0d46 100644 --- a/docs/source/en/training/controlnet.md +++ b/docs/source/en/training/controlnet.md @@ -252,9 +252,9 @@ from diffusers import StableDiffusionControlNetPipeline, ControlNetModel from diffusers.utils import load_image import torch -controlnet = ControlNetModel.from_pretrained("path/to/controlnet", torch_dtype=torch.float16) +controlnet = ControlNetModel.from_pretrained("path/to/controlnet", dtype=torch.float16) pipeline = StableDiffusionControlNetPipeline.from_pretrained( - "path/to/base/model", controlnet=controlnet, torch_dtype=torch.float16 + "path/to/base/model", controlnet=controlnet, dtype=torch.float16 ).to("cuda") control_image = load_image("./conditioning_image_1.png") diff --git a/docs/source/en/training/custom_diffusion.md b/docs/source/en/training/custom_diffusion.md index bfa4fe6f9e66..0e7a439c461a 100644 --- a/docs/source/en/training/custom_diffusion.md +++ b/docs/source/en/training/custom_diffusion.md @@ -305,7 +305,7 @@ import torch from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( - "CompVis/stable-diffusion-v1-4", torch_dtype=torch.float16, + "CompVis/stable-diffusion-v1-4", dtype=torch.float16, ).to("cuda") pipeline.unet.load_attn_procs("path-to-save-model", weight_name="pytorch_custom_diffusion_weights.bin") pipeline.load_textual_inversion("path-to-save-model", weight_name=".bin") @@ -328,7 +328,7 @@ from huggingface_hub.repocard import RepoCard from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( - "CompVis/stable-diffusion-v1-4", torch_dtype=torch.float16, + "CompVis/stable-diffusion-v1-4", dtype=torch.float16, ).to("cuda") model_id = "sayakpaul/custom-diffusion-cat-wooden-pot" pipeline.unet.load_attn_procs(model_id, weight_name="pytorch_custom_diffusion_weights.bin") diff --git a/docs/source/en/training/distributed_inference.md b/docs/source/en/training/distributed_inference.md index 08b0262a9ef9..a0a07b7f678a 100644 --- a/docs/source/en/training/distributed_inference.md +++ b/docs/source/en/training/distributed_inference.md @@ -36,7 +36,7 @@ from accelerate import PartialState from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( - "Qwen/Qwen-Image", torch_dtype=torch.float16 + "Qwen/Qwen-Image", dtype=torch.float16 ) distributed_state = PartialState() pipeline.to(distributed_state.device) @@ -73,7 +73,7 @@ import torch.multiprocessing as mp from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( - "Qwen/Qwen-Image", torch_dtype=torch.float16, + "Qwen/Qwen-Image", dtype=torch.float16, ) ``` @@ -135,7 +135,7 @@ pipeline = FluxPipeline.from_pretrained( vae=None, device_map="balanced", max_memory={0: "16GB", 1: "16GB"}, - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) with torch.no_grad(): print("Encoding prompts.") @@ -174,7 +174,7 @@ transformer = AutoModel.from_pretrained( "black-forest-labs/FLUX.1-dev", subfolder="transformer", device_map="auto", - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) ``` @@ -192,7 +192,7 @@ pipeline = FluxPipeline.from_pretrained( tokenizer_2=None, vae=None, transformer=transformer, - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) print("Running denoising.") @@ -215,7 +215,7 @@ import torch from diffusers import AutoencoderKL from diffusers.image_processor import VaeImageProcessor -vae = AutoencoderKL.from_pretrained(ckpt_id, subfolder="vae", torch_dtype=torch.bfloat16).to("cuda") +vae = AutoencoderKL.from_pretrained(ckpt_id, subfolder="vae", dtype=torch.bfloat16).to("cuda") vae_scale_factor = 2 ** (len(vae.config.block_out_channels) - 1) image_processor = VaeImageProcessor(vae_scale_factor=vae_scale_factor) @@ -263,7 +263,7 @@ def main(): world_size = dist.get_world_size() pipeline = DiffusionPipeline.from_pretrained( - "black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16 + "black-forest-labs/FLUX.1-dev", dtype=torch.bfloat16 ).to(device) pipeline.transformer.set_attention_backend("_native_cudnn") @@ -423,11 +423,11 @@ cp_config = ContextParallelConfig(ring_degree=2) transformer = AutoModel.from_pretrained( CKPT_ID, subfolder="transformer", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, parallel_config=cp_config ) pipeline = DiffusionPipeline.from_pretrained( - CKPT_ID, transformer=transformer, torch_dtype=torch.bfloat16, + CKPT_ID, transformer=transformer, dtype=torch.bfloat16, ).to(device) ``` diff --git a/docs/source/en/training/dreambooth.md b/docs/source/en/training/dreambooth.md index 2302739a0e34..ddcc92c2059d 100644 --- a/docs/source/en/training/dreambooth.md +++ b/docs/source/en/training/dreambooth.md @@ -335,7 +335,7 @@ Once training is complete, you can use your newly trained model for inference! from diffusers import DiffusionPipeline import torch -pipeline = DiffusionPipeline.from_pretrained("path_to_saved_model", torch_dtype=torch.float16, use_safetensors=True).to("cuda") +pipeline = DiffusionPipeline.from_pretrained("path_to_saved_model", dtype=torch.float16, use_safetensors=True).to("cuda") image = pipeline("A photo of sks dog in a bucket", num_inference_steps=50, guidance_scale=7.5).images[0] image.save("dog-bucket.png") ``` diff --git a/docs/source/en/training/instructpix2pix.md b/docs/source/en/training/instructpix2pix.md index a1c94bb33ffe..28e72ad3c3c8 100644 --- a/docs/source/en/training/instructpix2pix.md +++ b/docs/source/en/training/instructpix2pix.md @@ -208,7 +208,7 @@ import torch from diffusers import StableDiffusionInstructPix2PixPipeline from diffusers.utils import load_image -pipeline = StableDiffusionInstructPix2PixPipeline.from_pretrained("your_cool_model", torch_dtype=torch.float16).to("cuda") +pipeline = StableDiffusionInstructPix2PixPipeline.from_pretrained("your_cool_model", dtype=torch.float16).to("cuda") generator = torch.Generator("cuda").manual_seed(0) image = load_image("https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/test_pix2pix_4.png") diff --git a/docs/source/en/training/kandinsky.md b/docs/source/en/training/kandinsky.md index afed0b17568e..89532c556031 100644 --- a/docs/source/en/training/kandinsky.md +++ b/docs/source/en/training/kandinsky.md @@ -108,10 +108,10 @@ tokenizer = CLIPTokenizer.from_pretrained(args.pretrained_prior_model_name_or_pa with ContextManagers(deepspeed_zero_init_disabled_context_manager()): image_encoder = CLIPVisionModelWithProjection.from_pretrained( - args.pretrained_prior_model_name_or_path, subfolder="image_encoder", torch_dtype=weight_dtype + args.pretrained_prior_model_name_or_path, subfolder="image_encoder", dtype=weight_dtype ).eval() text_encoder = CLIPTextModelWithProjection.from_pretrained( - args.pretrained_prior_model_name_or_path, subfolder="text_encoder", torch_dtype=weight_dtype + args.pretrained_prior_model_name_or_path, subfolder="text_encoder", dtype=weight_dtype ).eval() ``` @@ -163,10 +163,10 @@ Unlike the prior model, the decoder initializes a [`VQModel`] to decode the late ```py with ContextManagers(deepspeed_zero_init_disabled_context_manager()): vae = VQModel.from_pretrained( - args.pretrained_decoder_model_name_or_path, subfolder="movq", torch_dtype=weight_dtype + args.pretrained_decoder_model_name_or_path, subfolder="movq", dtype=weight_dtype ).eval() image_encoder = CLIPVisionModelWithProjection.from_pretrained( - args.pretrained_prior_model_name_or_path, subfolder="image_encoder", torch_dtype=weight_dtype + args.pretrained_prior_model_name_or_path, subfolder="image_encoder", dtype=weight_dtype ).eval() unet = UNet2DConditionModel.from_pretrained(args.pretrained_decoder_model_name_or_path, subfolder="unet") ``` @@ -262,9 +262,9 @@ Once training is finished, you can use your newly trained model for inference! from diffusers import AutoPipelineForText2Image, DiffusionPipeline import torch -prior_pipeline = DiffusionPipeline.from_pretrained(output_dir, torch_dtype=torch.float16) +prior_pipeline = DiffusionPipeline.from_pretrained(output_dir, dtype=torch.float16) prior_components = {"prior_" + k: v for k,v in prior_pipeline.components.items()} -pipeline = AutoPipelineForText2Image.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", **prior_components, torch_dtype=torch.float16) +pipeline = AutoPipelineForText2Image.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", **prior_components, dtype=torch.float16) pipe.enable_model_cpu_offload() prompt="A robot naruto, 4k photo" @@ -281,7 +281,7 @@ image = pipeline(prompt=prompt, negative_prompt=negative_prompt).images[0] from diffusers import AutoPipelineForText2Image import torch -pipeline = AutoPipelineForText2Image.from_pretrained("path/to/saved/model", torch_dtype=torch.float16) +pipeline = AutoPipelineForText2Image.from_pretrained("path/to/saved/model", dtype=torch.float16) pipeline.enable_model_cpu_offload() prompt="A robot naruto, 4k photo" @@ -295,7 +295,7 @@ from diffusers import AutoPipelineForText2Image, UNet2DConditionModel unet = UNet2DConditionModel.from_pretrained("path/to/saved/model" + "/checkpoint-/unet") -pipeline = AutoPipelineForText2Image.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", unet=unet, torch_dtype=torch.float16) +pipeline = AutoPipelineForText2Image.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", unet=unet, dtype=torch.float16) pipeline.enable_model_cpu_offload() image = pipeline(prompt="A robot naruto, 4k photo").images[0] diff --git a/docs/source/en/training/lcm_distill.md b/docs/source/en/training/lcm_distill.md index 383368c7ca08..414551d1f2ef 100644 --- a/docs/source/en/training/lcm_distill.md +++ b/docs/source/en/training/lcm_distill.md @@ -218,8 +218,8 @@ Once training is complete, you can use your new LCM for inference. from diffusers import UNet2DConditionModel, DiffusionPipeline, LCMScheduler import torch -unet = UNet2DConditionModel.from_pretrained("your-username/your-model", torch_dtype=torch.float16, variant="fp16") -pipeline = DiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", unet=unet, torch_dtype=torch.float16, variant="fp16") +unet = UNet2DConditionModel.from_pretrained("your-username/your-model", dtype=torch.float16, variant="fp16") +pipeline = DiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", unet=unet, dtype=torch.float16, variant="fp16") pipeline.scheduler = LCMScheduler.from_config(pipe.scheduler.config) pipeline.to("cuda") diff --git a/docs/source/en/training/lora.md b/docs/source/en/training/lora.md index efb170e329d4..33bc7e6a931d 100644 --- a/docs/source/en/training/lora.md +++ b/docs/source/en/training/lora.md @@ -189,7 +189,7 @@ Once training has been completed, you can use your model for inference: from diffusers import AutoPipelineForText2Image import torch -pipeline = AutoPipelineForText2Image.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16).to("cuda") +pipeline = AutoPipelineForText2Image.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16).to("cuda") pipeline.load_lora_weights("path/to/lora/model", weight_name="pytorch_lora_weights.safetensors") image = pipeline("A naruto with blue eyes").images[0] ``` diff --git a/docs/source/en/training/sdxl.md b/docs/source/en/training/sdxl.md index 0dbd8b883004..5da04e06fda7 100644 --- a/docs/source/en/training/sdxl.md +++ b/docs/source/en/training/sdxl.md @@ -212,7 +212,7 @@ After you've finished training, you can use your newly trained SDXL model for in from diffusers import DiffusionPipeline import torch -pipeline = DiffusionPipeline.from_pretrained("path/to/your/model", torch_dtype=torch.float16).to("cuda") +pipeline = DiffusionPipeline.from_pretrained("path/to/your/model", dtype=torch.float16).to("cuda") prompt = "A naruto with green eyes and red legs." image = pipeline(prompt, num_inference_steps=30, guidance_scale=7.5).images[0] diff --git a/docs/source/en/training/t2i_adapters.md b/docs/source/en/training/t2i_adapters.md index 6d760040731d..19c2059aa687 100644 --- a/docs/source/en/training/t2i_adapters.md +++ b/docs/source/en/training/t2i_adapters.md @@ -192,9 +192,9 @@ from diffusers import StableDiffusionXLAdapterPipeline, T2IAdapter, EulerAncestr from diffusers.utils import load_image import torch -adapter = T2IAdapter.from_pretrained("path/to/adapter", torch_dtype=torch.float16) +adapter = T2IAdapter.from_pretrained("path/to/adapter", dtype=torch.float16) pipeline = StableDiffusionXLAdapterPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", adapter=adapter, torch_dtype=torch.float16 + "stabilityai/stable-diffusion-xl-base-1.0", adapter=adapter, dtype=torch.float16 ) pipeline.scheduler = EulerAncestralDiscreteSchedulerTest.from_config(pipe.scheduler.config) diff --git a/docs/source/en/training/text2image.md b/docs/source/en/training/text2image.md index d11e55e91018..688747ee714e 100644 --- a/docs/source/en/training/text2image.md +++ b/docs/source/en/training/text2image.md @@ -168,7 +168,7 @@ Once training is complete, you can use your newly trained model for inference: from diffusers import StableDiffusionPipeline import torch -pipeline = StableDiffusionPipeline.from_pretrained("path/to/saved_model", torch_dtype=torch.float16, use_safetensors=True).to("cuda") +pipeline = StableDiffusionPipeline.from_pretrained("path/to/saved_model", dtype=torch.float16, use_safetensors=True).to("cuda") image = pipeline(prompt="yoda").images[0] image.save("yoda-naruto.png") diff --git a/docs/source/en/training/text_inversion.md b/docs/source/en/training/text_inversion.md index 4912b6730a61..d06f0d35faf2 100644 --- a/docs/source/en/training/text_inversion.md +++ b/docs/source/en/training/text_inversion.md @@ -193,7 +193,7 @@ After training is complete, you can use your newly trained model for inference l from diffusers import StableDiffusionPipeline import torch -pipeline = StableDiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16).to("cuda") +pipeline = StableDiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16).to("cuda") pipeline.load_textual_inversion("sd-concepts-library/cat-toy") image = pipeline("A train", num_inference_steps=50).images[0] image.save("cat-train.png") diff --git a/docs/source/en/training/wuerstchen.md b/docs/source/en/training/wuerstchen.md index c8418df1989e..9e1785b8ea72 100644 --- a/docs/source/en/training/wuerstchen.md +++ b/docs/source/en/training/wuerstchen.md @@ -160,7 +160,7 @@ import torch from diffusers import AutoPipelineForText2Image from diffusers.pipelines.wuerstchen import DEFAULT_STAGE_C_TIMESTEPS -pipeline = AutoPipelineForText2Image.from_pretrained("path/to/saved/model", torch_dtype=torch.float16).to("cuda") +pipeline = AutoPipelineForText2Image.from_pretrained("path/to/saved/model", dtype=torch.float16).to("cuda") caption = "A cute bird naruto holding a shield" images = pipeline( diff --git a/docs/source/en/tutorials/autopipeline.md b/docs/source/en/tutorials/autopipeline.md index 0c5e57ca07de..22d9e0ffe011 100644 --- a/docs/source/en/tutorials/autopipeline.md +++ b/docs/source/en/tutorials/autopipeline.md @@ -23,7 +23,7 @@ import torch from diffusers import AutoPipelineForImage2Image pipeline = AutoPipelineForImage2Image.from_pretrained( - "RunDiffusion/Juggernaut-XL-v9", torch_dtype=torch.bfloat16, device_map="cuda", + "RunDiffusion/Juggernaut-XL-v9", dtype=torch.bfloat16, device_map="cuda", ) print(pipeline) "StableDiffusionXLImg2ImgPipeline { @@ -39,7 +39,7 @@ import torch from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( - "RunDiffusion/Juggernaut-XL-v9", torch_dtype=torch.bfloat16, device_map="cuda", + "RunDiffusion/Juggernaut-XL-v9", dtype=torch.bfloat16, device_map="cuda", ) print(pipeline) "StableDiffusionXLPipeline { @@ -57,7 +57,7 @@ import torch from diffusers import AutoPipelineForImage2Image pipeline = AutoPipelineForImage2Image.from_pretrained( - "openai/shap-e-img2img", torch_dtype=torch.float16, + "openai/shap-e-img2img", dtype=torch.float16, ) "ValueError: AutoPipeline can't find a pipeline linked to ShapEImg2ImgPipeline for None" ``` diff --git a/docs/source/en/tutorials/using_peft_for_inference.md b/docs/source/en/tutorials/using_peft_for_inference.md index 7bdd2a1ee969..39df8fc06872 100644 --- a/docs/source/en/tutorials/using_peft_for_inference.md +++ b/docs/source/en/tutorials/using_peft_for_inference.md @@ -25,7 +25,7 @@ from diffusers import AutoPipelineForText2Image pipeline = AutoPipelineForText2Image.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.load_lora_weights( "ostris/super-cereal-sdxl-lora", @@ -44,7 +44,7 @@ from diffusers import LTXConditionPipeline from diffusers.utils import export_to_video, load_image pipeline = LTXConditionPipeline.from_pretrained( - "Lightricks/LTX-Video-0.9.5", torch_dtype=torch.bfloat16 + "Lightricks/LTX-Video-0.9.5", dtype=torch.bfloat16 ) pipeline.load_lora_weights( @@ -89,7 +89,7 @@ from diffusers import AutoPipelineForText2Image pipeline = AutoPipelineForText2Image.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.unet.load_lora_adapter( "jbilcke-hf/sdxl-cinematic-1", @@ -112,7 +112,7 @@ from diffusers import DiffusionPipeline # load base model and LoRA pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.load_lora_weights( "ostris/ikea-instructions-lora-sdxl", @@ -154,7 +154,7 @@ from diffusers import AutoPipelineForText2Image pipeline = AutoPipelineForText2Image.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.load_lora_weights( "ostris/super-cereal-sdxl-lora", @@ -178,7 +178,7 @@ from diffusers import AutoPipelineForText2Image pipeline = AutoPipelineForText2Image.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.load_lora_weights( "ostris/super-cereal-sdxl-lora", @@ -214,7 +214,7 @@ import torch from diffusers import FluxPipeline pipeline = FluxPipeline.from_pretrained( - "black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16 + "black-forest-labs/FLUX.1-dev", dtype=torch.bfloat16 ).to("cuda") pipelne.load_lora_weights("alvarobartt/ghibli-characters-flux-lora", "lora") @@ -254,7 +254,7 @@ from diffusers import DiffusionPipeline # load base model and LoRAs pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.load_lora_weights( "ostris/ikea-instructions-lora-sdxl", @@ -292,7 +292,7 @@ from diffusers import DiffusionPipeline # load base model and LoRAs pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") # 1. enable_lora_hotswap pipeline.enable_lora_hotswap(target_rank=max_rank) @@ -349,7 +349,7 @@ from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.load_lora_weights( "ostris/ikea-instructions-lora-sdxl", @@ -393,7 +393,7 @@ from peft import get_peft_model, LoraConfig, PeftModel unet = AutoModel.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16, + dtype=torch.float16, use_safetensors=True, variant="fp16", subfolder="unet", @@ -406,7 +406,7 @@ Load a pipeline, pass the UNet to it, and load a LoRA. pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", variant="fp16", - torch_dtype=torch.float16, + dtype=torch.float16, unet=unet ).to("cuda") pipeline.load_lora_weights( @@ -464,7 +464,7 @@ Load a base UNet model and load the adapters. ```py base_unet = AutoModel.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16, + dtype=torch.float16, use_safetensors=True, variant="fp16", subfolder="unet", @@ -502,7 +502,7 @@ pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", unet=model, variant="fp16", - torch_dtype=torch.float16, + dtype=torch.float16, ).to("cuda") pipeline("A bowl of ramen shaped like a cute kawaii bear, by Feng Zikai").images[0] ``` @@ -521,7 +521,7 @@ from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.load_lora_weights( "ostris/ikea-instructions-lora-sdxl", @@ -567,7 +567,7 @@ The fused pipeline can now be quickly loaded for inference without requiring eac ```py pipeline = DiffusionPipeline.from_pretrained( - "username/fused-ikea-feng", torch_dtype=torch.float16, + "username/fused-ikea-feng", dtype=torch.float16, ).to("cuda") pipeline("A bowl of ramen shaped like a cute kawaii bear, by Feng Zikai").images[0] ``` @@ -596,7 +596,7 @@ from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.load_lora_weights( "ostris/ikea-instructions-lora-sdxl", @@ -622,7 +622,7 @@ from diffusers import AutoPipelineForText2Image pipeline = AutoPipelineForText2Image.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.unet.load_lora_adapter( "jbilcke-hf/sdxl-cinematic-1", diff --git a/docs/source/en/using-diffusers/automodel.md b/docs/source/en/using-diffusers/automodel.md index 82d4d14a10a9..ddae567d4153 100644 --- a/docs/source/en/using-diffusers/automodel.md +++ b/docs/source/en/using-diffusers/automodel.md @@ -21,11 +21,11 @@ import torch from diffusers import AutoModel, DiffusionPipeline transformer = AutoModel.from_pretrained( - "Qwen/Qwen-Image", subfolder="transformer", torch_dtype=torch.bfloat16, device_map="cuda" + "Qwen/Qwen-Image", subfolder="transformer", dtype=torch.bfloat16, device_map="cuda" ) text_encoder = AutoModel.from_pretrained( - "Qwen/Qwen-Image", subfolder="text_encoder", torch_dtype=torch.bfloat16, device_map="cuda" + "Qwen/Qwen-Image", subfolder="text_encoder", dtype=torch.bfloat16, device_map="cuda" ) ``` @@ -59,7 +59,7 @@ import torch from diffusers import AutoModel transformer = AutoModel.from_pretrained( - "custom/custom-transformer-model", trust_remote_code=True, torch_dtype=torch.bfloat16, device_map="cuda" + "custom/custom-transformer-model", trust_remote_code=True, dtype=torch.bfloat16, device_map="cuda" ) ``` @@ -82,7 +82,7 @@ import torch from diffusers import AutoModel transformer = AutoModel.from_pretrained( - "Overworld/Waypoint-1-Small", subfolder="transformer", trust_remote_code=True, torch_dtype=torch.bfloat16, device_map="cuda" + "Overworld/Waypoint-1-Small", subfolder="transformer", trust_remote_code=True, dtype=torch.bfloat16, device_map="cuda" ) ``` diff --git a/docs/source/en/using-diffusers/batched_inference.md b/docs/source/en/using-diffusers/batched_inference.md index cdb16ac1212b..a6be304a421e 100644 --- a/docs/source/en/using-diffusers/batched_inference.md +++ b/docs/source/en/using-diffusers/batched_inference.md @@ -25,7 +25,7 @@ from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16, + dtype=torch.float16, device_map="cuda" ) @@ -65,7 +65,7 @@ from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16, + dtype=torch.float16, device_map="cuda" ) @@ -141,7 +141,7 @@ from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16, + dtype=torch.float16, device_map="cuda" ) diff --git a/docs/source/en/using-diffusers/callback.md b/docs/source/en/using-diffusers/callback.md index 60b839805ff2..3847b376cc49 100644 --- a/docs/source/en/using-diffusers/callback.md +++ b/docs/source/en/using-diffusers/callback.md @@ -36,7 +36,7 @@ callback = SDXLCFGCutoffCallback(cutoff_step_ratio=0.4) pipeline = StableDiffusionXLPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16, + dtype=torch.float16, device_map="cuda" ) pipeline.scheduler = DPMSolverMultistepScheduler.from_config(pipeline.scheduler.config, use_karras_sigmas=True) @@ -123,7 +123,7 @@ from diffusers import AutoPipelineForText2Image pipeline = AutoPipelineForText2Image.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16, + dtype=torch.float16, device_map="cuda" ) diff --git a/docs/source/en/using-diffusers/conditional_image_generation.md b/docs/source/en/using-diffusers/conditional_image_generation.md index 72cc3397a30c..adf595bffb99 100644 --- a/docs/source/en/using-diffusers/conditional_image_generation.md +++ b/docs/source/en/using-diffusers/conditional_image_generation.md @@ -30,7 +30,7 @@ from diffusers import AutoPipelineForText2Image import torch pipeline = AutoPipelineForText2Image.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16, variant="fp16" + "stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16, variant="fp16" ).to("cuda") ``` @@ -60,7 +60,7 @@ from diffusers import AutoPipelineForText2Image import torch pipeline = AutoPipelineForText2Image.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16, variant="fp16" + "stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16, variant="fp16" ).to("cuda") generator = torch.Generator("cuda").manual_seed(31) image = pipeline("Astronaut in a jungle, cold color palette, muted colors, detailed, 8k", generator=generator).images[0] @@ -76,7 +76,7 @@ from diffusers import AutoPipelineForText2Image import torch pipeline = AutoPipelineForText2Image.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16, variant="fp16" + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16, variant="fp16" ).to("cuda") generator = torch.Generator("cuda").manual_seed(31) image = pipeline("Astronaut in a jungle, cold color palette, muted colors, detailed, 8k", generator=generator).images[0] @@ -94,7 +94,7 @@ from diffusers import AutoPipelineForText2Image import torch pipeline = AutoPipelineForText2Image.from_pretrained( - "kandinsky-community/kandinsky-2-2-decoder", torch_dtype=torch.float16 + "kandinsky-community/kandinsky-2-2-decoder", dtype=torch.float16 ).to("cuda") generator = torch.Generator("cuda").manual_seed(31) image = pipeline("Astronaut in a jungle, cold color palette, muted colors, detailed, 8k", generator=generator).images[0] @@ -113,7 +113,7 @@ from diffusers.utils import load_image import torch controlnet = ControlNetModel.from_pretrained( - "lllyasviel/control_v11p_sd15_openpose", torch_dtype=torch.float16, variant="fp16" + "lllyasviel/control_v11p_sd15_openpose", dtype=torch.float16, variant="fp16" ).to("cuda") pose_image = load_image("https://huggingface.co/lllyasviel/control_v11p_sd15_openpose/resolve/main/images/control.png") ``` @@ -122,7 +122,7 @@ Pass the `controlnet` to the [`AutoPipelineForText2Image`], and provide the prom ```py pipeline = AutoPipelineForText2Image.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-v1-5", controlnet=controlnet, torch_dtype=torch.float16, variant="fp16" + "stable-diffusion-v1-5/stable-diffusion-v1-5", controlnet=controlnet, dtype=torch.float16, variant="fp16" ).to("cuda") generator = torch.Generator("cuda").manual_seed(31) image = pipeline("Astronaut in a jungle, cold color palette, muted colors, detailed, 8k", image=pose_image, generator=generator).images[0] @@ -161,7 +161,7 @@ from diffusers import AutoPipelineForText2Image import torch pipeline = AutoPipelineForText2Image.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16, variant="fp16" + "stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16, variant="fp16" ).to("cuda") image = pipeline( "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k", height=768, width=512 @@ -185,7 +185,7 @@ from diffusers import AutoPipelineForText2Image import torch pipeline = AutoPipelineForText2Image.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16 + "stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16 ).to("cuda") image = pipeline( "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k", guidance_scale=3.5 @@ -217,7 +217,7 @@ from diffusers import AutoPipelineForText2Image import torch pipeline = AutoPipelineForText2Image.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16 + "stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16 ).to("cuda") image = pipeline( prompt="Astronaut in a jungle, cold color palette, muted colors, detailed, 8k", @@ -248,7 +248,7 @@ from diffusers import AutoPipelineForText2Image import torch pipeline = AutoPipelineForText2Image.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16 + "stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16 ).to("cuda") generator = torch.Generator(device="cuda").manual_seed(30) image = pipeline( @@ -276,7 +276,7 @@ from diffusers import AutoPipelineForText2Image import torch pipeline = AutoPipelineForText2Image.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16 + "stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16 ).to("cuda") image = pipeline( prompt_embeds=prompt_embeds, # generated from Compel @@ -300,7 +300,7 @@ PyTorch 2.0 also supports a more memory-efficient attention mechanism called [*s from diffusers import AutoPipelineForText2Image import torch -pipeline = AutoPipelineForText2Image.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16, variant="fp16").to("cuda") +pipeline = AutoPipelineForText2Image.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16, variant="fp16").to("cuda") pipeline.unet = torch.compile(pipeline.unet, mode="reduce-overhead", fullgraph=True) ``` diff --git a/docs/source/en/using-diffusers/controlnet.md b/docs/source/en/using-diffusers/controlnet.md index 4aa5abd04f3b..4bf068def57e 100644 --- a/docs/source/en/using-diffusers/controlnet.md +++ b/docs/source/en/using-diffusers/controlnet.md @@ -55,10 +55,10 @@ from diffusers.utils import load_image from diffusers import FluxControlNetPipeline, FluxControlNetModel controlnet = FluxControlNetModel.from_pretrained( - "InstantX/FLUX.1-dev-Controlnet-Canny", torch_dtype=torch.bfloat16 + "InstantX/FLUX.1-dev-Controlnet-Canny", dtype=torch.bfloat16 ) pipeline = FluxControlNetPipeline.from_pretrained( - "black-forest-labs/FLUX.1-dev", controlnet=controlnet, torch_dtype=torch.bfloat16 + "black-forest-labs/FLUX.1-dev", controlnet=controlnet, dtype=torch.bfloat16 ).to("cuda") prompt = """ @@ -135,14 +135,14 @@ Pass the depth map to the pipeline. Use the `controlnet_conditioning_scale` para ```py controlnet = ControlNetModel.from_pretrained( "diffusers/controlnet-depth-sdxl-1.0-small", - torch_dtype=torch.float16, + dtype=torch.float16, ) -vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16) +vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", dtype=torch.float16) pipeline = StableDiffusionXLControlNetImg2ImgPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", controlnet=controlnet, vae=vae, - torch_dtype=torch.float16, + dtype=torch.float16, ).to("cuda") prompt = """ @@ -215,10 +215,10 @@ Pass the mask and control image to the pipeline. Use the `controlnet_conditionin ```py controlnet = ControlNetModel.from_pretrained( - "diffusers/controlnet-canny-sdxl-1.0", torch_dtype=torch.float16 + "diffusers/controlnet-canny-sdxl-1.0", dtype=torch.float16 ) pipeline = StableDiffusionXLControlNetInpaintPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", controlnet=controlnet, torch_dtype=torch.float16 + "stabilityai/stable-diffusion-xl-base-1.0", controlnet=controlnet, dtype=torch.float16 ) pipeline( "a cute and fluffy bunny rabbit", @@ -263,16 +263,16 @@ from diffusers import StableDiffusionXLControlNetPipeline, ControlNetModel, Auto controlnets = [ ControlNetModel.from_pretrained( - "diffusers/controlnet-depth-sdxl-1.0-small", torch_dtype=torch.float16 + "diffusers/controlnet-depth-sdxl-1.0-small", dtype=torch.float16 ), ControlNetModel.from_pretrained( - "diffusers/controlnet-canny-sdxl-1.0", torch_dtype=torch.float16, + "diffusers/controlnet-canny-sdxl-1.0", dtype=torch.float16, ), ] -vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16) +vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", dtype=torch.float16) pipeline = StableDiffusionXLControlNetPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", controlnet=controlnets, vae=vae, torch_dtype=torch.float16 + "stabilityai/stable-diffusion-xl-base-1.0", controlnet=controlnets, vae=vae, dtype=torch.float16 ).to("cuda") prompt = """ @@ -318,12 +318,12 @@ from diffusers.utils import load_iamge from diffusers import StableDiffusionXLControlNetPipeline, ControlNetModel controlnet = ControlNetModel.from_pretrained( - "diffusers/controlnet-canny-sdxl-1.0", torch_dtype=torch.float16 + "diffusers/controlnet-canny-sdxl-1.0", dtype=torch.float16 ) pipeline = StableDiffusionXLControlNetPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", controlnet=controlnet, - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") canny_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/canny-cat.png") diff --git a/docs/source/en/using-diffusers/custom_pipeline_overview.md b/docs/source/en/using-diffusers/custom_pipeline_overview.md index b087e57056dd..6687667acac9 100644 --- a/docs/source/en/using-diffusers/custom_pipeline_overview.md +++ b/docs/source/en/using-diffusers/custom_pipeline_overview.md @@ -38,7 +38,7 @@ from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-3-medium-diffusers", custom_pipeline="pipeline_stable_diffusion_3_instruct_pix2pix", - torch_dtype=torch.float16, + dtype=torch.float16, device_map="cuda" ) ``` @@ -53,7 +53,7 @@ pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-3-medium-diffusers", custom_pipeline="pipeline_stable_diffusion_3_instruct_pix2pix", custom_revision="main" - torch_dtype=torch.float16, + dtype=torch.float16, device_map="cuda" ) ``` @@ -72,7 +72,7 @@ There are a few ways to load a community pipeline. pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-3-medium-diffusers", custom_pipeline="path/to/pipeline_directory", - torch_dtype=torch.float16, + dtype=torch.float16, device_map="cuda" ) ``` @@ -83,7 +83,7 @@ There are a few ways to load a community pipeline. import torch from diffusers import DiffusionPipeline - pipeline_sd = DiffusionPipeline.from_pretrained("emilianJR/CyberRealistic_V3", torch_dtype=torch.float16, device_map="cuda") + pipeline_sd = DiffusionPipeline.from_pretrained("emilianJR/CyberRealistic_V3", dtype=torch.float16, device_map="cuda") pipeline_lpw = DiffusionPipeline.from_pipe( pipeline_sd, custom_pipeline="lpw_stable_diffusion", device_map="cuda" ) @@ -136,7 +136,7 @@ pipeline = TextToVideoIFPipeline( scheduler=scheduler, feature_extractor=feature_extractor, device_map="cuda", - torch_dtype=torch.float16 + dtype=torch.float16 ) ``` @@ -159,7 +159,7 @@ import torch from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( - "/", trust_remote_code=True, torch_dtype=torch.float16 + "/", trust_remote_code=True, dtype=torch.float16 ) ``` diff --git a/docs/source/en/using-diffusers/depth2img.md b/docs/source/en/using-diffusers/depth2img.md index 7e04b04520fa..ae7bbf162ac9 100644 --- a/docs/source/en/using-diffusers/depth2img.md +++ b/docs/source/en/using-diffusers/depth2img.md @@ -25,7 +25,7 @@ from diffusers.utils import load_image, make_image_grid pipeline = StableDiffusionDepth2ImgPipeline.from_pretrained( "stabilityai/stable-diffusion-2-depth", - torch_dtype=torch.float16, + dtype=torch.float16, use_safetensors=True, ).to("cuda") ``` diff --git a/docs/source/en/using-diffusers/dreambooth.md b/docs/source/en/using-diffusers/dreambooth.md index 6c37124cb7ff..14c4550d9c76 100644 --- a/docs/source/en/using-diffusers/dreambooth.md +++ b/docs/source/en/using-diffusers/dreambooth.md @@ -24,7 +24,7 @@ from diffusers import AutoPipelineForText2Image pipeline = AutoPipelineForText2Image.from_pretrained( "sd-dreambooth-library/herge-style", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") prompt = "A cute sks herge_style brown bear eating a slice of pizza, stunning color scheme, masterpiece, illustration" pipeline(prompt).images[0] diff --git a/docs/source/en/using-diffusers/image_quality.md b/docs/source/en/using-diffusers/image_quality.md index 29ce483d5ecc..65b51d858f4d 100644 --- a/docs/source/en/using-diffusers/image_quality.md +++ b/docs/source/en/using-diffusers/image_quality.md @@ -24,7 +24,7 @@ import torch from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16, safety_checker=None + "stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16, safety_checker=None ).to("cuda") pipeline.enable_freeu(s1=0.9, s2=0.2, b1=1.5, b2=1.6) generator = torch.Generator(device="cpu").manual_seed(33) @@ -52,7 +52,7 @@ import torch from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( - "stabilityai/stable-diffusion-2-1", torch_dtype=torch.float16, safety_checker=None + "stabilityai/stable-diffusion-2-1", dtype=torch.float16, safety_checker=None ).to("cuda") pipeline.enable_freeu(s1=0.9, s2=0.2, b1=1.4, b2=1.6) generator = torch.Generator(device="cpu").manual_seed(80) @@ -80,7 +80,7 @@ import torch from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16, + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16, ).to("cuda") pipeline.enable_freeu(s1=0.9, s2=0.2, b1=1.3, b2=1.4) generator = torch.Generator(device="cpu").manual_seed(13) @@ -109,7 +109,7 @@ from diffusers import DiffusionPipeline from diffusers.utils import export_to_video pipeline = DiffusionPipeline.from_pretrained( - "damo-vilab/text-to-video-ms-1.7b", torch_dtype=torch.float16 + "damo-vilab/text-to-video-ms-1.7b", dtype=torch.float16 ).to("cuda") # values come from https://github.com/lyn-rgb/FreeU_Diffusers#video-pipelines pipeline.enable_freeu(b1=1.2, b2=1.4, s1=0.9, s2=0.2) diff --git a/docs/source/en/using-diffusers/img2img.md b/docs/source/en/using-diffusers/img2img.md index 64f9212dddcb..366be176bd86 100644 --- a/docs/source/en/using-diffusers/img2img.md +++ b/docs/source/en/using-diffusers/img2img.md @@ -26,7 +26,7 @@ from diffusers import AutoPipelineForImage2Image from diffusers.utils import load_image, make_image_grid pipeline = AutoPipelineForImage2Image.from_pretrained( - "kandinsky-community/kandinsky-2-2-decoder", torch_dtype=torch.float16, use_safetensors=True + "kandinsky-community/kandinsky-2-2-decoder", dtype=torch.float16, use_safetensors=True ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed @@ -75,7 +75,7 @@ from diffusers import AutoPipelineForImage2Image from diffusers.utils import make_image_grid, load_image pipeline = AutoPipelineForImage2Image.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16, variant="fp16", use_safetensors=True + "stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16, variant="fp16", use_safetensors=True ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed @@ -113,7 +113,7 @@ from diffusers import AutoPipelineForImage2Image from diffusers.utils import make_image_grid, load_image pipeline = AutoPipelineForImage2Image.from_pretrained( - "stabilityai/stable-diffusion-xl-refiner-1.0", torch_dtype=torch.float16, variant="fp16", use_safetensors=True + "stabilityai/stable-diffusion-xl-refiner-1.0", dtype=torch.float16, variant="fp16", use_safetensors=True ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed @@ -153,7 +153,7 @@ from diffusers import AutoPipelineForImage2Image from diffusers.utils import make_image_grid, load_image pipeline = AutoPipelineForImage2Image.from_pretrained( - "kandinsky-community/kandinsky-2-2-decoder", torch_dtype=torch.float16, use_safetensors=True + "kandinsky-community/kandinsky-2-2-decoder", dtype=torch.float16, use_safetensors=True ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed @@ -200,7 +200,7 @@ from diffusers import AutoPipelineForImage2Image from diffusers.utils import make_image_grid, load_image pipeline = AutoPipelineForImage2Image.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16, variant="fp16", use_safetensors=True + "stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16, variant="fp16", use_safetensors=True ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed @@ -244,7 +244,7 @@ from diffusers import AutoPipelineForImage2Image from diffusers.utils import make_image_grid, load_image pipeline = AutoPipelineForImage2Image.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16, variant="fp16", use_safetensors=True + "stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16, variant="fp16", use_safetensors=True ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed @@ -286,7 +286,7 @@ from diffusers import AutoPipelineForImage2Image from diffusers.utils import make_image_grid, load_image pipeline = AutoPipelineForImage2Image.from_pretrained( - "stabilityai/stable-diffusion-xl-refiner-1.0", torch_dtype=torch.float16, variant="fp16", use_safetensors=True + "stabilityai/stable-diffusion-xl-refiner-1.0", dtype=torch.float16, variant="fp16", use_safetensors=True ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed @@ -331,7 +331,7 @@ import torch from diffusers.utils import make_image_grid pipeline = AutoPipelineForText2Image.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16, variant="fp16", use_safetensors=True + "stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16, variant="fp16", use_safetensors=True ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed @@ -345,7 +345,7 @@ Now you can pass this generated image to the image-to-image pipeline: ```py pipeline = AutoPipelineForImage2Image.from_pretrained( - "kandinsky-community/kandinsky-2-2-decoder", torch_dtype=torch.float16, use_safetensors=True + "kandinsky-community/kandinsky-2-2-decoder", dtype=torch.float16, use_safetensors=True ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed @@ -367,7 +367,7 @@ from diffusers import AutoPipelineForImage2Image from diffusers.utils import make_image_grid, load_image pipeline = AutoPipelineForImage2Image.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16, variant="fp16", use_safetensors=True + "stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16, variant="fp16", use_safetensors=True ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed @@ -390,7 +390,7 @@ Pass the latent output from this pipeline to the next pipeline to generate an im ```py pipeline = AutoPipelineForImage2Image.from_pretrained( - "ogkalu/Comic-Diffusion", torch_dtype=torch.float16 + "ogkalu/Comic-Diffusion", dtype=torch.float16 ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed @@ -404,7 +404,7 @@ Repeat one more time to generate the final image in a [pixel art style](https:// ```py pipeline = AutoPipelineForImage2Image.from_pretrained( - "kohbanye/pixel-art-style", torch_dtype=torch.float16 + "kohbanye/pixel-art-style", dtype=torch.float16 ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed @@ -427,7 +427,7 @@ from diffusers import AutoPipelineForImage2Image from diffusers.utils import make_image_grid, load_image pipeline = AutoPipelineForImage2Image.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16, variant="fp16", use_safetensors=True + "stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16, variant="fp16", use_safetensors=True ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed @@ -452,7 +452,7 @@ Chain it to an upscaler pipeline to increase the image resolution: from diffusers import StableDiffusionLatentUpscalePipeline upscaler = StableDiffusionLatentUpscalePipeline.from_pretrained( - "stabilityai/sd-x2-latent-upscaler", torch_dtype=torch.float16, use_safetensors=True + "stabilityai/sd-x2-latent-upscaler", dtype=torch.float16, use_safetensors=True ) upscaler.enable_model_cpu_offload() upscaler.enable_xformers_memory_efficient_attention() @@ -466,7 +466,7 @@ Finally, chain it to a super-resolution pipeline to further enhance the resoluti from diffusers import StableDiffusionUpscalePipeline super_res = StableDiffusionUpscalePipeline.from_pretrained( - "stabilityai/stable-diffusion-x4-upscaler", torch_dtype=torch.float16, variant="fp16", use_safetensors=True + "stabilityai/stable-diffusion-x4-upscaler", dtype=torch.float16, variant="fp16", use_safetensors=True ) super_res.enable_model_cpu_offload() super_res.enable_xformers_memory_efficient_attention() @@ -490,7 +490,7 @@ from diffusers import AutoPipelineForImage2Image import torch pipeline = AutoPipelineForImage2Image.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16, variant="fp16", use_safetensors=True + "stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16, variant="fp16", use_safetensors=True ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed @@ -525,9 +525,9 @@ Load a ControlNet model conditioned on depth maps and the [`AutoPipelineForImage from diffusers import ControlNetModel, AutoPipelineForImage2Image import torch -controlnet = ControlNetModel.from_pretrained("lllyasviel/control_v11f1p_sd15_depth", torch_dtype=torch.float16, variant="fp16", use_safetensors=True) +controlnet = ControlNetModel.from_pretrained("lllyasviel/control_v11f1p_sd15_depth", dtype=torch.float16, variant="fp16", use_safetensors=True) pipeline = AutoPipelineForImage2Image.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-v1-5", controlnet=controlnet, torch_dtype=torch.float16, variant="fp16", use_safetensors=True + "stable-diffusion-v1-5/stable-diffusion-v1-5", controlnet=controlnet, dtype=torch.float16, variant="fp16", use_safetensors=True ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed @@ -561,7 +561,7 @@ Let's apply a new [style](https://huggingface.co/nitrosocke/elden-ring-diffusion ```py pipeline = AutoPipelineForImage2Image.from_pretrained( - "nitrosocke/elden-ring-diffusion", torch_dtype=torch.float16, + "nitrosocke/elden-ring-diffusion", dtype=torch.float16, ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed diff --git a/docs/source/en/using-diffusers/inpaint.md b/docs/source/en/using-diffusers/inpaint.md index d6b6f6f3b08d..02b257e09cd6 100644 --- a/docs/source/en/using-diffusers/inpaint.md +++ b/docs/source/en/using-diffusers/inpaint.md @@ -26,7 +26,7 @@ from diffusers import AutoPipelineForInpainting from diffusers.utils import load_image, make_image_grid pipeline = AutoPipelineForInpainting.from_pretrained( - "kandinsky-community/kandinsky-2-2-decoder-inpaint", torch_dtype=torch.float16 + "kandinsky-community/kandinsky-2-2-decoder-inpaint", dtype=torch.float16 ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed @@ -92,7 +92,7 @@ from diffusers import AutoPipelineForInpainting from diffusers.utils import load_image from PIL import Image -pipeline = AutoPipelineForInpainting.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16).to('cuda') +pipeline = AutoPipelineForInpainting.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16).to('cuda') mask = load_image("https://huggingface.co/datasets/YiYiXu/testing-images/resolve/main/seashore_mask.png") blurred_mask = pipeline.mask_processor.blur(mask, blur_factor=33) @@ -124,7 +124,7 @@ from diffusers import AutoPipelineForInpainting from diffusers.utils import load_image, make_image_grid pipeline = AutoPipelineForInpainting.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-inpainting", torch_dtype=torch.float16, variant="fp16" + "stable-diffusion-v1-5/stable-diffusion-inpainting", dtype=torch.float16, variant="fp16" ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed @@ -150,7 +150,7 @@ from diffusers import AutoPipelineForInpainting from diffusers.utils import load_image, make_image_grid pipeline = AutoPipelineForInpainting.from_pretrained( - "diffusers/stable-diffusion-xl-1.0-inpainting-0.1", torch_dtype=torch.float16, variant="fp16" + "diffusers/stable-diffusion-xl-1.0-inpainting-0.1", dtype=torch.float16, variant="fp16" ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed @@ -176,7 +176,7 @@ from diffusers import AutoPipelineForInpainting from diffusers.utils import load_image, make_image_grid pipeline = AutoPipelineForInpainting.from_pretrained( - "kandinsky-community/kandinsky-2-2-decoder-inpaint", torch_dtype=torch.float16 + "kandinsky-community/kandinsky-2-2-decoder-inpaint", dtype=torch.float16 ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed @@ -227,7 +227,7 @@ from diffusers import AutoPipelineForInpainting from diffusers.utils import load_image, make_image_grid pipeline = AutoPipelineForInpainting.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16, variant="fp16" + "stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16, variant="fp16" ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed @@ -252,7 +252,7 @@ from diffusers import AutoPipelineForInpainting from diffusers.utils import load_image, make_image_grid pipeline = AutoPipelineForInpainting.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-inpainting", torch_dtype=torch.float16, variant="fp16" + "stable-diffusion-v1-5/stable-diffusion-inpainting", dtype=torch.float16, variant="fp16" ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed @@ -293,7 +293,7 @@ from diffusers import AutoPipelineForInpainting from diffusers.utils import load_image, make_image_grid pipeline = AutoPipelineForInpainting.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16, variant="fp16" + "stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16, variant="fp16" ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed @@ -316,7 +316,7 @@ from diffusers import AutoPipelineForInpainting from diffusers.utils import load_image, make_image_grid pipeline = AutoPipelineForInpainting.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-inpainting", torch_dtype=torch.float16, variant="fp16" + "stable-diffusion-v1-5/stable-diffusion-inpainting", dtype=torch.float16, variant="fp16" ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed @@ -359,7 +359,7 @@ from diffusers.utils import load_image, make_image_grid device = "cuda" pipeline = AutoPipelineForInpainting.from_pretrained( "stable-diffusion-v1-5/stable-diffusion-inpainting", - torch_dtype=torch.float16, + dtype=torch.float16, variant="fp16" ) pipeline = pipeline.to(device) @@ -396,7 +396,7 @@ from diffusers import AutoPipelineForInpainting from diffusers.utils import load_image, make_image_grid pipeline = AutoPipelineForInpainting.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-inpainting", torch_dtype=torch.float16, variant="fp16" + "stable-diffusion-v1-5/stable-diffusion-inpainting", dtype=torch.float16, variant="fp16" ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed @@ -441,7 +441,7 @@ from diffusers import AutoPipelineForInpainting from diffusers.utils import load_image, make_image_grid pipeline = AutoPipelineForInpainting.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-inpainting", torch_dtype=torch.float16, variant="fp16" + "stable-diffusion-v1-5/stable-diffusion-inpainting", dtype=torch.float16, variant="fp16" ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed @@ -481,7 +481,7 @@ from diffusers import AutoPipelineForInpainting from diffusers.utils import load_image, make_image_grid pipeline = AutoPipelineForInpainting.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-inpainting", torch_dtype=torch.float16, variant="fp16" + "stable-diffusion-v1-5/stable-diffusion-inpainting", dtype=torch.float16, variant="fp16" ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed @@ -517,7 +517,7 @@ from diffusers.utils import load_image from PIL import Image generator = torch.Generator(device='cuda').manual_seed(0) -pipeline = AutoPipelineForInpainting.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16).to('cuda') +pipeline = AutoPipelineForInpainting.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16).to('cuda') base = load_image("https://huggingface.co/datasets/YiYiXu/testing-images/resolve/main/seashore.png") mask = load_image("https://huggingface.co/datasets/YiYiXu/testing-images/resolve/main/seashore_mask.png") @@ -553,7 +553,7 @@ from diffusers import AutoPipelineForText2Image, AutoPipelineForInpainting from diffusers.utils import load_image, make_image_grid pipeline = AutoPipelineForText2Image.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16, variant="fp16", use_safetensors=True + "stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16, variant="fp16", use_safetensors=True ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed @@ -572,7 +572,7 @@ And let's inpaint the masked area with a waterfall: ```py pipeline = AutoPipelineForInpainting.from_pretrained( - "kandinsky-community/kandinsky-2-2-decoder-inpaint", torch_dtype=torch.float16 + "kandinsky-community/kandinsky-2-2-decoder-inpaint", dtype=torch.float16 ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed @@ -606,7 +606,7 @@ from diffusers import AutoPipelineForInpainting, AutoPipelineForImage2Image from diffusers.utils import load_image, make_image_grid pipeline = AutoPipelineForInpainting.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-inpainting", torch_dtype=torch.float16, variant="fp16" + "stable-diffusion-v1-5/stable-diffusion-inpainting", dtype=torch.float16, variant="fp16" ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed @@ -627,7 +627,7 @@ Now let's pass the image to another inpainting pipeline with SDXL's refiner mode ```py pipeline = AutoPipelineForInpainting.from_pretrained( - "stabilityai/stable-diffusion-xl-refiner-1.0", torch_dtype=torch.float16, variant="fp16" + "stabilityai/stable-diffusion-xl-refiner-1.0", dtype=torch.float16, variant="fp16" ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed @@ -683,7 +683,7 @@ from diffusers import AutoPipelineForInpainting from diffusers.utils import make_image_grid pipeline = AutoPipelineForInpainting.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-inpainting", torch_dtype=torch.float16, + "stable-diffusion-v1-5/stable-diffusion-inpainting", dtype=torch.float16, ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed @@ -710,11 +710,11 @@ from diffusers import ControlNetModel, StableDiffusionControlNetInpaintPipeline from diffusers.utils import load_image, make_image_grid # load ControlNet -controlnet = ControlNetModel.from_pretrained("lllyasviel/control_v11p_sd15_inpaint", torch_dtype=torch.float16, variant="fp16") +controlnet = ControlNetModel.from_pretrained("lllyasviel/control_v11p_sd15_inpaint", dtype=torch.float16, variant="fp16") # pass ControlNet to the pipeline pipeline = StableDiffusionControlNetInpaintPipeline.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-inpainting", controlnet=controlnet, torch_dtype=torch.float16, variant="fp16" + "stable-diffusion-v1-5/stable-diffusion-inpainting", controlnet=controlnet, dtype=torch.float16, variant="fp16" ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed @@ -752,7 +752,7 @@ You can take this a step further and chain it with an image-to-image pipeline to from diffusers import AutoPipelineForImage2Image pipeline = AutoPipelineForImage2Image.from_pretrained( - "nitrosocke/elden-ring-diffusion", torch_dtype=torch.float16, + "nitrosocke/elden-ring-diffusion", dtype=torch.float16, ) pipeline.enable_model_cpu_offload() # remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed diff --git a/docs/source/en/using-diffusers/ip_adapter.md b/docs/source/en/using-diffusers/ip_adapter.md index e552bd254739..7535b6762f2a 100644 --- a/docs/source/en/using-diffusers/ip_adapter.md +++ b/docs/source/en/using-diffusers/ip_adapter.md @@ -28,7 +28,7 @@ from diffusers.utils import load_image pipeline = AutoPipelineForText2Image.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.load_ip_adapter( "h94/IP-Adapter", @@ -72,7 +72,7 @@ from diffusers.utils import load_image pipeline = AutoPipelineForImage2Image.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.load_ip_adapter( "h94/IP-Adapter", @@ -116,7 +116,7 @@ from diffusers.utils import load_image pipeline = AutoPipelineForImage2Image.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.load_ip_adapter( "h94/IP-Adapter", @@ -164,12 +164,12 @@ from diffusers.utils import load_image adapter = MotionAdapter.from_pretrained( "guoyww/animatediff-motion-adapter-v1-5-2", - torch_dtype=torch.float16 + dtype=torch.float16 ) pipeline = AnimateDiffPipeline.from_pretrained( "emilianJR/epiCRealism", motion_adapter=adapter, - torch_dtype=torch.float16 + dtype=torch.float16 ) scheduler = DDIMScheduler.from_pretrained( "emilianJR/epiCRealism", @@ -223,13 +223,13 @@ from transformers import CLIPVisionModelWithProjection, AutoPipelineForText2Imag image_encoder = CLIPVisionModelWithProjection.from_pretrained( "h94/IP-Adapter", subfolder="models/image_encoder", - torch_dtype=torch.float16 + dtype=torch.float16 ) pipeline = AutoPipelineForText2Image.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", image_encoder=image_encoder, - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.load_ip_adapter( @@ -248,7 +248,7 @@ from transformers import AutoPipelineForText2Image pipeline = AutoPipelineForText2Image.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.load_ip_adapter( @@ -266,13 +266,13 @@ from transformers import AutoPipelineForText2Image, CLIPVisionModelWithProjectio image_encoder = CLIPVisionModelWithProjection.from_pretrained( "laion/CLIP-ViT-H-14-laion2B-s32B-b79K", - torch_dtype=torch.float16, + dtype=torch.float16, ) pipeline = AutoPipelineForText2Image.from_pretrained( "stable-diffusion-v1-5/stable-diffusion-v1-5", image_encoder=image_encoder, - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.load_ip_adapter( @@ -295,7 +295,7 @@ from diffusers import AutoPipelineForText2Image pipeline = AutoPipelineForImage2Image.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") image_embeds = pipeline.prepare_ip_adapter_image_embeds( @@ -346,7 +346,7 @@ from diffusers.utils import load_image pipeline = AutoPipelineForImage2Image.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") mask1 = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/ip_mask_mask1.png") @@ -434,7 +434,7 @@ from diffusers.utils import load_image pipeline = StableDiffusionPipeline.from_pretrained( "stable-diffusion-v1-5/stable-diffusion-v1-5", - torch_dtype=torch.float16, + dtype=torch.float16, ).to("cuda") pipeline.scheduler = DDIMScheduler.from_config(pipeline.scheduler.config) pipeline.load_ip_adapter( @@ -479,7 +479,7 @@ from insightface.app import FaceAnalysis pipeline = StableDiffusionPipeline.from_pretrained( "stable-diffusion-v1-5/stable-diffusion-v1-5", - torch_dtype=torch.float16, + dtype=torch.float16, ).to("cuda") pipeline.scheduler = DDIMScheduler.from_config(pipeline.scheduler.config) pipeline.load_ip_adapter( @@ -539,7 +539,7 @@ from diffusers.utils import load_image image_encoder = CLIPVisionModelWithProjection.from_pretrained( "h94/IP-Adapter", subfolder="models/image_encoder", - torch_dtype=torch.float16, + dtype=torch.float16, ) ``` @@ -551,7 +551,7 @@ Load a base model, scheduler and the following IP-Adapters. ```py pipeline = AutoPipelineForText2Image.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16, + dtype=torch.float16, image_encoder=image_encoder, ) pipeline.scheduler = DDIMScheduler.from_config(pipeline.scheduler.config) @@ -616,7 +616,7 @@ from diffusers.utils import load_image pipeline = DiffusionPipeline.from_pretrained( "sd-dreambooth-library/herge-style", - torch_dtype=torch.float16 + dtype=torch.float16 ) pipeline.load_ip_adapter( @@ -666,13 +666,13 @@ from diffusers import StableDiffusionControlNetPipeline, ControlNetModel controlnet = ControlNetModel.from_pretrained( "lllyasviel/control_v11f1p_sd15_depth", - torch_dtype=torch.float16 + dtype=torch.float16 ) pipeline = StableDiffusionControlNetPipeline.from_pretrained( "stable-diffusion-v1-5/stable-diffusion-v1-5", controlnet=controlnet, - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.load_ip_adapter( "h94/IP-Adapter", @@ -720,7 +720,7 @@ from diffusers.utils import load_image pipeline = AutoPipelineForText2Image.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.load_ip_adapter( "h94/IP-Adapter", diff --git a/docs/source/en/using-diffusers/loading.md b/docs/source/en/using-diffusers/loading.md index 3fb608b1c26c..0f3ab4c289e8 100644 --- a/docs/source/en/using-diffusers/loading.md +++ b/docs/source/en/using-diffusers/loading.md @@ -29,7 +29,7 @@ import torch from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( - "Qwen/Qwen-Image", torch_dtype=torch.bfloat16, device_map="cuda" + "Qwen/Qwen-Image", dtype=torch.bfloat16, device_map="cuda" ) ``` @@ -48,7 +48,7 @@ import torch from diffusers import QwenImagePipeline pipeline = QwenImagePipeline.from_pretrained( - "Qwen/Qwen-Image", torch_dtype=torch.bfloat16, device_map="cuda" + "Qwen/Qwen-Image", dtype=torch.bfloat16, device_map="cuda" ) ``` @@ -72,7 +72,7 @@ import torch from diffusers import QwenImagePipeline pipeline = QwenImagePipeline.from_pretrained( - "path/to/your/cache", torch_dtype=torch.bfloat16, device_map="cuda" + "path/to/your/cache", dtype=torch.bfloat16, device_map="cuda" ) ``` @@ -80,9 +80,9 @@ The [`~QwenImagePipeline.from_pretrained`] method won't download files from the ## Pipeline data types -Use the `torch_dtype` argument in [`~DiffusionPipeline.from_pretrained`] to load a model with a specific data type. This allows you to load different models in different precisions. For example, loading a large transformer model in half-precision reduces the memory required. +Use the `dtype` argument in [`~DiffusionPipeline.from_pretrained`] to load a model with a specific data type. This allows you to load different models in different precisions. For example, loading a large transformer model in half-precision reduces the memory required. -Pass the data type for each model as a dictionary to `torch_dtype`. Use the `default` key to set the default data type. If a model isn't in the dictionary and `default` isn't provided, it is loaded in full precision (`torch.float32`). +Pass the data type for each model as a dictionary to `dtype`. Use the `default` key to set the default data type. If a model isn't in the dictionary and `default` isn't provided, it is loaded in full precision (`torch.float32`). ```py import torch @@ -90,7 +90,7 @@ from diffusers import QwenImagePipeline pipeline = QwenImagePipeline.from_pretrained( "Qwen/Qwen-Image", - torch_dtype={"transformer": torch.bfloat16, "default": torch.float16}, + dtype={"transformer": torch.bfloat16, "default": torch.float16}, ) print(pipeline.transformer.dtype, pipeline.vae.dtype) ``` @@ -102,7 +102,7 @@ import torch from diffusers import QwenImagePipeline pipeline = QwenImagePipeline.from_pretrained( - "Qwen/Qwen-Image", torch_dtype=torch.bfloat16 + "Qwen/Qwen-Image", dtype=torch.bfloat16 ) print(pipeline.transformer.dtype, pipeline.vae.dtype) ``` @@ -127,7 +127,7 @@ from diffusers import DiffusionPipeline max_memory = {0: "16GB", 1: "16GB"} pipeline = DiffusionPipeline.from_pretrained( "Qwen/Qwen-Image", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, device_map="cuda", ) ``` @@ -161,7 +161,7 @@ from diffusers import DiffusionPipeline os.environ["HF_ENABLE_PARALLEL_LOADING"] = "YES" pipeline = DiffusionPipeline.from_pretrained( - "Wan-AI/Wan2.2-I2V-A14B-Diffusers", torch_dtype=torch.bfloat16, device_map="cuda" + "Wan-AI/Wan2.2-I2V-A14B-Diffusers", dtype=torch.bfloat16, device_map="cuda" ) ``` @@ -176,13 +176,13 @@ import torch from diffusers import DiffusionPipeline, AutoModel vae = AutoModel.from_pretrained( - "madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16 + "madebyollin/sdxl-vae-fp16-fix", dtype=torch.float16 ) pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", vae=vae, - torch_dtype=torch.float16, + dtype=torch.float16, device_map="cuda" ) ``` @@ -203,7 +203,7 @@ import torch from diffusers import AutoPipelineForText2Image pipeline_sdxl = AutoPipelineForText2Image.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16, device_map="cuda" + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16, device_map="cuda" ) prompt = """ cinematic film still of a cat sipping a margarita in a pool in Palm Springs, California diff --git a/docs/source/en/using-diffusers/other-formats.md b/docs/source/en/using-diffusers/other-formats.md index c4b150098e8d..263829f3de19 100644 --- a/docs/source/en/using-diffusers/other-formats.md +++ b/docs/source/en/using-diffusers/other-formats.md @@ -45,7 +45,7 @@ from diffusers import StableDiffusionXLPipeline pipeline = StableDiffusionXLPipeline.from_single_file( "https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/blob/main/sd_xl_base_1.0.safetensors", - torch_dtype=torch.float16, + dtype=torch.float16, device_map="cuda" ) ``` @@ -57,12 +57,12 @@ import torch from diffusers import FluxPipeline, FluxTransformer2DModel transformer = FluxTransformer2DModel.from_single_file( - "https://huggingface.co/Kijai/flux-fp8/blob/main/flux1-dev-fp8.safetensors", torch_dtype=torch.bfloat16 + "https://huggingface.co/Kijai/flux-fp8/blob/main/flux1-dev-fp8.safetensors", dtype=torch.bfloat16 ) pipeline = FluxPipeline.from_pretrained( "black-forest-labs/FLUX.1-dev", transformer=transformer, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, device_map="cuda" ) ``` @@ -180,7 +180,7 @@ pipeline = DiffusionPipeline.from_pretrained( pipeline = DiffusionPipeline.from_single_file( "https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/blob/main/sd_xl_base_1.0.safetensors", - torch_dtype=torch.float16, + dtype=torch.float16, ) ``` @@ -193,7 +193,7 @@ import torch from diffusers import FluxPipeline pipeline = FluxPipeline.from_pretrained( - "black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16 + "black-forest-labs/FLUX.1-dev", dtype=torch.bfloat16 ).to("cuda") pipeline.load_lora_weights("linoyts/yarn_art_Flux_LoRA") pipeline.save_lora_weights( @@ -237,7 +237,7 @@ from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( "DDUF/FLUX.1-dev-DDUF", dduf_file="FLUX.1-dev.dduf", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, device_map="cuda" ) ``` @@ -249,7 +249,7 @@ import torch from diffusers import DiffusionPipeline from huggingface_hub import export_folder_as_dduf -pipeline = DiffusionPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16) +pipeline = DiffusionPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", dtype=torch.bfloat16) save_folder = "flux-dev" pipeline.save_pretrained("flux-dev") diff --git a/docs/source/en/using-diffusers/schedulers.md b/docs/source/en/using-diffusers/schedulers.md index 0e236e4e3e1d..aa4acde8254e 100644 --- a/docs/source/en/using-diffusers/schedulers.md +++ b/docs/source/en/using-diffusers/schedulers.md @@ -29,7 +29,7 @@ import torch from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16, device_map="cuda" + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16, device_map="cuda" ) pipeline.scheduler ``` @@ -45,7 +45,7 @@ dpm = DPMSolverMultistepScheduler.from_pretrained( pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", scheduler=dpm, - torch_dtype=torch.float16, + dtype=torch.float16, device_map="cuda" ) pipeline.scheduler @@ -73,7 +73,7 @@ print(sampling_schedule) pipeline = DiffusionPipeline.from_pretrained( "SG161222/RealVisXL_V4.0", - torch_dtype=torch.float16, + dtype=torch.float16, device_map="cuda" ) pipeline.scheduler = DPMSolverMultistepScheduler.from_config( @@ -167,7 +167,7 @@ from diffusers import DiffusionPipeline, DPMSolverMultistepScheduler pipeline = DiffusionPipeline.from_pretrained( "SG161222/RealVisXL_V4.0", - torch_dtype=torch.float16, + dtype=torch.float16, device_map="cuda" ) pipeline.scheduler = DPMSolverMultistepScheduler.from_config( @@ -209,7 +209,7 @@ from diffusers import DiffusionPipeline, DPMSolverMultistepScheduler pipeline = DiffusionPipeline.from_pretrained( "SG161222/RealVisXL_V4.0", - torch_dtype=torch.float16, + dtype=torch.float16, device_map="cuda" ) pipeline.scheduler = DPMSolverMultistepScheduler.from_config( @@ -237,7 +237,7 @@ from diffusers import DiffusionPipeline, DPMSolverMultistepScheduler pipeline = DiffusionPipeline.from_pretrained( "SG161222/RealVisXL_V4.0", - torch_dtype=torch.float16, + dtype=torch.float16, device_map="cuda" ) pipeline.scheduler = DPMSolverMultistepScheduler.from_config( diff --git a/docs/source/en/using-diffusers/t2i_adapter.md b/docs/source/en/using-diffusers/t2i_adapter.md index 734adf625cab..4c397ea017d3 100644 --- a/docs/source/en/using-diffusers/t2i_adapter.md +++ b/docs/source/en/using-diffusers/t2i_adapter.md @@ -22,7 +22,7 @@ from diffusers import T2IAdapter, StableDiffusionXLAdapterPipeline, AutoencoderK t2i_adapter = T2IAdapter.from_pretrained( "TencentARC/t2i-adapter-canny-sdxl-1.0", - torch_dtype=torch.float16, + dtype=torch.float16, ) ``` @@ -52,12 +52,12 @@ canny_image = Image.fromarray(image) Pass the canny image to the pipeline to generate an image. ```py -vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16) +vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", dtype=torch.float16) pipeline = StableDiffusionXLAdapterPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", adapter=t2i_adapter, vae=vae, - torch_dtype=torch.float16, + dtype=torch.float16, ).to("cuda") prompt = """ @@ -115,8 +115,8 @@ bright sunny day, vacation scene, 35mm photograph, film, professional, 4k, highl adapters = MultiAdapter( [ - T2IAdapter.from_pretrained("TencentARC/t2i-adapter-canny-sdxl-1.0", torch_dtype=torch.float16), - T2IAdapter.from_pretrained("TencentARC/t2i-adapter-depth-midas-sdxl-1.0", torch_dtype=torch.float16), + T2IAdapter.from_pretrained("TencentARC/t2i-adapter-canny-sdxl-1.0", dtype=torch.float16), + T2IAdapter.from_pretrained("TencentARC/t2i-adapter-depth-midas-sdxl-1.0", dtype=torch.float16), ] ) ``` @@ -124,10 +124,10 @@ adapters = MultiAdapter( Pass the adapters, prompt, and control images to [`StableDiffusionXLAdapterPipeline`]. Use the `adapter_conditioning_scale` parameter to determine how much weight to assign to each control. ```py -vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16) +vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", dtype=torch.float16) pipeline = StableDiffusionXLAdapterPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16, + dtype=torch.float16, vae=vae, adapter=adapters, ).to("cuda") diff --git a/docs/source/en/using-diffusers/text-img2vid.md b/docs/source/en/using-diffusers/text-img2vid.md index 9b69a2fded5c..0c846a383ee3 100644 --- a/docs/source/en/using-diffusers/text-img2vid.md +++ b/docs/source/en/using-diffusers/text-img2vid.md @@ -30,9 +30,9 @@ from diffusers.hooks.group_offloading import apply_group_offloading from diffusers.utils import export_to_video, load_image from transformers import UMT5EncoderModel -text_encoder = UMT5EncoderModel.from_pretrained("Wan-AI/Wan2.1-T2V-14B-Diffusers", subfolder="text_encoder", torch_dtype=torch.bfloat16) -vae = AutoModel.from_pretrained("Wan-AI/Wan2.1-T2V-14B-Diffusers", subfolder="vae", torch_dtype=torch.float32) -transformer = AutoModel.from_pretrained("Wan-AI/Wan2.1-T2V-14B-Diffusers", subfolder="transformer", torch_dtype=torch.bfloat16) +text_encoder = UMT5EncoderModel.from_pretrained("Wan-AI/Wan2.1-T2V-14B-Diffusers", subfolder="text_encoder", dtype=torch.bfloat16) +vae = AutoModel.from_pretrained("Wan-AI/Wan2.1-T2V-14B-Diffusers", subfolder="vae", dtype=torch.float32) +transformer = AutoModel.from_pretrained("Wan-AI/Wan2.1-T2V-14B-Diffusers", subfolder="transformer", dtype=torch.bfloat16) # group-offloading onload_device = torch.device("cuda") @@ -55,7 +55,7 @@ pipeline = WanPipeline.from_pretrained( vae=vae, transformer=transformer, text_encoder=text_encoder, - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) pipeline.to("cuda") @@ -104,7 +104,7 @@ pipeline_quant_config = PipelineQuantizationConfig( pipeline = HunyuanVideoPipeline.from_pretrained( "hunyuanvideo-community/HunyuanVideo", quantization_config=pipeline_quant_config, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ) # model-offloading and tiling @@ -129,13 +129,13 @@ from diffusers.utils import export_to_video transformer = AutoModel.from_pretrained( "Lightricks/LTX-Video", subfolder="transformer", - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) transformer.enable_layerwise_casting( storage_dtype=torch.float8_e4m3fn, compute_dtype=torch.bfloat16 ) -pipeline = LTXPipeline.from_pretrained("Lightricks/LTX-Video", transformer=transformer, torch_dtype=torch.bfloat16) +pipeline = LTXPipeline.from_pretrained("Lightricks/LTX-Video", transformer=transformer, dtype=torch.bfloat16) # group-offloading onload_device = torch.device("cuda") @@ -186,7 +186,7 @@ from diffusers import LTXPipeline from diffusers.utils import export_to_video pipeline = LTXPipeline.from_pretrained( - "Lightricks/LTX-Video", torch_dtype=torch.bfloat16 + "Lightricks/LTX-Video", dtype=torch.bfloat16 ).to("cuda") prompt = """ @@ -221,7 +221,7 @@ from diffusers.utils import export_to_video pipeline = CogVideoXPipeline.from_pretrained( "THUDM/CogVideoX-2b", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") prompt = """ @@ -253,10 +253,10 @@ from diffusers.schedulers.scheduling_unipc_multistep import UniPCMultistepSchedu from diffusers.utils import export_to_video vae = AutoencoderKLWan.from_pretrained( - "Wan-AI/Wan2.1-T2V-14B-Diffusers", subfolder="vae", torch_dtype=torch.float32 + "Wan-AI/Wan2.1-T2V-14B-Diffusers", subfolder="vae", dtype=torch.float32 ) pipeline = WanPipeline.from_pretrained( - "Wan-AI/Wan2.1-T2V-14B-Diffusers", vae=vae, torch_dtype=torch.bfloat16 + "Wan-AI/Wan2.1-T2V-14B-Diffusers", vae=vae, dtype=torch.bfloat16 ) pipeline.scheduler = UniPCMultistepScheduler.from_config( pipeline.scheduler.config, flow_shift=5.0 @@ -303,9 +303,9 @@ from diffusers.hooks.group_offloading import apply_group_offloading from diffusers.utils import export_to_video, load_image from transformers import UMT5EncoderModel -text_encoder = UMT5EncoderModel.from_pretrained("Wan-AI/Wan2.1-T2V-14B-Diffusers", subfolder="text_encoder", torch_dtype=torch.bfloat16) -vae = AutoModel.from_pretrained("Wan-AI/Wan2.1-T2V-14B-Diffusers", subfolder="vae", torch_dtype=torch.float32) -transformer = AutoModel.from_pretrained("Wan-AI/Wan2.1-T2V-14B-Diffusers", subfolder="transformer", torch_dtype=torch.bfloat16) +text_encoder = UMT5EncoderModel.from_pretrained("Wan-AI/Wan2.1-T2V-14B-Diffusers", subfolder="text_encoder", dtype=torch.bfloat16) +vae = AutoModel.from_pretrained("Wan-AI/Wan2.1-T2V-14B-Diffusers", subfolder="vae", dtype=torch.float32) +transformer = AutoModel.from_pretrained("Wan-AI/Wan2.1-T2V-14B-Diffusers", subfolder="transformer", dtype=torch.bfloat16) # group-offloading onload_device = torch.device("cuda") @@ -328,7 +328,7 @@ pipeline = WanPipeline.from_pretrained( vae=vae, transformer=transformer, text_encoder=text_encoder, - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) pipeline.to("cuda") @@ -377,10 +377,10 @@ pipeline_quant_config = PipelineQuantizationConfig( ) vae = AutoModel.from_pretrained( - "Wan-AI/Wan2.1-T2V-14B-Diffusers", subfolder="vae", torch_dtype=torch.float32 + "Wan-AI/Wan2.1-T2V-14B-Diffusers", subfolder="vae", dtype=torch.float32 ) pipeline = WanPipeline.from_pretrained( - "Wan-AI/Wan2.1-T2V-14B-Diffusers", vae=vae, quantization_config=pipeline_quant_config, torch_dtype=torch.bfloat16 + "Wan-AI/Wan2.1-T2V-14B-Diffusers", vae=vae, quantization_config=pipeline_quant_config, dtype=torch.bfloat16 ) pipeline.scheduler = UniPCMultistepScheduler.from_config( pipeline.scheduler.config, flow_shift=5.0 @@ -422,7 +422,7 @@ from diffusers.utils import export_to_video pipeline = CogVideoXPipeline.from_pretrained( "THUDM/CogVideoX-2b", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") # torch.compile diff --git a/docs/source/en/using-diffusers/textual_inversion_inference.md b/docs/source/en/using-diffusers/textual_inversion_inference.md index c29844a90b14..21105e1d4012 100644 --- a/docs/source/en/using-diffusers/textual_inversion_inference.md +++ b/docs/source/en/using-diffusers/textual_inversion_inference.md @@ -22,7 +22,7 @@ from diffusers import AutoPipelineForText2Image pipeline = AutoPipelineForText2Image.from_pretrained( "stable-diffusion-v1-5/stable-diffusion-v1-5", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") ``` @@ -48,7 +48,7 @@ from diffusers import AutoPipelineForText2Image pipeline = AutoPipelineForText2Image.from_pretrained( "stable-diffusion-v1-5/stable-diffusion-v1-5", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.load_textual_inversion( "EvilEngine/easynegative", diff --git a/docs/source/en/using-diffusers/weighted_prompts.md b/docs/source/en/using-diffusers/weighted_prompts.md index f89ebfe4a289..11d667180392 100644 --- a/docs/source/en/using-diffusers/weighted_prompts.md +++ b/docs/source/en/using-diffusers/weighted_prompts.md @@ -79,7 +79,7 @@ from diffusers import DiffusionPipeline from sd_embed.embedding_funcs import get_weighted_text_embeddings_sdxl pipeline = DiffusionPipeline.from_pretrained( - "Lykon/dreamshaper-xl-1-0", torch_dtype=torch.bfloat16, device_map="cuda" + "Lykon/dreamshaper-xl-1-0", dtype=torch.bfloat16, device_map="cuda" ) prompt = """ diff --git a/docs/source/ja/stable_diffusion.md b/docs/source/ja/stable_diffusion.md index 79abfa005d62..bae7ea4774c4 100644 --- a/docs/source/ja/stable_diffusion.md +++ b/docs/source/ja/stable_diffusion.md @@ -72,7 +72,7 @@ image ```python import torch -pipeline = DiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16, use_safetensors=True) +pipeline = DiffusionPipeline.from_pretrained(model_id, dtype=torch.float16, use_safetensors=True) pipeline = pipeline.to("cuda") generator = torch.Generator("cuda").manual_seed(0) image = pipeline(prompt, generator=generator).images[0] @@ -191,7 +191,7 @@ make_image_grid(images, rows=2, cols=4) ```python from diffusers import AutoencoderKL -vae = AutoencoderKL.from_pretrained("stabilityai/sd-vae-ft-mse", torch_dtype=torch.float16).to("cuda") +vae = AutoencoderKL.from_pretrained("stabilityai/sd-vae-ft-mse", dtype=torch.float16).to("cuda") pipeline.vae = vae images = pipeline(**get_inputs(batch_size=8)).images make_image_grid(images, rows=2, cols=4) diff --git a/docs/source/ja/tutorials/autopipeline.md b/docs/source/ja/tutorials/autopipeline.md index 7dc678da90be..94bc82344dbc 100644 --- a/docs/source/ja/tutorials/autopipeline.md +++ b/docs/source/ja/tutorials/autopipeline.md @@ -29,7 +29,7 @@ from diffusers import AutoPipelineForText2Image import torch pipeline = AutoPipelineForText2Image.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16, use_safetensors=True + "stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16, use_safetensors=True ).to("cuda") prompt = "peasant and dragon combat, wood cutting style, viking era, bevel with rune" @@ -57,7 +57,7 @@ from io import BytesIO pipeline = AutoPipelineForImage2Image.from_pretrained( "stable-diffusion-v1-5/stable-diffusion-v1-5", - torch_dtype=torch.float16, + dtype=torch.float16, use_safetensors=True, ).to("cuda") prompt = "a portrait of a dog wearing a pearl earring" @@ -84,7 +84,7 @@ from diffusers.utils import load_image import torch pipeline = AutoPipelineForInpainting.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16, use_safetensors=True + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16, use_safetensors=True ).to("cuda") img_url = "https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo.png" @@ -109,7 +109,7 @@ from diffusers import AutoPipelineForImage2Image import torch pipeline = AutoPipelineForImage2Image.from_pretrained( - "openai/shap-e-img2img", torch_dtype=torch.float16, use_safetensors=True + "openai/shap-e-img2img", dtype=torch.float16, use_safetensors=True ) "ValueError: AutoPipeline can't find a pipeline linked to ShapEImg2ImgPipeline for None" ``` @@ -125,7 +125,7 @@ from diffusers import AutoPipelineForText2Image, AutoPipelineForImage2Image import torch pipeline_text2img = AutoPipelineForText2Image.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16, use_safetensors=True + "stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16, use_safetensors=True ) print(type(pipeline_text2img)) "" @@ -146,7 +146,7 @@ import torch pipeline_text2img = AutoPipelineForText2Image.from_pretrained( "stable-diffusion-v1-5/stable-diffusion-v1-5", - torch_dtype=torch.float16, + dtype=torch.float16, use_safetensors=True, requires_safety_checker=False, ).to("cuda") diff --git a/docs/source/ko/api/pipelines/stable_diffusion/stable_diffusion_xl.md b/docs/source/ko/api/pipelines/stable_diffusion/stable_diffusion_xl.md index ba85b4a855d3..76058ca9e10a 100644 --- a/docs/source/ko/api/pipelines/stable_diffusion/stable_diffusion_xl.md +++ b/docs/source/ko/api/pipelines/stable_diffusion/stable_diffusion_xl.md @@ -67,7 +67,7 @@ from diffusers import StableDiffusionXLPipeline import torch pipe = StableDiffusionXLPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16, variant="fp16", use_safetensors=True + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16, variant="fp16", use_safetensors=True ) pipe.to("cuda") @@ -85,7 +85,7 @@ from diffusers import StableDiffusionXLImg2ImgPipeline from diffusers.utils import load_image pipe = StableDiffusionXLImg2ImgPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-refiner-1.0", torch_dtype=torch.float16, variant="fp16", use_safetensors=True + "stabilityai/stable-diffusion-xl-refiner-1.0", dtype=torch.float16, variant="fp16", use_safetensors=True ) pipe = pipe.to("cuda") url = "https://huggingface.co/datasets/patrickvonplaten/images/resolve/main/aa_xl/000000009.png" @@ -105,7 +105,7 @@ from diffusers import StableDiffusionXLInpaintPipeline from diffusers.utils import load_image pipe = StableDiffusionXLInpaintPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16, variant="fp16", use_safetensors=True + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16, variant="fp16", use_safetensors=True ) pipe.to("cuda") @@ -154,7 +154,7 @@ from diffusers import DiffusionPipeline import torch base = DiffusionPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16, variant="fp16", use_safetensors=True + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16, variant="fp16", use_safetensors=True ) pipe.to("cuda") @@ -162,7 +162,7 @@ refiner = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-refiner-1.0", text_encoder_2=base.text_encoder_2, vae=base.vae, - torch_dtype=torch.float16, + dtype=torch.float16, use_safetensors=True, variant="fp16", ) @@ -221,7 +221,7 @@ from diffusers import DiffusionPipeline import torch pipe = DiffusionPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16, variant="fp16", use_safetensors=True + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16, variant="fp16", use_safetensors=True ) pipe.to("cuda") @@ -229,7 +229,7 @@ refiner = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-refiner-1.0", text_encoder_2=pipe.text_encoder_2, vae=pipe.vae, - torch_dtype=torch.float16, + dtype=torch.float16, use_safetensors=True, variant="fp16", ) @@ -255,7 +255,7 @@ from diffusers import StableDiffusionXLInpaintPipeline from diffusers.utils import load_image pipe = StableDiffusionXLInpaintPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16, variant="fp16", use_safetensors=True + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16, variant="fp16", use_safetensors=True ) pipe.to("cuda") @@ -263,7 +263,7 @@ refiner = StableDiffusionXLInpaintPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-refiner-1.0", text_encoder_2=pipe.text_encoder_2, vae=pipe.vae, - torch_dtype=torch.float16, + dtype=torch.float16, use_safetensors=True, variant="fp16", ) @@ -307,12 +307,12 @@ from diffusers import StableDiffusionXLPipeline, StableDiffusionXLImg2ImgPipelin import torch pipe = StableDiffusionXLPipeline.from_single_file( - "./sd_xl_base_1.0.safetensors", torch_dtype=torch.float16 + "./sd_xl_base_1.0.safetensors", dtype=torch.float16 ) pipe.to("cuda") refiner = StableDiffusionXLImg2ImgPipeline.from_single_file( - "./sd_xl_refiner_1.0.safetensors", torch_dtype=torch.float16 + "./sd_xl_refiner_1.0.safetensors", dtype=torch.float16 ) refiner.to("cuda") ``` @@ -382,7 +382,7 @@ from diffusers import StableDiffusionXLPipeline import torch pipe = StableDiffusionXLPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-0.9", torch_dtype=torch.float16, variant="fp16", use_safetensors=True + "stabilityai/stable-diffusion-xl-base-0.9", dtype=torch.float16, variant="fp16", use_safetensors=True ) pipe.to("cuda") diff --git a/docs/source/ko/conceptual/evaluation.md b/docs/source/ko/conceptual/evaluation.md index 731b511485c3..f1aa05951a61 100644 --- a/docs/source/ko/conceptual/evaluation.md +++ b/docs/source/ko/conceptual/evaluation.md @@ -117,7 +117,7 @@ from diffusers import StableDiffusionPipeline import torch model_ckpt = "CompVis/stable-diffusion-v1-4" -sd_pipeline = StableDiffusionPipeline.from_pretrained(model_ckpt, torch_dtype=torch.float16).to("cuda") +sd_pipeline = StableDiffusionPipeline.from_pretrained(model_ckpt, dtype=torch.float16).to("cuda") ``` 여러 개의 프롬프트를 사용하여 이미지를 생성합니다: @@ -171,7 +171,7 @@ images = sd_pipeline(prompts, num_images_per_prompt=1, generator=generator, outp ```python model_ckpt_1_5 = "stable-diffusion-v1-5/stable-diffusion-v1-5" -sd_pipeline_1_5 = StableDiffusionPipeline.from_pretrained(model_ckpt_1_5, torch_dtype=weight_dtype).to(device) +sd_pipeline_1_5 = StableDiffusionPipeline.from_pretrained(model_ckpt_1_5, dtype=weight_dtype).to(device) images_1_5 = sd_pipeline_1_5(prompts, num_images_per_prompt=1, generator=generator, output_type="np").images ``` @@ -263,7 +263,7 @@ dataset[idx]["image"] from diffusers import StableDiffusionInstructPix2PixPipeline instruct_pix2pix_pipeline = StableDiffusionInstructPix2PixPipeline.from_pretrained( - "timbrooks/instruct-pix2pix", torch_dtype=torch.float16 + "timbrooks/instruct-pix2pix", dtype=torch.float16 ).to(device) ``` @@ -474,7 +474,7 @@ print(real_images.shape) ```python from diffusers import DiTPipeline, DPMSolverMultistepScheduler -dit_pipeline = DiTPipeline.from_pretrained("facebook/DiT-XL-2-256", torch_dtype=torch.float16) +dit_pipeline = DiTPipeline.from_pretrained("facebook/DiT-XL-2-256", dtype=torch.float16) dit_pipeline.scheduler = DPMSolverMultistepScheduler.from_config(dit_pipeline.scheduler.config) dit_pipeline = dit_pipeline.to("cuda") diff --git a/docs/source/ko/optimization/fp16.md b/docs/source/ko/optimization/fp16.md index 0642517c0557..c33480dbe0c9 100644 --- a/docs/source/ko/optimization/fp16.md +++ b/docs/source/ko/optimization/fp16.md @@ -66,7 +66,7 @@ torch.backends.cuda.matmul.allow_tf32 = True pipe = StableDiffusionPipeline.from_pretrained( "stable-diffusion-v1-5/stable-diffusion-v1-5", - torch_dtype=torch.float16, + dtype=torch.float16, ) pipe = pipe.to("cuda") @@ -94,7 +94,7 @@ from diffusers import StableDiffusionPipeline pipe = StableDiffusionPipeline.from_pretrained( "stable-diffusion-v1-5/stable-diffusion-v1-5", - torch_dtype=torch.float16, + dtype=torch.float16, ) pipe = pipe.to("cuda") @@ -121,7 +121,7 @@ from diffusers import StableDiffusionPipeline pipe = StableDiffusionPipeline.from_pretrained( "stable-diffusion-v1-5/stable-diffusion-v1-5", - torch_dtype=torch.float16, + dtype=torch.float16, ) pipe = pipe.to("cuda") @@ -147,7 +147,7 @@ from diffusers import StableDiffusionPipeline pipe = StableDiffusionPipeline.from_pretrained( "stable-diffusion-v1-5/stable-diffusion-v1-5", - torch_dtype=torch.float16, + dtype=torch.float16, ) prompt = "a photo of an astronaut riding a horse on mars" @@ -172,7 +172,7 @@ from diffusers import StableDiffusionPipeline pipe = StableDiffusionPipeline.from_pretrained( "stable-diffusion-v1-5/stable-diffusion-v1-5", - torch_dtype=torch.float16, + dtype=torch.float16, ) prompt = "a photo of an astronaut riding a horse on mars" @@ -202,7 +202,7 @@ from diffusers import StableDiffusionPipeline pipe = StableDiffusionPipeline.from_pretrained( "stable-diffusion-v1-5/stable-diffusion-v1-5", - torch_dtype=torch.float16, + dtype=torch.float16, ) prompt = "a photo of an astronaut riding a horse on mars" @@ -218,7 +218,7 @@ from diffusers import StableDiffusionPipeline pipe = StableDiffusionPipeline.from_pretrained( "stable-diffusion-v1-5/stable-diffusion-v1-5", - torch_dtype=torch.float16, + dtype=torch.float16, ) prompt = "a photo of an astronaut riding a horse on mars" @@ -277,7 +277,7 @@ def generate_inputs(): pipe = StableDiffusionPipeline.from_pretrained( "stable-diffusion-v1-5/stable-diffusion-v1-5", - torch_dtype=torch.float16, + dtype=torch.float16, ).to("cuda") unet = pipe.unet unet.eval() @@ -340,7 +340,7 @@ class UNet2DConditionOutput: pipe = StableDiffusionPipeline.from_pretrained( "stable-diffusion-v1-5/stable-diffusion-v1-5", - torch_dtype=torch.float16, + dtype=torch.float16, ).to("cuda") # jitted unet 사용 @@ -393,7 +393,7 @@ import torch pipe = StableDiffusionPipeline.from_pretrained( "stable-diffusion-v1-5/stable-diffusion-v1-5", - torch_dtype=torch.float16, + dtype=torch.float16, ).to("cuda") pipe.enable_xformers_memory_efficient_attention() diff --git a/docs/source/ko/optimization/tome.md b/docs/source/ko/optimization/tome.md index db16afb0fa3e..e5df3011fb8c 100644 --- a/docs/source/ko/optimization/tome.md +++ b/docs/source/ko/optimization/tome.md @@ -25,7 +25,7 @@ from diffusers import StableDiffusionPipeline import tomesd pipeline = StableDiffusionPipeline.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16 + "stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16 ).to("cuda") + tomesd.apply_patch(pipeline, ratio=0.5) diff --git a/docs/source/ko/optimization/torch2.0.md b/docs/source/ko/optimization/torch2.0.md index 354f7243cf50..ff644f3689d5 100644 --- a/docs/source/ko/optimization/torch2.0.md +++ b/docs/source/ko/optimization/torch2.0.md @@ -37,7 +37,7 @@ pip install --upgrade torch diffusers import torch from diffusers import DiffusionPipeline - pipe = DiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16) + pipe = DiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16) pipe = pipe.to("cuda") prompt = "a photo of an astronaut riding a horse on mars" @@ -51,7 +51,7 @@ pip install --upgrade torch diffusers from diffusers import DiffusionPipeline + from diffusers.models.attention_processor import AttnProcessor2_0 - pipe = DiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16).to("cuda") + pipe = DiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16).to("cuda") + pipe.unet.set_attn_processor(AttnProcessor2_0()) prompt = "a photo of an astronaut riding a horse on mars" @@ -67,7 +67,7 @@ pip install --upgrade torch diffusers from diffusers import DiffusionPipeline from diffusers.models.attention_processor import AttnProcessor - pipe = DiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16).to("cuda") + pipe = DiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16).to("cuda") pipe.unet.set_default_attn_processor() prompt = "a photo of an astronaut riding a horse on mars" @@ -104,7 +104,7 @@ path = "stable-diffusion-v1-5/stable-diffusion-v1-5" run_compile = True # Set True / False -pipe = DiffusionPipeline.from_pretrained(path, torch_dtype=torch.float16) +pipe = DiffusionPipeline.from_pretrained(path, dtype=torch.float16) pipe = pipe.to("cuda") pipe.unet.to(memory_format=torch.channels_last) @@ -137,7 +137,7 @@ path = "stable-diffusion-v1-5/stable-diffusion-v1-5" run_compile = True # Set True / False -pipe = StableDiffusionImg2ImgPipeline.from_pretrained(path, torch_dtype=torch.float16) +pipe = StableDiffusionImg2ImgPipeline.from_pretrained(path, dtype=torch.float16) pipe = pipe.to("cuda") pipe.unet.to(memory_format=torch.channels_last) @@ -177,7 +177,7 @@ path = "stable-diffusion-v1-5/stable-diffusion-inpainting" run_compile = True # Set True / False -pipe = StableDiffusionInpaintPipeline.from_pretrained(path, torch_dtype=torch.float16) +pipe = StableDiffusionInpaintPipeline.from_pretrained(path, dtype=torch.float16) pipe = pipe.to("cuda") pipe.unet.to(memory_format=torch.channels_last) @@ -209,9 +209,9 @@ init_image = init_image.resize((512, 512)) path = "stable-diffusion-v1-5/stable-diffusion-v1-5" run_compile = True # Set True / False -controlnet = ControlNetModel.from_pretrained("lllyasviel/sd-controlnet-canny", torch_dtype=torch.float16) +controlnet = ControlNetModel.from_pretrained("lllyasviel/sd-controlnet-canny", dtype=torch.float16) pipe = StableDiffusionControlNetPipeline.from_pretrained( - path, controlnet=controlnet, torch_dtype=torch.float16 + path, controlnet=controlnet, dtype=torch.float16 ) pipe = pipe.to("cuda") @@ -237,11 +237,11 @@ import torch run_compile = True # Set True / False -pipe = DiffusionPipeline.from_pretrained("DeepFloyd/IF-I-M-v1.0", variant="fp16", text_encoder=None, torch_dtype=torch.float16) +pipe = DiffusionPipeline.from_pretrained("DeepFloyd/IF-I-M-v1.0", variant="fp16", text_encoder=None, dtype=torch.float16) pipe.to("cuda") -pipe_2 = DiffusionPipeline.from_pretrained("DeepFloyd/IF-II-M-v1.0", variant="fp16", text_encoder=None, torch_dtype=torch.float16) +pipe_2 = DiffusionPipeline.from_pretrained("DeepFloyd/IF-II-M-v1.0", variant="fp16", text_encoder=None, dtype=torch.float16) pipe_2.to("cuda") -pipe_3 = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-x4-upscaler", torch_dtype=torch.float16) +pipe_3 = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-x4-upscaler", dtype=torch.float16) pipe_3.to("cuda") diff --git a/docs/source/ko/stable_diffusion.md b/docs/source/ko/stable_diffusion.md index 0f61e16d2a9c..ebd464217058 100644 --- a/docs/source/ko/stable_diffusion.md +++ b/docs/source/ko/stable_diffusion.md @@ -73,7 +73,7 @@ image ```python import torch -pipeline = DiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16) +pipeline = DiffusionPipeline.from_pretrained(model_id, dtype=torch.float16) pipeline = pipeline.to("cuda") generator = torch.Generator("cuda").manual_seed(0) image = pipeline(prompt, generator=generator).images[0] @@ -209,7 +209,7 @@ image_grid(images, rows=2, cols=4) ```python from diffusers import AutoencoderKL -vae = AutoencoderKL.from_pretrained("stabilityai/sd-vae-ft-mse", torch_dtype=torch.float16).to("cuda") +vae = AutoencoderKL.from_pretrained("stabilityai/sd-vae-ft-mse", dtype=torch.float16).to("cuda") pipeline.vae = vae images = pipeline(**get_inputs(batch_size=8)).images image_grid(images, rows=2, cols=4) diff --git a/docs/source/ko/training/controlnet.md b/docs/source/ko/training/controlnet.md index e868b57c5546..f6c45ed9b1e1 100644 --- a/docs/source/ko/training/controlnet.md +++ b/docs/source/ko/training/controlnet.md @@ -299,9 +299,9 @@ import torch base_model_path = "path to model" controlnet_path = "path to controlnet" -controlnet = ControlNetModel.from_pretrained(controlnet_path, torch_dtype=torch.float16) +controlnet = ControlNetModel.from_pretrained(controlnet_path, dtype=torch.float16) pipe = StableDiffusionControlNetPipeline.from_pretrained( - base_model_path, controlnet=controlnet, torch_dtype=torch.float16 + base_model_path, controlnet=controlnet, dtype=torch.float16 ) # 더 빠른 스케줄러와 메모리 최적화로 diffusion 프로세스 속도 올리기 diff --git a/docs/source/ko/training/custom_diffusion.md b/docs/source/ko/training/custom_diffusion.md index fcf292483aff..b901fd34d694 100644 --- a/docs/source/ko/training/custom_diffusion.md +++ b/docs/source/ko/training/custom_diffusion.md @@ -221,7 +221,7 @@ accelerate launch train_custom_diffusion.py \ import torch from diffusers import DiffusionPipeline -pipe = DiffusionPipeline.from_pretrained("CompVis/stable-diffusion-v1-4", torch_dtype=torch.float16).to("cuda") +pipe = DiffusionPipeline.from_pretrained("CompVis/stable-diffusion-v1-4", dtype=torch.float16).to("cuda") pipe.unet.load_attn_procs("path-to-save-model", weight_name="pytorch_custom_diffusion_weights.bin") pipe.load_textual_inversion("path-to-save-model", weight_name=".bin") @@ -245,7 +245,7 @@ model_id = "sayakpaul/custom-diffusion-cat" card = RepoCard.load(model_id) base_model_id = card.data.to_dict()["base_model"] -pipe = DiffusionPipeline.from_pretrained(base_model_id, torch_dtype=torch.float16).to("cuda") +pipe = DiffusionPipeline.from_pretrained(base_model_id, dtype=torch.float16).to("cuda") pipe.unet.load_attn_procs(model_id, weight_name="pytorch_custom_diffusion_weights.bin") pipe.load_textual_inversion(model_id, weight_name=".bin") @@ -269,7 +269,7 @@ model_id = "sayakpaul/custom-diffusion-cat-wooden-pot" card = RepoCard.load(model_id) base_model_id = card.data.to_dict()["base_model"] -pipe = DiffusionPipeline.from_pretrained(base_model_id, torch_dtype=torch.float16).to("cuda") +pipe = DiffusionPipeline.from_pretrained(base_model_id, dtype=torch.float16).to("cuda") pipe.unet.load_attn_procs(model_id, weight_name="pytorch_custom_diffusion_weights.bin") pipe.load_textual_inversion(model_id, weight_name=".bin") pipe.load_textual_inversion(model_id, weight_name=".bin") diff --git a/docs/source/ko/training/distributed_inference.md b/docs/source/ko/training/distributed_inference.md index e63764f5eb8c..cf73da79882a 100644 --- a/docs/source/ko/training/distributed_inference.md +++ b/docs/source/ko/training/distributed_inference.md @@ -17,7 +17,7 @@ from accelerate import PartialState from diffusers import DiffusionPipeline -pipeline = DiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16) +pipeline = DiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16) distributed_state = PartialState() pipeline.to(distributed_state.device) @@ -50,7 +50,7 @@ import torch.multiprocessing as mp from diffusers import DiffusionPipeline -sd = DiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16) +sd = DiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16) ``` 사용할 백엔드 유형, 현재 프로세스의 `rank`, `world_size` 또는 참여하는 프로세스 수로 분산 환경 생성을 처리하는 함수[`init_process_group`]를 만들어 추론을 실행해야 합니다. diff --git a/docs/source/ko/training/dreambooth.md b/docs/source/ko/training/dreambooth.md index 35b5a0c4d2f1..e4a53a7ec414 100644 --- a/docs/source/ko/training/dreambooth.md +++ b/docs/source/ko/training/dreambooth.md @@ -382,7 +382,7 @@ from diffusers import StableDiffusionPipeline import torch model_id = "path_to_saved_model" -pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16).to("cuda") +pipe = StableDiffusionPipeline.from_pretrained(model_id, dtype=torch.float16).to("cuda") prompt = "A photo of sks dog in a bucket" image = pipe(prompt, num_inference_steps=50, guidance_scale=7.5).images[0] diff --git a/docs/source/ko/training/instructpix2pix.md b/docs/source/ko/training/instructpix2pix.md index 2f3302800d33..acbaa6d3913f 100644 --- a/docs/source/ko/training/instructpix2pix.md +++ b/docs/source/ko/training/instructpix2pix.md @@ -167,7 +167,7 @@ import torch from diffusers import StableDiffusionInstructPix2PixPipeline model_id = "your_model_id" # <- 이를 수정하세요. -pipe = StableDiffusionInstructPix2PixPipeline.from_pretrained(model_id, torch_dtype=torch.float16).to("cuda") +pipe = StableDiffusionInstructPix2PixPipeline.from_pretrained(model_id, dtype=torch.float16).to("cuda") generator = torch.Generator("cuda").manual_seed(0) url = "https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/test_pix2pix_4.png" diff --git a/docs/source/ko/training/lora.md b/docs/source/ko/training/lora.md index 515e3fd65e89..0351bbd4d46f 100644 --- a/docs/source/ko/training/lora.md +++ b/docs/source/ko/training/lora.md @@ -93,7 +93,7 @@ accelerate launch train_dreambooth_lora.py \ >>> model_base = "stable-diffusion-v1-5/stable-diffusion-v1-5" ->>> pipe = StableDiffusionPipeline.from_pretrained(model_base, torch_dtype=torch.float16) +>>> pipe = StableDiffusionPipeline.from_pretrained(model_base, dtype=torch.float16) ``` *기본 모델의 가중치 위에* 파인튜닝된 DreamBooth 모델에서 LoRA 가중치를 불러온 다음, 더 빠른 추론을 위해 파이프라인을 GPU로 이동합니다. LoRA 가중치를 프리징된 사전 훈련된 모델 가중치와 병합할 때, 선택적으로 'scale' 매개변수로 어느 정도의 가중치를 병합할 지 조절할 수 있습니다: diff --git a/docs/source/ko/training/text2image.md b/docs/source/ko/training/text2image.md index f7cc57543357..7413648d0897 100644 --- a/docs/source/ko/training/text2image.md +++ b/docs/source/ko/training/text2image.md @@ -134,7 +134,7 @@ Text-to-image 모델 파인튜닝을 위해, 대규모 모델 학습을 가속 from diffusers import StableDiffusionPipeline model_path = "path_to_saved_model" -pipe = StableDiffusionPipeline.from_pretrained(model_path, torch_dtype=torch.float16) +pipe = StableDiffusionPipeline.from_pretrained(model_path, dtype=torch.float16) pipe.to("cuda") image = pipe(prompt="yoda").images[0] diff --git a/docs/source/ko/training/text_inversion.md b/docs/source/ko/training/text_inversion.md index 4c8659280d5a..b085daf3ab5a 100644 --- a/docs/source/ko/training/text_inversion.md +++ b/docs/source/ko/training/text_inversion.md @@ -163,7 +163,7 @@ from diffusers import StableDiffusionPipeline import torch model_id = "stable-diffusion-v1-5/stable-diffusion-v1-5" -pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16).to("cuda") +pipe = StableDiffusionPipeline.from_pretrained(model_id, dtype=torch.float16).to("cuda") ``` 다음으로 `TextualInversionLoaderMixin.load_textual_inversion` 함수를 통해, textual-inversion 임베딩 벡터를 불러와야 합니다. 여기서 우리는 이전의 `` 예제의 임베딩을 불러올 것입니다. diff --git a/docs/source/ko/using-diffusers/depth2img.md b/docs/source/ko/using-diffusers/depth2img.md index 90644ca087d0..d0b5881d4e08 100644 --- a/docs/source/ko/using-diffusers/depth2img.md +++ b/docs/source/ko/using-diffusers/depth2img.md @@ -28,7 +28,7 @@ from diffusers import StableDiffusionDepth2ImgPipeline pipe = StableDiffusionDepth2ImgPipeline.from_pretrained( "stabilityai/stable-diffusion-2-depth", - torch_dtype=torch.float16, + dtype=torch.float16, ).to("cuda") ``` diff --git a/docs/source/ko/using-diffusers/diffedit.md b/docs/source/ko/using-diffusers/diffedit.md index 626331a05236..1e4d508e8ec2 100644 --- a/docs/source/ko/using-diffusers/diffedit.md +++ b/docs/source/ko/using-diffusers/diffedit.md @@ -46,7 +46,7 @@ from diffusers import DDIMScheduler, DDIMInverseScheduler, StableDiffusionDiffEd pipeline = StableDiffusionDiffEditPipeline.from_pretrained( "stabilityai/stable-diffusion-2-1", - torch_dtype=torch.float16, + dtype=torch.float16, safety_checker=None, use_safetensors=True, ) @@ -122,7 +122,7 @@ import torch from transformers import AutoTokenizer, T5ForConditionalGeneration tokenizer = AutoTokenizer.from_pretrained("google/flan-t5-large") -model = T5ForConditionalGeneration.from_pretrained("google/flan-t5-large", device_map="auto", torch_dtype=torch.float16) +model = T5ForConditionalGeneration.from_pretrained("google/flan-t5-large", device_map="auto", dtype=torch.float16) ``` 모델에 프롬프트할 source와 target 프롬프트를 생성하기 위해 초기 텍스트들을 제공합니다. @@ -166,7 +166,7 @@ import torch from diffusers import StableDiffusionDiffEditPipeline pipeline = StableDiffusionDiffEditPipeline.from_pretrained( - "stabilityai/stable-diffusion-2-1", torch_dtype=torch.float16, use_safetensors=True + "stabilityai/stable-diffusion-2-1", dtype=torch.float16, use_safetensors=True ) pipeline.enable_model_cpu_offload() pipeline.vae.enable_slicing() @@ -241,7 +241,7 @@ import torch from transformers import BlipForConditionalGeneration, BlipProcessor processor = BlipProcessor.from_pretrained("Salesforce/blip-image-captioning-base") -model = BlipForConditionalGeneration.from_pretrained("Salesforce/blip-image-captioning-base", torch_dtype=torch.float16, low_cpu_mem_usage=True) +model = BlipForConditionalGeneration.from_pretrained("Salesforce/blip-image-captioning-base", dtype=torch.float16, low_cpu_mem_usage=True) ``` 입력 이미지에서 캡션을 생성하는 유틸리티 함수를 만듭니다: diff --git a/docs/source/ko/using-diffusers/img2img.md b/docs/source/ko/using-diffusers/img2img.md index 3901fb755f8d..4e22e2d4eb8e 100644 --- a/docs/source/ko/using-diffusers/img2img.md +++ b/docs/source/ko/using-diffusers/img2img.md @@ -33,7 +33,7 @@ from io import BytesIO from diffusers import StableDiffusionImg2ImgPipeline device = "cuda" -pipe = StableDiffusionImg2ImgPipeline.from_pretrained("nitrosocke/Ghibli-Diffusion", torch_dtype=torch.float16).to( +pipe = StableDiffusionImg2ImgPipeline.from_pretrained("nitrosocke/Ghibli-Diffusion", dtype=torch.float16).to( device ) ``` diff --git a/docs/source/ko/using-diffusers/inpaint.md b/docs/source/ko/using-diffusers/inpaint.md index 6c0c08bf7312..fefbef0add63 100644 --- a/docs/source/ko/using-diffusers/inpaint.md +++ b/docs/source/ko/using-diffusers/inpaint.md @@ -28,7 +28,7 @@ from diffusers import StableDiffusionInpaintPipeline pipeline = StableDiffusionInpaintPipeline.from_pretrained( "stable-diffusion-v1-5/stable-diffusion-inpainting", - torch_dtype=torch.float16, + dtype=torch.float16, ) pipeline = pipeline.to("cuda") ``` diff --git a/docs/source/ko/using-diffusers/kandinsky.md b/docs/source/ko/using-diffusers/kandinsky.md index 8eff8f5629a5..393d7a0a6751 100644 --- a/docs/source/ko/using-diffusers/kandinsky.md +++ b/docs/source/ko/using-diffusers/kandinsky.md @@ -49,8 +49,8 @@ Kandinsky 모델은 일련의 다국어 text-to-image 생성 모델입니다. Ka from diffusers import KandinskyPriorPipeline, KandinskyPipeline import torch -prior_pipeline = KandinskyPriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-1-prior", torch_dtype=torch.float16).to("cuda") -pipeline = KandinskyPipeline.from_pretrained("kandinsky-community/kandinsky-2-1", torch_dtype=torch.float16).to("cuda") +prior_pipeline = KandinskyPriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-1-prior", dtype=torch.float16).to("cuda") +pipeline = KandinskyPipeline.from_pretrained("kandinsky-community/kandinsky-2-1", dtype=torch.float16).to("cuda") prompt = "A alien cheeseburger creature eating itself, claymation, cinematic, moody lighting" negative_prompt = "low quality, bad quality" # negative 프롬프트 포함은 선택적이지만, 보통 결과는 더 좋습니다 @@ -75,8 +75,8 @@ image from diffusers import KandinskyV22PriorPipeline, KandinskyV22Pipeline import torch -prior_pipeline = KandinskyV22PriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-2-prior", torch_dtype=torch.float16).to("cuda") -pipeline = KandinskyV22Pipeline.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", torch_dtype=torch.float16).to("cuda") +prior_pipeline = KandinskyV22PriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-2-prior", dtype=torch.float16).to("cuda") +pipeline = KandinskyV22Pipeline.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", dtype=torch.float16).to("cuda") prompt = "A alien cheeseburger creature eating itself, claymation, cinematic, moody lighting" negative_prompt = "low quality, bad quality" # negative 프롬프트 포함은 선택적이지만, 보통 결과는 더 좋습니다 @@ -103,7 +103,7 @@ Kandinsky 3는 prior 모델이 필요하지 않으므로 [`Kandinsky3Pipeline`] from diffusers import Kandinsky3Pipeline import torch -pipeline = Kandinsky3Pipeline.from_pretrained("kandinsky-community/kandinsky-3", variant="fp16", torch_dtype=torch.float16) +pipeline = Kandinsky3Pipeline.from_pretrained("kandinsky-community/kandinsky-3", variant="fp16", dtype=torch.float16) pipeline.enable_model_cpu_offload() prompt = "A alien cheeseburger creature eating itself, claymation, cinematic, moody lighting" @@ -125,7 +125,7 @@ image from diffusers import AutoPipelineForText2Image import torch -pipeline = AutoPipelineForText2Image.from_pretrained("kandinsky-community/kandinsky-2-1", torch_dtype=torch.float16) +pipeline = AutoPipelineForText2Image.from_pretrained("kandinsky-community/kandinsky-2-1", dtype=torch.float16) pipeline.enable_model_cpu_offload() prompt = "A alien cheeseburger creature eating itself, claymation, cinematic, moody lighting" @@ -142,7 +142,7 @@ image from diffusers import AutoPipelineForText2Image import torch -pipeline = AutoPipelineForText2Image.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", torch_dtype=torch.float16) +pipeline = AutoPipelineForText2Image.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", dtype=torch.float16) pipeline.enable_model_cpu_offload() prompt = "A alien cheeseburger creature eating itself, claymation, cinematic, moody lighting" @@ -166,8 +166,8 @@ Image-to-image 경우, 초기 이미지와 텍스트 프롬프트를 전달하 import torch from diffusers import KandinskyImg2ImgPipeline, KandinskyPriorPipeline -prior_pipeline = KandinskyPriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-1-prior", torch_dtype=torch.float16, use_safetensors=True).to("cuda") -pipeline = KandinskyImg2ImgPipeline.from_pretrained("kandinsky-community/kandinsky-2-1", torch_dtype=torch.float16, use_safetensors=True).to("cuda") +prior_pipeline = KandinskyPriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-1-prior", dtype=torch.float16, use_safetensors=True).to("cuda") +pipeline = KandinskyImg2ImgPipeline.from_pretrained("kandinsky-community/kandinsky-2-1", dtype=torch.float16, use_safetensors=True).to("cuda") ``` @@ -177,8 +177,8 @@ pipeline = KandinskyImg2ImgPipeline.from_pretrained("kandinsky-community/kandins import torch from diffusers import KandinskyV22Img2ImgPipeline, KandinskyPriorPipeline -prior_pipeline = KandinskyPriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-2-prior", torch_dtype=torch.float16, use_safetensors=True).to("cuda") -pipeline = KandinskyV22Img2ImgPipeline.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", torch_dtype=torch.float16, use_safetensors=True).to("cuda") +prior_pipeline = KandinskyPriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-2-prior", dtype=torch.float16, use_safetensors=True).to("cuda") +pipeline = KandinskyV22Img2ImgPipeline.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", dtype=torch.float16, use_safetensors=True).to("cuda") ``` @@ -191,7 +191,7 @@ from diffusers import Kandinsky3Img2ImgPipeline from diffusers.utils import load_image import torch -pipeline = Kandinsky3Img2ImgPipeline.from_pretrained("kandinsky-community/kandinsky-3", variant="fp16", torch_dtype=torch.float16) +pipeline = Kandinsky3Img2ImgPipeline.from_pretrained("kandinsky-community/kandinsky-3", variant="fp16", dtype=torch.float16) pipeline.enable_model_cpu_offload() ``` @@ -275,7 +275,7 @@ from diffusers import AutoPipelineForImage2Image from diffusers.utils import make_image_grid, load_image import torch -pipeline = AutoPipelineForImage2Image.from_pretrained("kandinsky-community/kandinsky-2-1", torch_dtype=torch.float16, use_safetensors=True) +pipeline = AutoPipelineForImage2Image.from_pretrained("kandinsky-community/kandinsky-2-1", dtype=torch.float16, use_safetensors=True) pipeline.enable_model_cpu_offload() prompt = "A fantasy landscape, Cinematic lighting" @@ -298,7 +298,7 @@ from diffusers import AutoPipelineForImage2Image from diffusers.utils import make_image_grid, load_image import torch -pipeline = AutoPipelineForImage2Image.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", torch_dtype=torch.float16) +pipeline = AutoPipelineForImage2Image.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", dtype=torch.float16) pipeline.enable_model_cpu_offload() prompt = "A fantasy landscape, Cinematic lighting" @@ -342,8 +342,8 @@ import torch import numpy as np from PIL import Image -prior_pipeline = KandinskyPriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-1-prior", torch_dtype=torch.float16, use_safetensors=True).to("cuda") -pipeline = KandinskyInpaintPipeline.from_pretrained("kandinsky-community/kandinsky-2-1-inpaint", torch_dtype=torch.float16, use_safetensors=True).to("cuda") +prior_pipeline = KandinskyPriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-1-prior", dtype=torch.float16, use_safetensors=True).to("cuda") +pipeline = KandinskyInpaintPipeline.from_pretrained("kandinsky-community/kandinsky-2-1-inpaint", dtype=torch.float16, use_safetensors=True).to("cuda") ``` @@ -356,8 +356,8 @@ import torch import numpy as np from PIL import Image -prior_pipeline = KandinskyV22PriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-2-prior", torch_dtype=torch.float16, use_safetensors=True).to("cuda") -pipeline = KandinskyV22InpaintPipeline.from_pretrained("kandinsky-community/kandinsky-2-2-decoder-inpaint", torch_dtype=torch.float16, use_safetensors=True).to("cuda") +prior_pipeline = KandinskyV22PriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-2-prior", dtype=torch.float16, use_safetensors=True).to("cuda") +pipeline = KandinskyV22InpaintPipeline.from_pretrained("kandinsky-community/kandinsky-2-2-decoder-inpaint", dtype=torch.float16, use_safetensors=True).to("cuda") ``` @@ -422,7 +422,7 @@ from PIL import Image from diffusers import AutoPipelineForInpainting from diffusers.utils import load_image, make_image_grid -pipe = AutoPipelineForInpainting.from_pretrained("kandinsky-community/kandinsky-2-1-inpaint", torch_dtype=torch.float16) +pipe = AutoPipelineForInpainting.from_pretrained("kandinsky-community/kandinsky-2-1-inpaint", dtype=torch.float16) pipe.enable_model_cpu_offload() init_image = load_image("https://huggingface.co/datasets/hf-internal-testing/diffusers-images/resolve/main/kandinsky/cat.png") @@ -446,7 +446,7 @@ from PIL import Image from diffusers import AutoPipelineForInpainting from diffusers.utils import load_image, make_image_grid -pipe = AutoPipelineForInpainting.from_pretrained("kandinsky-community/kandinsky-2-2-decoder-inpaint", torch_dtype=torch.float16) +pipe = AutoPipelineForInpainting.from_pretrained("kandinsky-community/kandinsky-2-2-decoder-inpaint", dtype=torch.float16) pipe.enable_model_cpu_offload() init_image = load_image("https://huggingface.co/datasets/hf-internal-testing/diffusers-images/resolve/main/kandinsky/cat.png") @@ -475,7 +475,7 @@ from diffusers import KandinskyPriorPipeline, KandinskyPipeline from diffusers.utils import load_image, make_image_grid import torch -prior_pipeline = KandinskyPriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-1-prior", torch_dtype=torch.float16, use_safetensors=True).to("cuda") +prior_pipeline = KandinskyPriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-1-prior", dtype=torch.float16, use_safetensors=True).to("cuda") img_1 = load_image("https://huggingface.co/datasets/hf-internal-testing/diffusers-images/resolve/main/kandinsky/cat.png") img_2 = load_image("https://huggingface.co/datasets/hf-internal-testing/diffusers-images/resolve/main/kandinsky/starry_night.jpeg") make_image_grid([img_1.resize((512,512)), img_2.resize((512,512))], rows=1, cols=2) @@ -489,7 +489,7 @@ from diffusers import KandinskyV22PriorPipeline, KandinskyV22Pipeline from diffusers.utils import load_image, make_image_grid import torch -prior_pipeline = KandinskyV22PriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-2-prior", torch_dtype=torch.float16, use_safetensors=True).to("cuda") +prior_pipeline = KandinskyV22PriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-2-prior", dtype=torch.float16, use_safetensors=True).to("cuda") img_1 = load_image("https://huggingface.co/datasets/hf-internal-testing/diffusers-images/resolve/main/kandinsky/cat.png") img_2 = load_image("https://huggingface.co/datasets/hf-internal-testing/diffusers-images/resolve/main/kandinsky/starry_night.jpeg") make_image_grid([img_1.resize((512,512)), img_2.resize((512,512))], rows=1, cols=2) @@ -526,7 +526,7 @@ weights = [0.3, 0.3, 0.4] prompt = "" prior_out = prior_pipeline.interpolate(images_texts, weights) -pipeline = KandinskyPipeline.from_pretrained("kandinsky-community/kandinsky-2-1", torch_dtype=torch.float16, use_safetensors=True).to("cuda") +pipeline = KandinskyPipeline.from_pretrained("kandinsky-community/kandinsky-2-1", dtype=torch.float16, use_safetensors=True).to("cuda") image = pipeline(prompt, **prior_out, height=768, width=768).images[0] image @@ -544,7 +544,7 @@ image prompt = "" prior_out = prior_pipeline.interpolate(images_texts, weights) -pipeline = KandinskyV22Pipeline.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", torch_dtype=torch.float16, use_safetensors=True).to("cuda") +pipeline = KandinskyV22Pipeline.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", dtype=torch.float16, use_safetensors=True).to("cuda") image = pipeline(prompt, **prior_out, height=768, width=768).images[0] image @@ -608,11 +608,11 @@ Prior 파이프라인과 [`KandinskyV22ControlnetPipeline`]를 불러옵니다: from diffusers import KandinskyV22PriorPipeline, KandinskyV22ControlnetPipeline prior_pipeline = KandinskyV22PriorPipeline.from_pretrained( - "kandinsky-community/kandinsky-2-2-prior", torch_dtype=torch.float16, use_safetensors=True + "kandinsky-community/kandinsky-2-2-prior", dtype=torch.float16, use_safetensors=True ).to("cuda") pipeline = KandinskyV22ControlnetPipeline.from_pretrained( - "kandinsky-community/kandinsky-2-2-controlnet-depth", torch_dtype=torch.float16 + "kandinsky-community/kandinsky-2-2-controlnet-depth", dtype=torch.float16 ).to("cuda") ``` @@ -678,11 +678,11 @@ Prior 파이프라인과 [`KandinskyV22ControlnetImg2ImgPipeline`]을 불러옵 ```py prior_pipeline = KandinskyV22PriorEmb2EmbPipeline.from_pretrained( - "kandinsky-community/kandinsky-2-2-prior", torch_dtype=torch.float16, use_safetensors=True + "kandinsky-community/kandinsky-2-2-prior", dtype=torch.float16, use_safetensors=True ).to("cuda") pipeline = KandinskyV22ControlnetImg2ImgPipeline.from_pretrained( - "kandinsky-community/kandinsky-2-2-controlnet-depth", torch_dtype=torch.float16 + "kandinsky-community/kandinsky-2-2-controlnet-depth", dtype=torch.float16 ).to("cuda") ``` @@ -719,7 +719,7 @@ Kandinsky는 mapping을 생성하기 위한 prior 파이프라인과 latents를 from diffusers import DiffusionPipeline import torch - pipe = DiffusionPipeline.from_pretrained("kandinsky-community/kandinsky-2-1", torch_dtype=torch.float16) + pipe = DiffusionPipeline.from_pretrained("kandinsky-community/kandinsky-2-1", dtype=torch.float16) + pipe.enable_xformers_memory_efficient_attention() ``` @@ -744,7 +744,7 @@ pipe.unet.set_attn_processor(AttnAddedKVProcessor2_0()) from diffusers import DiffusionPipeline import torch - pipe = DiffusionPipeline.from_pretrained("kandinsky-community/kandinsky-2-1", torch_dtype=torch.float16) + pipe = DiffusionPipeline.from_pretrained("kandinsky-community/kandinsky-2-1", dtype=torch.float16) + pipe.enable_model_cpu_offload() ``` @@ -755,5 +755,5 @@ from diffusers import DDPMScheduler from diffusers import DiffusionPipeline scheduler = DDPMScheduler.from_pretrained("kandinsky-community/kandinsky-2-1", subfolder="ddpm_scheduler") -pipe = DiffusionPipeline.from_pretrained("kandinsky-community/kandinsky-2-1", scheduler=scheduler, torch_dtype=torch.float16, use_safetensors=True).to("cuda") +pipe = DiffusionPipeline.from_pretrained("kandinsky-community/kandinsky-2-1", scheduler=scheduler, dtype=torch.float16, use_safetensors=True).to("cuda") ``` diff --git a/docs/source/ko/using-diffusers/loading.md b/docs/source/ko/using-diffusers/loading.md index 2160acacc2e0..96a4606fecee 100644 --- a/docs/source/ko/using-diffusers/loading.md +++ b/docs/source/ko/using-diffusers/loading.md @@ -178,12 +178,12 @@ Variant란 일반적으로 다음과 같은 체크포인트들을 의미합니 | **checkpoint type** | **weight name** | **argument for loading weights** | | ------------------- | ----------------------------------- | -------------------------------- | | original | diffusion_pytorch_model.bin | | -| floating point | diffusion_pytorch_model.fp16.bin | `variant`, `torch_dtype` | +| floating point | diffusion_pytorch_model.fp16.bin | `variant`, `dtype` | | non-EMA | diffusion_pytorch_model.non_ema.bin | `variant` | variant를 로드할 때 2개의 중요한 argument가 있습니다. -* `torch_dtype`은 불러올 체크포인트의 부동소수점을 정의합니다. 예를 들어 `torch_dtype=torch.float16`을 명시함으로써 가중치의 부동소수점 타입을 `fl16`으로 변환할 수 있습니다. (만약 따로 설정하지 않을 경우, 기본값으로 `fp32` 타입의 가중치가 로딩됩니다.) 또한 `variant` 인자를 명시하지 않은 채로 체크포인트를 불러온 다음, 해당 체크포인트를 `torch_dtype=torch.float16` 인자를 통해 `fp16` 타입으로 변환하는 것 역시 가능합니다. 이 경우 기본으로 설정된 `fp32` 가중치가 먼저 다운로드되고, 해당 가중치들을 불러온 다음 `fp16` 타입으로 변환하게 됩니다. +* `dtype`은 불러올 체크포인트의 부동소수점을 정의합니다. 예를 들어 `dtype=torch.float16`을 명시함으로써 가중치의 부동소수점 타입을 `fl16`으로 변환할 수 있습니다. (만약 따로 설정하지 않을 경우, 기본값으로 `fp32` 타입의 가중치가 로딩됩니다.) 또한 `variant` 인자를 명시하지 않은 채로 체크포인트를 불러온 다음, 해당 체크포인트를 `dtype=torch.float16` 인자를 통해 `fp16` 타입으로 변환하는 것 역시 가능합니다. 이 경우 기본으로 설정된 `fp32` 가중치가 먼저 다운로드되고, 해당 가중치들을 불러온 다음 `fp16` 타입으로 변환하게 됩니다. * `variant` 인자는 리포지토리에서 어떤 variant를 불러올 것인가를 정의합니다. 가령 [`diffusers/stable-diffusion-variants`](https://huggingface.co/diffusers/stable-diffusion-variants/tree/main/unet) 리포지토리로부터 `non_ema` 체크포인트를 불러오고자 한다면, `variant="non_ema"` 인자를 전달해야 합니다. ```python @@ -191,7 +191,7 @@ from diffusers import DiffusionPipeline # load fp16 variant stable_diffusion = DiffusionPipeline.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-v1-5", variant="fp16", torch_dtype=torch.float16 + "stable-diffusion-v1-5/stable-diffusion-v1-5", variant="fp16", dtype=torch.float16 ) # load non_ema variant stable_diffusion = DiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", variant="non_ema") @@ -212,10 +212,10 @@ stable_diffusion.save_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", ```python # 👎 this won't work -stable_diffusion = DiffusionPipeline.from_pretrained("./stable-diffusion-v1-5", torch_dtype=torch.float16) +stable_diffusion = DiffusionPipeline.from_pretrained("./stable-diffusion-v1-5", dtype=torch.float16) # 👍 this works stable_diffusion = DiffusionPipeline.from_pretrained( - "./stable-diffusion-v1-5", variant="fp16", torch_dtype=torch.float16 + "./stable-diffusion-v1-5", variant="fp16", dtype=torch.float16 ) ``` diff --git a/docs/source/ko/using-diffusers/loading_adapters.md b/docs/source/ko/using-diffusers/loading_adapters.md index e7ae116575ae..236553a44b67 100644 --- a/docs/source/ko/using-diffusers/loading_adapters.md +++ b/docs/source/ko/using-diffusers/loading_adapters.md @@ -31,7 +31,7 @@ Hergé가 그린 단 10개의 이미지로 학습된 [herge_style](https://huggi from diffusers import AutoPipelineForText2Image import torch -pipeline = AutoPipelineForText2Image.from_pretrained("sd-dreambooth-library/herge-style", torch_dtype=torch.float16).to("cuda") +pipeline = AutoPipelineForText2Image.from_pretrained("sd-dreambooth-library/herge-style", dtype=torch.float16).to("cuda") prompt = "A cute herge_style brown bear eating a slice of pizza, stunning color scheme, masterpiece, illustration" image = pipeline(prompt).images[0] image @@ -51,7 +51,7 @@ Textual inversion은 임베딩을 생성하기 때문에 DreamBooth처럼 단독 from diffusers import AutoPipelineForText2Image import torch -pipeline = AutoPipelineForText2Image.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16).to("cuda") +pipeline = AutoPipelineForText2Image.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16).to("cuda") ``` 이제 [`~loaders.TextualInversionLoaderMixin.load_textual_inversion`] 메서드를 사용하여 textual inversion 임베딩을 불러와 이미지를 생성할 수 있습니다. [sd-concepts-library/gta5-artwork](https://huggingface.co/sd-concepts-library/gta5-artwork) 임베딩을 불러와 보겠습니다. 이를 트리거하려면 프롬프트에 특수 단어 ``를 포함시켜야 합니다: @@ -107,7 +107,7 @@ LoRA는 다른 모델과 함께 사용해야 합니다: from diffusers import AutoPipelineForText2Image import torch -pipeline = AutoPipelineForText2Image.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16).to("cuda") +pipeline = AutoPipelineForText2Image.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16).to("cuda") ``` 그리고 [`~loaders.LoraLoaderMixin.load_lora_weights`] 메서드를 사용하여 [ostris/super-cereal-sdxl-lora](https://huggingface.co/ostris/super-cereal-sdxl-lora) 가중치를 불러오고 리포지토리에서 가중치 파일명을 지정합니다: @@ -134,7 +134,7 @@ image from diffusers import AutoPipelineForText2Image import torch -pipeline = AutoPipelineForText2Image.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16).to("cuda") +pipeline = AutoPipelineForText2Image.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16).to("cuda") pipeline.unet.load_attn_procs("jbilcke-hf/sdxl-cinematic-1", weight_name="pytorch_lora_weights.safetensors") # 프롬프트에서 cnmt를 사용하여 LoRA를 트리거합니다. @@ -200,7 +200,7 @@ LoRA 체크포인트를 [`~loaders.LoraLoaderMixin.load_lora_weights`] 메서드 from diffusers import AutoPipelineForText2Image import torch -pipeline = AutoPipelineForText2Image.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16).to("cuda") +pipeline = AutoPipelineForText2Image.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16).to("cuda") pipeline.load_lora_weights("path/to/weights", weight_name="blueprintify-sd-xl-10.safetensors") ``` @@ -228,7 +228,7 @@ TheLastBen에서 체크포인트를 불러오는 방법은 매우 유사합니 from diffusers import AutoPipelineForText2Image import torch -pipeline = AutoPipelineForText2Image.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16).to("cuda") +pipeline = AutoPipelineForText2Image.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16).to("cuda") pipeline.load_lora_weights("TheLastBen/William_Eggleston_Style_SDXL", weight_name="wegg.safetensors") # LoRA를 트리거하기 위해 william eggleston를 프롬프트에 사용 @@ -257,7 +257,7 @@ from diffusers import AutoPipelineForText2Image import torch from diffusers.utils import load_image -pipeline = AutoPipelineForText2Image.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16).to("cuda") +pipeline = AutoPipelineForText2Image.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16).to("cuda") ``` 그런 다음 IP-Adapter 가중치를 불러와 [`~loaders.IPAdapterMixin.load_ip_adapter`] 메서드를 사용하여 파이프라인에 추가합니다. @@ -297,13 +297,13 @@ from transformers import CLIPVisionModelWithProjection image_encoder = CLIPVisionModelWithProjection.from_pretrained( "h94/IP-Adapter", subfolder="models/image_encoder", - torch_dtype=torch.float16 + dtype=torch.float16 ) pipeline = AutoPipelineForText2Image.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", image_encoder=image_encoder, - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.load_ip_adapter("h94/IP-Adapter", subfolder="sdxl_models", weight_name="ip-adapter-plus_sdxl_vit-h.safetensors") @@ -320,7 +320,7 @@ IP-Adapter FaceID 모델은 CLIP 이미지 임베딩 대신 `insightface`에서 ```py pipeline = AutoPipelineForText2Image.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.load_ip_adapter("h94/IP-Adapter-FaceID", subfolder=None, weight_name="ip-adapter-faceid_sdxl.bin", image_encoder_folder=None) @@ -333,13 +333,13 @@ from transformers import CLIPVisionModelWithProjection image_encoder = CLIPVisionModelWithProjection.from_pretrained( "laion/CLIP-ViT-H-14-laion2B-s32B-b79K", - torch_dtype=torch.float16, + dtype=torch.float16, ) pipeline = AutoPipelineForText2Image.from_pretrained( "stable-diffusion-v1-5/stable-diffusion-v1-5", image_encoder=image_encoder, - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.load_ip_adapter("h94/IP-Adapter-FaceID", subfolder=None, weight_name="ip-adapter-faceid-plus_sd15.bin") diff --git a/docs/source/ko/using-diffusers/other-formats.md b/docs/source/ko/using-diffusers/other-formats.md index f5a71f56ebef..41f07818f5e6 100644 --- a/docs/source/ko/using-diffusers/other-formats.md +++ b/docs/source/ko/using-diffusers/other-formats.md @@ -131,7 +131,7 @@ from diffusers import DiffusionPipeline, UniPCMultistepScheduler import torch pipeline = DiffusionPipeline.from_pretrained( - "andite/anything-v4.0", torch_dtype=torch.float16, safety_checker=None + "andite/anything-v4.0", dtype=torch.float16, safety_checker=None ).to("cuda") pipeline.scheduler = UniPCMultistepScheduler.from_config(pipeline.scheduler.config) ``` diff --git a/docs/source/ko/using-diffusers/schedulers.md b/docs/source/ko/using-diffusers/schedulers.md index 02fed1d9399d..35fbac943e9d 100644 --- a/docs/source/ko/using-diffusers/schedulers.md +++ b/docs/source/ko/using-diffusers/schedulers.md @@ -43,7 +43,7 @@ import torch login() # Now we can download the pipeline -pipeline = DiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16) +pipeline = DiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16) ``` 다음으로, GPU로 이동합니다. diff --git a/docs/source/ko/using-diffusers/sdxl_turbo.md b/docs/source/ko/using-diffusers/sdxl_turbo.md index 99b96fd3b875..dfe3b2194b3d 100644 --- a/docs/source/ko/using-diffusers/sdxl_turbo.md +++ b/docs/source/ko/using-diffusers/sdxl_turbo.md @@ -33,7 +33,7 @@ SDXL Turbo는 adversarial time-distilled(적대적 시간 전이) [Stable Diffus from diffusers import AutoPipelineForText2Image, AutoPipelineForImage2Image import torch -pipeline = AutoPipelineForText2Image.from_pretrained("stabilityai/sdxl-turbo", torch_dtype=torch.float16, variant="fp16") +pipeline = AutoPipelineForText2Image.from_pretrained("stabilityai/sdxl-turbo", dtype=torch.float16, variant="fp16") pipeline = pipeline.to("cuda") ``` @@ -44,7 +44,7 @@ from diffusers import StableDiffusionXLPipeline import torch pipeline = StableDiffusionXLPipeline.from_single_file( - "https://huggingface.co/stabilityai/sdxl-turbo/blob/main/sd_xl_turbo_1.0_fp16.safetensors", torch_dtype=torch.float16) + "https://huggingface.co/stabilityai/sdxl-turbo/blob/main/sd_xl_turbo_1.0_fp16.safetensors", dtype=torch.float16) pipeline = pipeline.to("cuda") ``` @@ -59,7 +59,7 @@ Text-to-image의 경우 텍스트 프롬프트를 전달합니다. 기본적으 from diffusers import AutoPipelineForText2Image import torch -pipeline_text2image = AutoPipelineForText2Image.from_pretrained("stabilityai/sdxl-turbo", torch_dtype=torch.float16, variant="fp16") +pipeline_text2image = AutoPipelineForText2Image.from_pretrained("stabilityai/sdxl-turbo", dtype=torch.float16, variant="fp16") pipeline_text2image = pipeline_text2image.to("cuda") prompt = "A cinematic shot of a baby racoon wearing an intricate italian priest robe." diff --git a/docs/source/ko/using-diffusers/shap-e.md b/docs/source/ko/using-diffusers/shap-e.md index 4c9d7fb7d1aa..aeb694e48938 100644 --- a/docs/source/ko/using-diffusers/shap-e.md +++ b/docs/source/ko/using-diffusers/shap-e.md @@ -38,7 +38,7 @@ from diffusers import ShapEPipeline device = torch.device("cuda" if torch.cuda.is_available() else "cpu") -pipe = ShapEPipeline.from_pretrained("openai/shap-e", torch_dtype=torch.float16, variant="fp16") +pipe = ShapEPipeline.from_pretrained("openai/shap-e", dtype=torch.float16, variant="fp16") pipe = pipe.to(device) guidance_scale = 15.0 @@ -80,8 +80,8 @@ export_to_gif(images[1], "cake_3d.gif") from diffusers import DiffusionPipeline import torch -prior_pipeline = DiffusionPipeline.from_pretrained("kandinsky-community/kandinsky-2-1-prior", torch_dtype=torch.float16, use_safetensors=True).to("cuda") -pipeline = DiffusionPipeline.from_pretrained("kandinsky-community/kandinsky-2-1", torch_dtype=torch.float16, use_safetensors=True).to("cuda") +prior_pipeline = DiffusionPipeline.from_pretrained("kandinsky-community/kandinsky-2-1-prior", dtype=torch.float16, use_safetensors=True).to("cuda") +pipeline = DiffusionPipeline.from_pretrained("kandinsky-community/kandinsky-2-1", dtype=torch.float16, use_safetensors=True).to("cuda") prompt = "A cheeseburger, white background" @@ -102,7 +102,7 @@ from PIL import Image from diffusers import ShapEImg2ImgPipeline from diffusers.utils import export_to_gif -pipe = ShapEImg2ImgPipeline.from_pretrained("openai/shap-e-img2img", torch_dtype=torch.float16, variant="fp16").to("cuda") +pipe = ShapEImg2ImgPipeline.from_pretrained("openai/shap-e-img2img", dtype=torch.float16, variant="fp16").to("cuda") guidance_scale = 3.0 image = Image.open("burger.png").resize((256, 256)) @@ -140,7 +140,7 @@ from diffusers import ShapEPipeline device = torch.device("cuda" if torch.cuda.is_available() else "cpu") -pipe = ShapEPipeline.from_pretrained("openai/shap-e", torch_dtype=torch.float16, variant="fp16") +pipe = ShapEPipeline.from_pretrained("openai/shap-e", dtype=torch.float16, variant="fp16") pipe = pipe.to(device) guidance_scale = 15.0 diff --git a/docs/source/ko/using-diffusers/svd.md b/docs/source/ko/using-diffusers/svd.md index 7c5b9f09e690..81dd34ce64ce 100644 --- a/docs/source/ko/using-diffusers/svd.md +++ b/docs/source/ko/using-diffusers/svd.md @@ -35,7 +35,7 @@ from diffusers import StableVideoDiffusionPipeline from diffusers.utils import load_image, export_to_video pipe = StableVideoDiffusionPipeline.from_pretrained( - "stabilityai/stable-video-diffusion-img2vid-xt", torch_dtype=torch.float16, variant="fp16" + "stabilityai/stable-video-diffusion-img2vid-xt", dtype=torch.float16, variant="fp16" ) pipe.enable_model_cpu_offload() @@ -105,7 +105,7 @@ from diffusers import StableVideoDiffusionPipeline from diffusers.utils import load_image, export_to_video pipe = StableVideoDiffusionPipeline.from_pretrained( - "stabilityai/stable-video-diffusion-img2vid-xt", torch_dtype=torch.float16, variant="fp16" + "stabilityai/stable-video-diffusion-img2vid-xt", dtype=torch.float16, variant="fp16" ) pipe.enable_model_cpu_offload() diff --git a/docs/source/ko/using-diffusers/textual_inversion_inference.md b/docs/source/ko/using-diffusers/textual_inversion_inference.md index 868c0f74b581..9c20267910c4 100644 --- a/docs/source/ko/using-diffusers/textual_inversion_inference.md +++ b/docs/source/ko/using-diffusers/textual_inversion_inference.md @@ -49,7 +49,7 @@ repo_id_embeds = "sd-concepts-library/cat-toy" 이제 파이프라인을 로드하고 사전학습된 컨셉을 파이프라인에 전달할 수 있습니다: ```py -pipeline = StableDiffusionPipeline.from_pretrained(pretrained_model_name_or_path, torch_dtype=torch.float16).to("cuda") +pipeline = StableDiffusionPipeline.from_pretrained(pretrained_model_name_or_path, dtype=torch.float16).to("cuda") pipeline.load_textual_inversion(repo_id_embeds) ``` diff --git a/docs/source/pt/stable_diffusion.md b/docs/source/pt/stable_diffusion.md index ffc5df643454..f9d1a992c52b 100644 --- a/docs/source/pt/stable_diffusion.md +++ b/docs/source/pt/stable_diffusion.md @@ -30,7 +30,7 @@ from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, device_map="cuda" ) pipeline.enable_model_cpu_offload() @@ -48,7 +48,7 @@ print(f"Memória máxima reservada: {torch.cuda.max_memory_allocated() / 1024**3 O processo de remoção de ruído é o mais exigente computacionalmente durante a difusão. Métodos que otimizam este processo aceleram a velocidade de inferência. Experimente os seguintes métodos para acelerar. - Adicione `device_map="cuda"` para colocar o pipeline em uma GPU. Colocar um modelo em um acelerador, como uma GPU, aumenta a velocidade porque realiza computações em paralelo. -- Defina `torch_dtype=torch.bfloat16` para executar o pipeline em meia-precisão. Reduzir a precisão do tipo de dado aumenta a velocidade porque leva menos tempo para realizar computações em precisão mais baixa. +- Defina `dtype=torch.bfloat16` para executar o pipeline em meia-precisão. Reduzir a precisão do tipo de dado aumenta a velocidade porque leva menos tempo para realizar computações em precisão mais baixa. ```py import torch @@ -57,7 +57,7 @@ from diffusers import DiffusionPipeline, DPMSolverMultistepScheduler pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, device_map="cuda" ) ``` @@ -92,7 +92,7 @@ Muitos modelos de difusão modernos entregam imagens de alta qualidade imediatam pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, device_map="cuda" ) @@ -114,7 +114,7 @@ Muitos modelos de difusão modernos entregam imagens de alta qualidade imediatam pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, device_map="cuda" ) pipeline.scheduler = HeunDiscreteScheduler.from_config(pipeline.scheduler.config) diff --git a/docs/source/zh/conceptual/evaluation.md b/docs/source/zh/conceptual/evaluation.md index 770d197be041..e41e06941e1b 100644 --- a/docs/source/zh/conceptual/evaluation.md +++ b/docs/source/zh/conceptual/evaluation.md @@ -113,7 +113,7 @@ from diffusers import StableDiffusionPipeline import torch model_ckpt = "CompVis/stable-diffusion-v1-4" -sd_pipeline = StableDiffusionPipeline.from_pretrained(model_ckpt, torch_dtype=torch.float16).to("cuda") +sd_pipeline = StableDiffusionPipeline.from_pretrained(model_ckpt, dtype=torch.float16).to("cuda") ``` 使用多个提示词生成图像： @@ -167,7 +167,7 @@ images = sd_pipeline(prompts, num_images_per_prompt=1, generator=generator, outp ```python model_ckpt_1_5 = "stable-diffusion-v1-5/stable-diffusion-v1-5" -sd_pipeline_1_5 = StableDiffusionPipeline.from_pretrained(model_ckpt_1_5, torch_dtype=torch.float16).to("cuda") +sd_pipeline_1_5 = StableDiffusionPipeline.from_pretrained(model_ckpt_1_5, dtype=torch.float16).to("cuda") images_1_5 = sd_pipeline_1_5(prompts, num_images_per_prompt=1, generator=generator, output_type="np").images ``` @@ -259,7 +259,7 @@ dataset[idx]["image"] from diffusers import StableDiffusionInstructPix2PixPipeline instruct_pix2pix_pipeline = StableDiffusionInstructPix2PixPipeline.from_pretrained( - "timbrooks/instruct-pix2pix", torch_dtype=torch.float16 + "timbrooks/instruct-pix2pix", dtype=torch.float16 ).to("cuda") ``` @@ -474,7 +474,7 @@ print(real_images.shape) ```python from diffusers import DiTPipeline, DPMSolverMultistepScheduler -dit_pipeline = DiTPipeline.from_pretrained("facebook/DiT-XL-2-256", torch_dtype=torch.float16) +dit_pipeline = DiTPipeline.from_pretrained("facebook/DiT-XL-2-256", dtype=torch.float16) dit_pipeline.scheduler = DPMSolverMultistepScheduler.from_config(dit_pipeline.scheduler.config) dit_pipeline = dit_pipeline.to("cuda") diff --git a/docs/source/zh/hybrid_inference/vae_encode.md b/docs/source/zh/hybrid_inference/vae_encode.md index 30aee9a6bfa4..422c65dfadb9 100644 --- a/docs/source/zh/hybrid_inference/vae_encode.md +++ b/docs/source/zh/hybrid_inference/vae_encode.md @@ -140,7 +140,7 @@ from diffusers.utils.remote_utils import remote_decode, remote_encode pipe = StableDiffusionImg2ImgPipeline.from_pretrained( "stable-diffusion-v1-5/stable-diffusion-v1-5", - torch_dtype=torch.float16, + dtype=torch.float16, variant="fp16", vae=None, ).to("cuda") diff --git a/docs/source/zh/modular_diffusers/modular_pipeline.md b/docs/source/zh/modular_diffusers/modular_pipeline.md index a57fdf227506..eb299137c99d 100644 --- a/docs/source/zh/modular_diffusers/modular_pipeline.md +++ b/docs/source/zh/modular_diffusers/modular_pipeline.md @@ -28,7 +28,7 @@ blocks = SequentialPipelineBlocks.from_blocks_dict(TEXT2IMAGE_BLOCKS) modular_repo_id = "YiYiXu/modular-loader-t2i-0704" pipeline = blocks.init_pipeline(modular_repo_id) -pipeline.load_components(torch_dtype=torch.float16) +pipeline.load_components(dtype=torch.float16) pipeline.to("cuda") image = pipeline(prompt="Astronaut in a jungle, cold color palette, muted colors, detailed, 8k", output="images")[0] @@ -48,7 +48,7 @@ blocks = SequentialPipelineBlocks.from_blocks_dict(IMAGE2IMAGE_BLOCKS) modular_repo_id = "YiYiXu/modular-loader-t2i-0704" pipeline = blocks.init_pipeline(modular_repo_id) -pipeline.load_components(torch_dtype=torch.float16) +pipeline.load_components(dtype=torch.float16) pipeline.to("cuda") url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/sdxl-text2img.png" @@ -72,7 +72,7 @@ blocks = SequentialPipelineBlocks.from_blocks_dict(INPAINT_BLOCKS) modular_repo_id = "YiYiXu/modular-loader-t2i-0704" pipeline = blocks.init_pipeline(modular_repo_id) -pipeline.load_components(torch_dtype=torch.float16) +pipeline.load_components(dtype=torch.float16) pipeline.to("cuda") img_url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/sdxl-text2img.png" @@ -184,7 +184,7 @@ diffdiff_pipeline = ModularPipeline.from_pretrained(modular_repo_id, trust_remot ```py import torch -t2i_pipeline.load_components(torch_dtype=torch.float16) +t2i_pipeline.load_components(dtype=torch.float16) t2i_pipeline.to("cuda") ``` @@ -196,7 +196,7 @@ t2i_pipeline.to("cuda") ```py import torch -t2i_pipeline.load_components(names=["unet", "vae"], torch_dtype=torch.float16) +t2i_pipeline.load_components(names=["unet", "vae"], dtype=torch.float16) ``` @@ -283,7 +283,7 @@ t2i_pipeline.config_component_names from diffusers import ComponentSpec, UNet2DConditionModel unet_spec = ComponentSpec(name="unet",type_hint=UNet2DConditionModel, repo="stabilityai/stable-diffusion-xl-base-1.0", subfolder="unet", variant="fp16") -unet = unet_spec.load(torch_dtype=torch.float16) +unet = unet_spec.load(dtype=torch.float16) ``` [`~ModularPipeline.update_components`] 方法用一个新的组件替换原来的组件。 @@ -302,7 +302,7 @@ t2i_pipeline.update_components(unet=unet2) spec = ComponentSpec.from_component("unet", unet2) spec ComponentSpec(name='unet', type_hint=, description=None, config=None, repo='stabilityai/stable-diffusion-xl-base-1.0', subfolder='unet', variant='fp16', revision=None, default_creation_method='from_pretrained') -unet2_recreated = spec.load(torch_dtype=torch.float16) +unet2_recreated = spec.load(dtype=torch.float16) ``` [`~ModularPipeline.get_component_spec`] 方法获取当前组件规范的副本以进行修改或更新。 @@ -323,7 +323,7 @@ ComponentSpec( unet_spec.pretrained_model_name_or_path = "stabilityai/stable-diffusion-xl-base-1.0" # 使用修改后的规范加载组件 -unet = unet_spec.load(torch_dtype=torch.float16) +unet = unet_spec.load(dtype=torch.float16) ``` ## 模块化仓库 diff --git a/docs/source/zh/modular_diffusers/quickstart.md b/docs/source/zh/modular_diffusers/quickstart.md index 2c4a6a51afde..64d192ea82fc 100644 --- a/docs/source/zh/modular_diffusers/quickstart.md +++ b/docs/source/zh/modular_diffusers/quickstart.md @@ -183,7 +183,7 @@ from diffusers.modular_pipelines import ComponentsManager components = ComponentManager() dd_pipeline = dd_blocks.init_pipeline("YiYiXu/modular-demo-auto", components_manager=components, collection="diffdiff") -dd_pipeline.load_default_componenets(torch_dtype=torch.float16) +dd_pipeline.load_default_componenets(dtype=torch.float16) dd_pipeline.to("cuda") ``` @@ -213,7 +213,7 @@ dd_blocks.sub_blocks.insert("ip_adapter", ip_adapter_block, 0) ```py dd_pipeline = dd_blocks.init_pipeline("YiYiXu/modular-demo-auto", collection="diffdiff") -dd_pipeline.load_components(torch_dtype=torch.float16) +dd_pipeline.load_components(dtype=torch.float16) dd_pipeline.loader.load_ip_adapter("h94/IP-Adapter", subfolder="sdxl_models", weight_name="ip-adapter_sdxl.bin") dd_pipeline.loader.set_ip_adapter_scale(0.6) dd_pipeline = dd_pipeline.to(device) @@ -268,7 +268,7 @@ dd_blocks.sub_blocks.insert("controlnet_input", control_input_block, 7) dd_blocks.sub_blocks["denoise"] = controlnet_denoise_block dd_pipeline = dd_blocks.init_pipeline("YiYiXu/modular-demo-auto", collection="diffdiff") -dd_pipeline.load_components(torch_dtype=torch.float16) +dd_pipeline.load_components(dtype=torch.float16) dd_pipeline = dd_pipeline.to(device) control_image = load_image("https://huggingface.co/datasets/YiYiXu/testing-images/resolve/main/diffdiff_tomato_canny.jpeg") @@ -322,7 +322,7 @@ DIFFDIFF_AUTO_BLOCKS.insert("controlnet_input",StableDiffusionXLControlNetAutoIn ```py dd_auto_blocks = SequentialPipelineBlocks.from_blocks_dict(DIFFDIFF_AUTO_BLOCKS) dd_pipeline = dd_auto_blocks.init_pipeline("YiYiXu/modular-demo-auto", collection="diffdiff") -dd_pipeline.load_components(torch_dtype=torch.float16) +dd_pipeline.load_components(dtype=torch.float16) ``` ## 分享 @@ -342,5 +342,5 @@ from diffusers.modular_pipelines import ModularPipeline, ComponentsManager components = ComponentsManager() diffdiff_pipeline = ModularPipeline.from_pretrained("YiYiXu/modular-diffdiff-0704", trust_remote_code=True, components_manager=components, collection="diffdiff") -diffdiff_pipeline.load_components(torch_dtype=torch.float16) +diffdiff_pipeline.load_components(dtype=torch.float16) ``` diff --git a/docs/source/zh/optimization/cache.md b/docs/source/zh/optimization/cache.md index f7a94de4f11f..7bf3b3c4286b 100644 --- a/docs/source/zh/optimization/cache.md +++ b/docs/source/zh/optimization/cache.md @@ -27,7 +27,7 @@ PAB 可以与其他技术（如序列并行性和无分类器引导并行性（ import torch from diffusers import CogVideoXPipeline, PyramidAttentionBroadcastConfig -pipeline = CogVideoXPipeline.from_pretrained("THUDM/CogVideoX-5b", torch_dtype=torch.bfloat16) +pipeline = CogVideoXPipeline.from_pretrained("THUDM/CogVideoX-5b", dtype=torch.bfloat16) pipeline.to("cuda") config = PyramidAttentionBroadcastConfig( @@ -51,7 +51,7 @@ pipeline.transformer.enable_cache(config) import torch from diffusers import CogVideoXPipeline, FasterCacheConfig -pipe line= CogVideoXPipeline.from_pretrained("THUDM/CogVideoX-5b", torch_dtype=torch.bfloat16) +pipe line= CogVideoXPipeline.from_pretrained("THUDM/CogVideoX-5b", dtype=torch.bfloat16) pipeline.to("cuda") config = FasterCacheConfig( diff --git a/docs/source/zh/optimization/deepcache.md b/docs/source/zh/optimization/deepcache.md index 4f19d4a36528..230098bcd850 100644 --- a/docs/source/zh/optimization/deepcache.md +++ b/docs/source/zh/optimization/deepcache.md @@ -20,7 +20,7 @@ pip install DeepCache ```diff import torch from diffusers import StableDiffusionPipeline - pipe = StableDiffusionPipeline.from_pretrained('stable-diffusion-v1-5/stable-diffusion-v1-5', torch_dtype=torch.float16).to("cuda") + pipe = StableDiffusionPipeline.from_pretrained('stable-diffusion-v1-5/stable-diffusion-v1-5', dtype=torch.float16).to("cuda") + from DeepCache import DeepCacheSDHelper + helper = DeepCacheSDHelper(pipe=pipe) diff --git a/docs/source/zh/optimization/fp16.md b/docs/source/zh/optimization/fp16.md index e1c4c7e57ae7..91dd73fc8581 100644 --- a/docs/source/zh/optimization/fp16.md +++ b/docs/source/zh/optimization/fp16.md @@ -32,7 +32,7 @@ import torch from diffusers import StableDiffusionXLPipeline pipeline = StableDiffusionXLPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.bfloat16 + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.bfloat16 ).to("cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" @@ -49,7 +49,7 @@ import torch from diffusers import StableDiffusionXLPipeline pipeline = StableDiffusionXLPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16 + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16 ).to("cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" @@ -70,7 +70,7 @@ from diffusers import StableDiffusionXLPipeline torch.backends.cuda.matmul.allow_tf32 = True pipeline = StableDiffusionXLPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.bfloat16 + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.bfloat16 ).to("cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" @@ -97,7 +97,7 @@ import torch from diffusers import StableDiffusionXLPipeline pipeline = StableDiffusionXLPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.bfloat16 + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.bfloat16 ).to("cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" @@ -131,7 +131,7 @@ torch._inductor.config.coordinate_descent_check_all_directions = True ```py pipeline = StableDiffusionXLPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16 + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16 ).to("cuda") pipeline.unet.to(memory_format=torch.channels_last) pipeline.vae.to(memory_format=torch.channels_last) @@ -185,7 +185,7 @@ from diffusers import StableDiffusionXLPipeline pipeline = StableDiffusionXLPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16, + dtype=torch.float16, ).to("cuda") # 仅编译UNet中重复的transformer层 @@ -211,7 +211,7 @@ from diffusers import StableDiffusionXLPipeline from accelerate.utils import compile regions pipeline = StableDiffusionXLPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16 + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16 ).to("cuda") pipeline.unet = compile_regions(pipeline.unet, mode="reduce-overhead", fullgraph=True) ``` @@ -275,7 +275,7 @@ torch._inductor.config.use_mixed_mm = True ```py pipeline = StableDiffusionXLPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.bfloat16 + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.bfloat16 ).to("cuda") apply_dynamic_quant(pipeline.unet, dynamic_quant_filter_fn) diff --git a/docs/source/zh/optimization/habana.md b/docs/source/zh/optimization/habana.md index 9b15847d63f4..b34cd1858394 100644 --- a/docs/source/zh/optimization/habana.md +++ b/docs/source/zh/optimization/habana.md @@ -18,7 +18,7 @@ Diffusers 管道可以利用 HPU 加速，即使管道尚未添加到 [Optimum f import torch from diffusers import DiffusionPipeline -pipeline = DiffusionPipeline.from_pretrained("black-forest-labs/FLUX.1-schnell", torch_dtype=torch.bfloat16) +pipeline = DiffusionPipeline.from_pretrained("black-forest-labs/FLUX.1-schnell", dtype=torch.bfloat16) pipeline.to("hpu") image = pipeline("一张松鼠在毕加索风格中的图像").images[0] diff --git a/docs/source/zh/optimization/memory.md b/docs/source/zh/optimization/memory.md index 7f08f63ff1d7..0d2d91827f4a 100644 --- a/docs/source/zh/optimization/memory.md +++ b/docs/source/zh/optimization/memory.md @@ -46,12 +46,12 @@ import torch from diffusers import AutoModel, StableDiffusionXLPipeline unet = AutoModel.from_pretrained( - "username/sdxl-unet-sharded", torch_dtype=torch.float16 + "username/sdxl-unet-sharded", dtype=torch.float16 ) pipeline = StableDiffusionXLPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", unet=unet, - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") ``` @@ -74,7 +74,7 @@ from diffusers import AutoModel, StableDiffusionXLPipeline pipeline = StableDiffusionXLPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16, + dtype=torch.float16, device_map="balanced" ) ``` @@ -99,7 +99,7 @@ transformer = AutoModel.from_pretrained( "black-forest-labs/FLUX.1-dev", subfolder="transformer", device_map="auto", - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) ``` @@ -135,7 +135,7 @@ transformer = AutoModel.from_pretrained( "black-forest-labs/FLUX.1-dev", subfolder="transformer", device_map=device_map, - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) ``` @@ -148,7 +148,7 @@ from diffusers import AutoModel, StableDiffusionXLPipeline max_memory = {0:"1GB", 1:"1GB"} pipeline = StableDiffusionXLPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16, + dtype=torch.float16, device_map="balanced", max_memory=max_memory ) @@ -179,7 +179,7 @@ from diffusers import AutoModel, StableDiffusionXLPipeline pipeline = StableDiffusionXLPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16, + dtype=torch.float16, ).to("cuda") pipeline.vae.enable_slicing() pipeline(["An astronaut riding a horse on Mars"]*32).images[0] @@ -201,7 +201,7 @@ from diffusers import AutoPipelineForImage2Image from diffusers.utils import load_image pipeline = AutoPipelineForImage2Image.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16 + "stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16 ).to("cuda") pipeline.vae.enable_tiling() @@ -237,7 +237,7 @@ import torch from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( - "black-forest-labs/FLUX.1-schnell", torch_dtype=torch.bfloat16 + "black-forest-labs/FLUX.1-schnell", dtype=torch.bfloat16 ) pipeline.enable_sequential_cpu_offload() @@ -266,7 +266,7 @@ import torch from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( - "black-forest-labs/FLUX.1-schnell", torch_dtype=torch.bfloat16 + "black-forest-labs/FLUX.1-schnell", dtype=torch.bfloat16 ) pipeline.enable_model_cpu_offload() @@ -305,7 +305,7 @@ from diffusers.utils import export_to_video onload_device = torch.device("cuda") offload_device = torch.device("cpu") -pipeline = CogVideoXPipeline.from_pretrained("THUDM/CogVideoX-5b", torch_dtype=torch.bfloat16) +pipeline = CogVideoXPipeline.from_pretrained("THUDM/CogVideoX-5b", dtype=torch.bfloat16) # 对 Diffusers 模型实现使用 enable_group_offload 方法 pipeline.transformer.enable_group_offload(onload_device=onload_device, offload_device=offload_device, offload_type="leaf_level") @@ -378,13 +378,13 @@ from diffusers.utils import export_to_video transformer = CogVideoXTransformer3DModel.from_pretrained( "THUDM/CogVideoX-5b", subfolder="transformer", - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) transformer.enable_layerwise_casting(storage_dtype=torch.float8_e4m3fn, compute_dtype=torch.bfloat16) pipeline = CogVideoXPipeline.from_pretrained("THUDM/CogVideoX-5b", transformer=transformer, - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ).to("cuda") prompt = ( "A panda, dressed in a small, red jacket and a tiny hat, sits on a wooden stool in a serene bamboo forest. " @@ -409,7 +409,7 @@ from diffusers.hooks import apply_layerwise_casting transformer = CogVideoXTransformer3DModel.from_pretrained( "THUDM/CogVideoX-5b", subfolder="transformer", - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) # 跳过归一化层 @@ -463,7 +463,7 @@ def generate_inputs(): pipeline = StableDiffusionPipeline.from_pretrained( "stable-diffusion-v1-5/stable-diffusion-v1-5", - torch_dtype=torch.float16, + dtype=torch.float16, use_safetensors=True, ).to("cuda") unet = pipeline.unet @@ -524,7 +524,7 @@ class UNet2DConditionOutput: pipeline = StableDiffusionPipeline.from_pretrained( "stable-diffusion-v1-5/stable-diffusion-v1-5", - torch_dtype=torch.float16, + dtype=torch.float16, use_safetensors=True, ).to("cuda") @@ -569,7 +569,7 @@ from diffusers import StableDiffusionXLPipeline pipeline = StableDiffusionXLPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16, + dtype=torch.float16, ).to("cuda") pipeline.enable_xformers_memory_efficient_attention() ``` diff --git a/docs/source/zh/optimization/mps.md b/docs/source/zh/optimization/mps.md index 48b08c5a12df..2fb2151cd6f8 100644 --- a/docs/source/zh/optimization/mps.md +++ b/docs/source/zh/optimization/mps.md @@ -68,7 +68,7 @@ M1/M2 性能对内存压力非常敏感。当发生这种情况时，系统会 from diffusers import DiffusionPipeline import torch -pipeline = DiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16, variant="fp16", use_safetensors=True).to("mps") +pipeline = DiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16, variant="fp16", use_safetensors=True).to("mps") pipeline.enable_attention_slicing() ``` diff --git a/docs/source/zh/optimization/para_attn.md b/docs/source/zh/optimization/para_attn.md index 106a8818c643..70f8363bb8ec 100644 --- a/docs/source/zh/optimization/para_attn.md +++ b/docs/source/zh/optimization/para_attn.md @@ -44,7 +44,7 @@ from diffusers import FluxPipeline pipe = FluxPipeline.from_pretrained( "black-forest-labs/FLUX.1-dev", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ).to("cuda") from para_attn.first_block_cache.diffusers_adapters import apply_cache_on_pipe @@ -90,13 +90,13 @@ model_id = "tencent/HunyuanVideo" transformer = HunyuanVideoTransformer3DModel.from_pretrained( model_id, subfolder="transformer", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, revision="refs/pr/18", ) pipe = HunyuanVideoPipeline.from_pretrained( model_id, transformer=transformer, - torch_dtype=torch.float16, + dtype=torch.float16, revision="refs/pr/18", ).to("cuda") @@ -171,7 +171,7 @@ from diffusers import FluxPipeline pipe = FluxPipeline.from_pretrained( "black-forest-labs/FLUX.1-dev", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ).to("cuda") from para_attn.first_block_cache.diffusers_adapters import apply_cache_on_pipe @@ -223,13 +223,13 @@ model_id = "tencent/HunyuanVideo" transformer = HunyuanVideoTransformer3DModel.from_pretrained( model_id, subfolder="transformer", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, revision="refs/pr/18", ) pipe = HunyuanVideoPipeline.from_pretrained( model_id, transformer=transformer, - torch_dtype=torch.float16, + dtype=torch.float16, revision="refs/pr/18", ).to("cuda") @@ -302,7 +302,7 @@ torch.cuda.set_device(dist.get_rank()) pipe = FluxPipeline.from_pretrained( "black-forest-labs/FLUX.1-dev", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ).to("cuda") from para_attn.context_parallel import init_context_parallel_mesh @@ -389,13 +389,13 @@ model_id = "tencent/HunyuanVideo" transformer = HunyuanVideoTransformer3DModel.from_pretrained( model_id, subfolder="transformer", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, revision="refs/pr/18", ) pipe = HunyuanVideoPipeline.from_pretrained( model_id, transformer=transformer, - torch_dtype=torch.float16, + dtype=torch.float16, revision="refs/pr/18", ).to("cuda") diff --git a/docs/source/zh/optimization/pruna.md b/docs/source/zh/optimization/pruna.md index 31cc3d52fa25..22f470df9977 100644 --- a/docs/source/zh/optimization/pruna.md +++ b/docs/source/zh/optimization/pruna.md @@ -56,7 +56,7 @@ from pruna import PrunaModel, SmashConfig, smash # 使用小GPU内存尝试segmind/Segmind-Vega或black-forest-labs/FLUX.1-schnell pipe = FluxPipeline.from_pretrained( "black-forest-labs/FLUX.1-dev", - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ).to("cuda") # 定义配置 @@ -163,7 +163,7 @@ from pruna import PrunaModel # 使用小GPU内存尝试 PrunaAI/Segmind-Vega-smashed 或 PrunaAI/FLUX.1-dev-smashed pipe = FluxPipeline.from_pretrained( "black-forest-labs/FLUX.1-dev", - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ).to("cpu") wrapped_pipe = PrunaModel(model=pipe) ``` diff --git a/docs/source/zh/optimization/speed-memory-optims.md b/docs/source/zh/optimization/speed-memory-optims.md index 48f1483d3e94..666ae863c0e1 100644 --- a/docs/source/zh/optimization/speed-memory-optims.md +++ b/docs/source/zh/optimization/speed-memory-optims.md @@ -56,7 +56,7 @@ pipeline_quant_config = PipelineQuantizationConfig( pipeline = DiffusionPipeline.from_pretrained( "black-forest-labs/FLUX.1-dev", quantization_config=pipeline_quant_config, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ).to("cuda") # 编译 @@ -97,7 +97,7 @@ pipeline_quant_config = PipelineQuantizationConfig( pipeline = DiffusionPipeline.from_pretrained( "black-forest-labs/FLUX.1-dev", quantization_config=pipeline_quant_config, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ).to("cuda") # 模型 CPU 卸载 @@ -137,12 +137,12 @@ pipeline_quant_config = PipelineQuantizationConfig( ) text_encoder = UMT5EncoderModel.from_pretrained( - "Wan-AI/Wan2.1-T2V-14B-Diffusers", subfolder="text_encoder", torch_dtype=torch.bfloat16 + "Wan-AI/Wan2.1-T2V-14B-Diffusers", subfolder="text_encoder", dtype=torch.bfloat16 ) pipeline = DiffusionPipeline.from_pretrained( "Wan-AI/Wan2.1-T2V-14B-Diffusers", quantization_config=pipeline_quant_config, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ).to("cuda") # 组卸载 diff --git a/docs/source/zh/optimization/tgate.md b/docs/source/zh/optimization/tgate.md index f15b9bde8413..cf62b23648e5 100644 --- a/docs/source/zh/optimization/tgate.md +++ b/docs/source/zh/optimization/tgate.md @@ -33,7 +33,7 @@ import torch from diffusers import PixArtAlphaPipeline from tgate import TgatePixArtLoader -pipe = PixArtAlphaPipeline.from_pretrained("PixArt-alpha/PixArt-XL-2-1024-MS", torch_dtype=torch.float16) +pipe = PixArtAlphaPipeline.from_pretrained("PixArt-alpha/PixArt-XL-2-1024-MS", dtype=torch.float16) gate_step = 8 inference_step = 25 @@ -62,7 +62,7 @@ from tgate import TgateSDXLLoader pipe = StableDiffusionXLPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16, + dtype=torch.float16, variant="fp16", use_safetensors=True, ) @@ -96,7 +96,7 @@ from tgate import TgateSDXLDeepCacheLoader pipe = StableDiffusionXLPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16, + dtype=torch.float16, variant="fp16", use_safetensors=True, ) @@ -130,13 +130,13 @@ from tgate import TgateSDXLLoader unet = UNet2DConditionModel.from_pretrained( "latent-consistency/lcm-sdxl", - torch_dtype=torch.float16, + dtype=torch.float16, variant="fp16", ) pipe = StableDiffusionXLPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", unet=unet, - torch_dtype=torch.float16, + dtype=torch.float16, variant="fp16", ) pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config) diff --git a/docs/source/zh/optimization/tome.md b/docs/source/zh/optimization/tome.md index 732777c5586c..bf9297529e5e 100644 --- a/docs/source/zh/optimization/tome.md +++ b/docs/source/zh/optimization/tome.md @@ -26,7 +26,7 @@ pip install tomesd import tomesd pipeline = StableDiffusionPipeline.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16, use_safetensors=True, + "stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16, use_safetensors=True, ).to("cuda") + tomesd.apply_patch(pipeline, ratio=0.5) diff --git a/docs/source/zh/optimization/xdit.md b/docs/source/zh/optimization/xdit.md index 3308536d06c1..79ca2155128b 100644 --- a/docs/source/zh/optimization/xdit.md +++ b/docs/source/zh/optimization/xdit.md @@ -36,7 +36,7 @@ pip install xfuser local_rank = get_world_group().local_rank pipe = StableDiffusion3Pipeline.from_pretrained( pretrained_model_name_or_path=engine_config.model_config.model, - torch_dtype=torch.float16, + dtype=torch.float16, ).to(f"cuda:{local_rank}") # 在这里对管道进行任何操作 diff --git a/docs/source/zh/stable_diffusion.md b/docs/source/zh/stable_diffusion.md index d337fb41a0ad..f20a784cf910 100644 --- a/docs/source/zh/stable_diffusion.md +++ b/docs/source/zh/stable_diffusion.md @@ -75,7 +75,7 @@ image ```python import torch -pipeline = DiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16, use_safetensors=True) +pipeline = DiffusionPipeline.from_pretrained(model_id, dtype=torch.float16, use_safetensors=True) pipeline = pipeline.to("cuda") generator = torch.Generator("cuda").manual_seed(0) image = pipeline(prompt, generator=generator).images[0] @@ -195,7 +195,7 @@ make_image_grid(images, rows=2, cols=4) ```python from diffusers import AutoencoderKL -vae = AutoencoderKL.from_pretrained("stabilityai/sd-vae-ft-mse", torch_dtype=torch.float16).to("cuda") +vae = AutoencoderKL.from_pretrained("stabilityai/sd-vae-ft-mse", dtype=torch.float16).to("cuda") pipeline.vae = vae images = pipeline(**get_inputs(batch_size=8)).images make_image_grid(images, rows=2, cols=4) diff --git a/docs/source/zh/training/controlnet.md b/docs/source/zh/training/controlnet.md index bab0bbcf7b52..cd2a9263314b 100644 --- a/docs/source/zh/training/controlnet.md +++ b/docs/source/zh/training/controlnet.md @@ -262,9 +262,9 @@ from diffusers import StableDiffusionControlNetPipeline, ControlNetModel from diffusers.utils import load_image import torch -controlnet = ControlNetModel.from_pretrained("path/to/controlnet", torch_dtype=torch.float16) +controlnet = ControlNetModel.from_pretrained("path/to/controlnet", dtype=torch.float16) pipeline = StableDiffusionControlNetPipeline.from_pretrained( - "path/to/base/model", controlnet=controlnet, torch_dtype=torch.float16 + "path/to/base/model", controlnet=controlnet, dtype=torch.float16 ).to("cuda") control_image = load_image("./conditioning_image_1.png") diff --git a/docs/source/zh/training/distributed_inference.md b/docs/source/zh/training/distributed_inference.md index 60297371d6be..b0693e10ab38 100644 --- a/docs/source/zh/training/distributed_inference.md +++ b/docs/source/zh/training/distributed_inference.md @@ -27,7 +27,7 @@ from accelerate import PartialState from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16, use_safetensors=True + "stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16, use_safetensors=True ) distributed_state = PartialState() pipeline.to(distributed_state.device) @@ -61,7 +61,7 @@ import torch.multiprocessing as mp from diffusers import DiffusionPipeline sd = DiffusionPipeline.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16, use_safetensors=True + "stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16, use_safetensors=True ) ``` @@ -128,7 +128,7 @@ pipeline = FluxPipeline.from_pretrained( vae=None, device_map="balanced", max_memory={0: "16GB", 1: "16GB"}, - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) with torch.no_grad(): print("Encoding prompts.") @@ -167,7 +167,7 @@ transformer = AutoModel.from_pretrained( "black-forest-labs/FLUX.1-dev", subfolder="transformer", device_map="auto", - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) ``` @@ -185,7 +185,7 @@ pipeline = FluxPipeline.from_pretrained( tokenizer_2=None, vae=None, transformer=transformer, - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) print("Running denoising.") @@ -219,7 +219,7 @@ from diffusers import AutoencoderKL from diffusers.image_processor import VaeImageProcessor import torch -vae = AutoencoderKL.from_pretrained(ckpt_id, subfolder="vae", torch_dtype=torch.bfloat16).to("cuda") +vae = AutoencoderKL.from_pretrained(ckpt_id, subfolder="vae", dtype=torch.bfloat16).to("cuda") vae_scale_factor = 2 ** (len(vae.config.block_out_channels) - 1) image_processor = VaeImageProcessor(vae_scale_factor=vae_scale_factor) diff --git a/docs/source/zh/training/dreambooth.md b/docs/source/zh/training/dreambooth.md index 00f94e46e8e1..bba8cbfd3f60 100644 --- a/docs/source/zh/training/dreambooth.md +++ b/docs/source/zh/training/dreambooth.md @@ -350,7 +350,7 @@ accelerate launch train_dreambooth.py \ from diffusers import DiffusionPipeline import torch -pipeline = DiffusionPipeline.from_pretrained("path_to_saved_model", torch_dtype=torch.float16, use_safetensors=True).to("cuda") +pipeline = DiffusionPipeline.from_pretrained("path_to_saved_model", dtype=torch.float16, use_safetensors=True).to("cuda") image = pipeline("A photo of sks dog in a bucket", num_inference_steps=50, guidance_scale=7.5).images[0] image.save("dog-bucket.png") ``` diff --git a/docs/source/zh/training/instructpix2pix.md b/docs/source/zh/training/instructpix2pix.md index 1f9f4eb21ec3..89cb7444c4e9 100644 --- a/docs/source/zh/training/instructpix2pix.md +++ b/docs/source/zh/training/instructpix2pix.md @@ -210,7 +210,7 @@ import torch from diffusers import StableDiffusionInstructPix2PixPipeline from diffusers.utils import load_image -pipeline = StableDiffusionInstructPix2PixPipeline.from_pretrained("your_cool_model", torch_dtype=torch.float16).to("cuda") +pipeline = StableDiffusionInstructPix2PixPipeline.from_pretrained("your_cool_model", dtype=torch.float16).to("cuda") generator = torch.Generator("cuda").manual_seed(0) image = load_image("https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/test_pix2pix_4.png") diff --git a/docs/source/zh/training/kandinsky.md b/docs/source/zh/training/kandinsky.md index 94fa5020efab..62ae4537b61b 100644 --- a/docs/source/zh/training/kandinsky.md +++ b/docs/source/zh/training/kandinsky.md @@ -107,10 +107,10 @@ tokenizer = CLIPTokenizer.from_pretrained(args.pretrained_prior_model_name_or_pa with ContextManagers(deepspeed_zero_init_disabled_context_manager()): image_encoder = CLIPVisionModelWithProjection.from_pretrained( - args.pretrained_prior_model_name_or_path, subfolder="image_encoder", torch_dtype=weight_dtype + args.pretrained_prior_model_name_or_path, subfolder="image_encoder", dtype=weight_dtype ).eval() text_encoder = CLIPTextModelWithProjection.from_pretrained( - args.pretrained_prior_model_name_or_path, subfolder="text_encoder", torch_dtype=weight_dtype + args.pretrained_prior_model_name_or_path, subfolder="text_encoder", dtype=weight_dtype ).eval() ``` @@ -163,10 +163,10 @@ ffusers/blob/6e68c71503682c8693cb5b06a4da4911dfd655ee/examples/kandinsky2_2/text ```py with ContextManagers(deepspeed_zero_init_disabled_context_manager()): vae = VQModel.from_pretrained( - args.pretrained_decoder_model_name_or_path, subfolder="movq", torch_dtype=weight_dtype + args.pretrained_decoder_model_name_or_path, subfolder="movq", dtype=weight_dtype ).eval() image_encoder = CLIPVisionModelWithProjection.from_pretrained( - args.pretrained_prior_model_name_or_path, subfolder="image_encoder", torch_dtype=weight_dtype + args.pretrained_prior_model_name_or_path, subfolder="image_encoder", dtype=weight_dtype ).eval() unet = UNet2DConditionModel.from_pretrained(args.pretrained_decoder_model_name_or_path, subfolder="unet") ``` @@ -263,9 +263,9 @@ accelerate launch --mixed_precision="fp16" train_text_to_image_decoder.py \ from diffusers import AutoPipelineForText2Image, DiffusionPipeline import torch -prior_pipeline = DiffusionPipeline.from_pretrained(output_dir, torch_dtype=torch.float16) +prior_pipeline = DiffusionPipeline.from_pretrained(output_dir, dtype=torch.float16) prior_components = {"prior_" + k: v for k,v in prior_pipeline.components.items()} -pipeline = AutoPipelineForText2Image.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", **prior_components, torch_dtype=torch.float16) +pipeline = AutoPipelineForText2Image.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", **prior_components, dtype=torch.float16) pipe.enable_model_cpu_offload() prompt="A robot naruto, 4k photo" @@ -282,7 +282,7 @@ image = pipeline(prompt=prompt, negative_prompt=negative_prompt).images[0] from diffusers import AutoPipelineForText2Image import torch -pipeline = AutoPipelineForText2Image.from_pretrained("path/to/saved/model", torch_dtype=torch.float16) +pipeline = AutoPipelineForText2Image.from_pretrained("path/to/saved/model", dtype=torch.float16) pipeline.enable_model_cpu_offload() prompt="A robot naruto, 4k photo" @@ -296,7 +296,7 @@ from diffusers import AutoPipelineForText2Image, UNet2DConditionModel unet = UNet2DConditionModel.from_pretrained("path/to/saved/model" + "/checkpoint-/unet") -pipeline = AutoPipelineForText2Image.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", unet=unet, torch_dtype=torch.float16) +pipeline = AutoPipelineForText2Image.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", unet=unet, dtype=torch.float16) pipeline.enable_model_cpu_offload() image = pipeline(prompt="A robot naruto, 4k photo").images[0] diff --git a/docs/source/zh/training/lora.md b/docs/source/zh/training/lora.md index 85bd195be27f..f80ae79f7503 100644 --- a/docs/source/zh/training/lora.md +++ b/docs/source/zh/training/lora.md @@ -195,7 +195,7 @@ accelerate launch --mixed_precision="fp16" train_text_to_image_lora.py \ from diffusers import AutoPipelineForText2Image import torch -pipeline = AutoPipelineForText2Image.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16).to("cuda") +pipeline = AutoPipelineForText2Image.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16).to("cuda") pipeline.load_lora_weights("path/to/lora/model", weight_name="pytorch_lora_weights.safetensors") image = pipeline("A naruto with blue eyes").images[0] ``` diff --git a/docs/source/zh/training/text2image.md b/docs/source/zh/training/text2image.md index d96f0cec200c..695ce4ddd58f 100644 --- a/docs/source/zh/training/text2image.md +++ b/docs/source/zh/training/text2image.md @@ -181,7 +181,7 @@ accelerate launch --mixed_precision="fp16" train_text_to_image.py \ from diffusers import StableDiffusionPipeline import torch -pipeline = StableDiffusionPipeline.from_pretrained("path/to/saved_model", torch_dtype=torch.float16, use_safetensors=True).to("cuda") +pipeline = StableDiffusionPipeline.from_pretrained("path/to/saved_model", dtype=torch.float16, use_safetensors=True).to("cuda") image = pipeline(prompt="yoda").images[0] image.save("yoda-naruto.png") diff --git a/docs/source/zh/training/text_inversion.md b/docs/source/zh/training/text_inversion.md index d41b0d5ed039..150b76d19d26 100644 --- a/docs/source/zh/training/text_inversion.md +++ b/docs/source/zh/training/text_inversion.md @@ -207,7 +207,7 @@ accelerate launch textual_inversion.py \ from diffusers import StableDiffusionPipeline import torch -pipeline = StableDiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16).to("cuda") +pipeline = StableDiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16).to("cuda") pipeline.load_textual_inversion("sd-concepts-library/cat-toy") image = pipeline("A train", num_inference_steps=50).images[0] image.save("cat-train.png") diff --git a/docs/source/zh/training/wuerstchen.md b/docs/source/zh/training/wuerstchen.md index c80cc944a3d8..393a6c50157e 100644 --- a/docs/source/zh/training/wuerstchen.md +++ b/docs/source/zh/training/wuerstchen.md @@ -161,7 +161,7 @@ import torch from diffusers import AutoPipelineForText2Image from diffusers.pipelines.wuerstchen import DEFAULT_STAGE_C_TIMESTEPS -pipeline = AutoPipelineForText2Image.from_pretrained("path/to/saved/model", torch_dtype=torch.float16).to("cuda") +pipeline = AutoPipelineForText2Image.from_pretrained("path/to/saved/model", dtype=torch.float16).to("cuda") caption = "A cute bird naruto holding a shield" images = pipeline( diff --git a/docs/source/zh/tutorials/autopipeline.md b/docs/source/zh/tutorials/autopipeline.md index 27438fc54884..69c119e3f971 100644 --- a/docs/source/zh/tutorials/autopipeline.md +++ b/docs/source/zh/tutorials/autopipeline.md @@ -23,7 +23,7 @@ import torch from diffusers import AutoPipelineForImage2Image pipeline = AutoPipelineForImage2Image.from_pretrained( - "RunDiffusion/Juggernaut-XL-v9", torch_dtype=torch.bfloat16, device_map="cuda", + "RunDiffusion/Juggernaut-XL-v9", dtype=torch.bfloat16, device_map="cuda", ) print(pipeline) "StableDiffusionXLImg2ImgPipeline { @@ -39,7 +39,7 @@ import torch from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( - "RunDiffusion/Juggernaut-XL-v9", torch_dtype=torch.bfloat16, device_map="cuda", + "RunDiffusion/Juggernaut-XL-v9", dtype=torch.bfloat16, device_map="cuda", ) print(pipeline) "StableDiffusionXLPipeline { @@ -57,7 +57,7 @@ import torch from diffusers import AutoPipelineForImage2Image pipeline = AutoPipelineForImage2Image.from_pretrained( - "openai/shap-e-img2img", torch_dtype=torch.float16, + "openai/shap-e-img2img", dtype=torch.float16, ) "ValueError: AutoPipeline can't find a pipeline linked to ShapEImg2ImgPipeline for None" ``` diff --git a/docs/source/zh/tutorials/using_peft_for_inference.md b/docs/source/zh/tutorials/using_peft_for_inference.md index fa9b2099bcb1..1531c4e95560 100644 --- a/docs/source/zh/tutorials/using_peft_for_inference.md +++ b/docs/source/zh/tutorials/using_peft_for_inference.md @@ -25,7 +25,7 @@ from diffusers import AutoPipelineForText2Image pipeline = AutoPipelineForText2Image.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.load_lora_weights( "ostris/super-cereal-sdxl-lora", @@ -44,7 +44,7 @@ from diffusers import LTXConditionPipeline from diffusers.utils import export_to_video, load_image pipeline = LTXConditionPipeline.from_pretrained( - "Lightricks/LTX-Video-0.9.5", torch_dtype=torch.bfloat16 + "Lightricks/LTX-Video-0.9.5", dtype=torch.bfloat16 ) pipeline.load_lora_weights( @@ -89,7 +89,7 @@ from diffusers import AutoPipelineForText2Image pipeline = AutoPipelineForText2Image.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.unet.load_lora_adapter( "jbilcke-hf/sdxl-cinematic-1", @@ -112,7 +112,7 @@ from diffusers import DiffusionPipeline # 加载基础模型和 LoRA pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.load_lora_weights( "ostris/ikea-instructions-lora-sdxl", @@ -154,7 +154,7 @@ from diffusers import AutoPipelineForText2Image pipeline = AutoPipelineForText2Image.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.load_lora_weights( "ostris/super-cereal-sdxl-lora", @@ -178,7 +178,7 @@ from diffusers import AutoPipelineForText2Image pipeline = AutoPipelineForText2Image.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.load_lora_weights( "ostris/super-cereal-sdxl-lora", @@ -214,7 +214,7 @@ import torch from diffusers import FluxPipeline pipeline = FluxPipeline.from_pretrained( - "black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16 + "black-forest-labs/FLUX.1-dev", dtype=torch.bfloat16 ).to("cuda") pipelne.load_lora_weights("alvarobartt/ghibli-characters-flux-lora", "lora") @@ -254,7 +254,7 @@ from diffusers import DiffusionPipeline # 加载基础模型和 LoRA pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.load_lora_weights( "ostris/ikea-instructions-lora-sdxl", @@ -292,7 +292,7 @@ from diffusers import DiffusionPipeline # 加载基础模型和 LoRA pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") # 1. 启用 enable_lora_hotswap pipeline.enable_lora_hotswap(target_rank=max_rank) @@ -349,7 +349,7 @@ from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.load_lora_weights( "ostris/ikea-instructions-lora-sdxl", @@ -393,7 +393,7 @@ from peft import get_peft_model, LoraConfig, PeftModel unet = AutoModel.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16, + dtype=torch.float16, use_safetensors=True, variant="fp16", subfolder="unet", @@ -406,7 +406,7 @@ unet = AutoModel.from_pretrained( pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", variant="fp16", - torch_dtype=torch.float16, + dtype=torch.float16, unet=unet ).to("cuda") pipeline.load_lora_weights( @@ -464,7 +464,7 @@ feng_peft_model.load_state_dict(original_state_dict, strict=True) ```py base_unet = AutoModel.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16, + dtype=torch.float16, use_safetensors=True, variant="fp16", subfolder="unet", @@ -502,7 +502,7 @@ pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", unet=model, variant="fp16", - torch_dtype=torch.float16, + dtype=torch.float16, ).to("cuda") pipeline("A bowl of ramen shaped like a cute kawaii bear, by Feng Zikai").images[0] ``` @@ -521,7 +521,7 @@ from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.load_lora_weights( "ostris/ikea-instructions-lora-sdxl", @@ -567,7 +567,7 @@ pipeline.push_to_hub("fused-ikea-feng") ```py pipeline = DiffusionPipeline.from_pretrained( - "username/fused-ikea-feng", torch_dtype=torch.float16, + "username/fused-ikea-feng", dtype=torch.float16, ).to("cuda") pipeline("A bowl of ramen shaped like a cute kawaii bear, by Feng Zikai").images[0] ``` @@ -596,7 +596,7 @@ from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.load_lora_weights( "ostris/ikea-instructions-lora-sdxl", @@ -622,7 +622,7 @@ from diffusers import AutoPipelineForText2Image pipeline = AutoPipelineForText2Image.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16 + dtype=torch.float16 ).to("cuda") pipeline.unet.load_lora_adapter( "jbilcke-hf/sdxl-cinematic-1", diff --git a/docs/source/zh/using-diffusers/anyflow.md b/docs/source/zh/using-diffusers/anyflow.md index e9c925a85256..5900d8cfbaca 100644 --- a/docs/source/zh/using-diffusers/anyflow.md +++ b/docs/source/zh/using-diffusers/anyflow.md @@ -59,22 +59,22 @@ from diffusers import AnyFlowPipeline, AnyFlowFARPipeline # Bidirectional, 轻量 pipe = AnyFlowPipeline.from_pretrained( - "nvidia/AnyFlow-Wan2.1-T2V-1.3B-Diffusers", torch_dtype=torch.bfloat16 + "nvidia/AnyFlow-Wan2.1-T2V-1.3B-Diffusers", dtype=torch.bfloat16 ).to("cuda") # Bidirectional, 满血 pipe = AnyFlowPipeline.from_pretrained( - "nvidia/AnyFlow-Wan2.1-T2V-14B-Diffusers", torch_dtype=torch.bfloat16 + "nvidia/AnyFlow-Wan2.1-T2V-14B-Diffusers", dtype=torch.bfloat16 ).to("cuda") # Causal (FAR), 1.3B pipe = AnyFlowFARPipeline.from_pretrained( - "nvidia/AnyFlow-FAR-Wan2.1-1.3B-Diffusers", torch_dtype=torch.bfloat16 + "nvidia/AnyFlow-FAR-Wan2.1-1.3B-Diffusers", dtype=torch.bfloat16 ).to("cuda") # Causal (FAR), 14B pipe = AnyFlowFARPipeline.from_pretrained( - "nvidia/AnyFlow-FAR-Wan2.1-14B-Diffusers", torch_dtype=torch.bfloat16 + "nvidia/AnyFlow-FAR-Wan2.1-14B-Diffusers", dtype=torch.bfloat16 ).to("cuda") ``` @@ -92,7 +92,7 @@ from diffusers import AnyFlowPipeline from diffusers.utils import export_to_video pipe = AnyFlowPipeline.from_pretrained( - "nvidia/AnyFlow-Wan2.1-T2V-1.3B-Diffusers", torch_dtype=torch.bfloat16 + "nvidia/AnyFlow-Wan2.1-T2V-1.3B-Diffusers", dtype=torch.bfloat16 ).to("cuda") prompt = "森林里一只小熊猫在啃竹子，电影感光照" @@ -138,7 +138,7 @@ from diffusers import AnyFlowFARPipeline from diffusers.utils import export_to_video, load_image, load_video pipe = AnyFlowFARPipeline.from_pretrained( - "nvidia/AnyFlow-FAR-Wan2.1-1.3B-Diffusers", torch_dtype=torch.bfloat16 + "nvidia/AnyFlow-FAR-Wan2.1-1.3B-Diffusers", dtype=torch.bfloat16 ).to("cuda") diff --git a/docs/source/zh/using-diffusers/consisid.md b/docs/source/zh/using-diffusers/consisid.md index 018c5e706fb7..0a9585b75f01 100644 --- a/docs/source/zh/using-diffusers/consisid.md +++ b/docs/source/zh/using-diffusers/consisid.md @@ -41,7 +41,7 @@ snapshot_download(repo_id="BestWishYsh/ConsisID-preview", local_dir="BestWishYsh face_helper_1, face_helper_2, face_clip_model, face_main_model, eva_transform_mean, eva_transform_std = prepare_face_models("BestWishYsh/ConsisID-preview", device="cuda", dtype=torch.bfloat16) # Load consisid base model -pipe = ConsisIDPipeline.from_pretrained("BestWishYsh/ConsisID-preview", torch_dtype=torch.bfloat16) +pipe = ConsisIDPipeline.from_pretrained("BestWishYsh/ConsisID-preview", dtype=torch.bfloat16) pipe.to("cuda") ``` diff --git a/docs/source/zh/using-diffusers/helios.md b/docs/source/zh/using-diffusers/helios.md index 5f7f067eb781..93caff90d9c6 100644 --- a/docs/source/zh/using-diffusers/helios.md +++ b/docs/source/zh/using-diffusers/helios.md @@ -30,17 +30,17 @@ from huggingface_hub import snapshot_download # For Best Quality snapshot_download(repo_id="BestWishYsh/Helios-Base", local_dir="BestWishYsh/Helios-Base") -pipe = HeliosPipeline.from_pretrained("BestWishYsh/Helios-Base", torch_dtype=torch.bfloat16) +pipe = HeliosPipeline.from_pretrained("BestWishYsh/Helios-Base", dtype=torch.bfloat16) pipe.to("cuda") # Intermediate Weight snapshot_download(repo_id="BestWishYsh/Helios-Mid", local_dir="BestWishYsh/Helios-Mid") -pipe = HeliosPyramidPipeline.from_pretrained("BestWishYsh/Helios-Mid", torch_dtype=torch.bfloat16) +pipe = HeliosPyramidPipeline.from_pretrained("BestWishYsh/Helios-Mid", dtype=torch.bfloat16) pipe.to("cuda") # For Best Efficiency snapshot_download(repo_id="BestWishYsh/Helios-Distilled", local_dir="BestWishYsh/Helios-Distilled") -pipe = HeliosPyramidPipeline.from_pretrained("BestWishYsh/Helios-Distilled", torch_dtype=torch.bfloat16) +pipe = HeliosPyramidPipeline.from_pretrained("BestWishYsh/Helios-Distilled", dtype=torch.bfloat16) pipe.to("cuda") ``` diff --git a/docs/source/zh/using-diffusers/schedulers.md b/docs/source/zh/using-diffusers/schedulers.md index d71e4150c165..615a2f024f7a 100644 --- a/docs/source/zh/using-diffusers/schedulers.md +++ b/docs/source/zh/using-diffusers/schedulers.md @@ -23,7 +23,7 @@ import torch from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16, use_safetensors=True + "stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16, use_safetensors=True ).to("cuda") ``` @@ -63,7 +63,7 @@ ddim = DDIMScheduler.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1- ```python pipeline = DiffusionPipeline.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-v1-5", scheduler=ddim, torch_dtype=torch.float16, use_safetensors=True + "stable-diffusion-v1-5/stable-diffusion-v1-5", scheduler=ddim, dtype=torch.float16, use_safetensors=True ).to("cuda") ``` @@ -78,7 +78,7 @@ import torch from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( - "stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16, use_safetensors=True + "stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16, use_safetensors=True ).to("cuda") prompt = "A photograph of an astronaut riding a horse on Mars, high resolution, high definition." @@ -196,14 +196,14 @@ unet = UNet2DConditionModel.from_pretrained( unet.save_pretrained("./local-unet", variant="non_ema") ``` -使用[`~ModelMixin.from_pretrained`]的`torch_dtype`参数指定模型加载精度： +使用[`~ModelMixin.from_pretrained`]的`dtype`参数指定模型加载精度： ```python from diffusers import AutoModel unet = AutoModel.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", subfolder="unet", torch_dtype=torch.float16 + "stabilityai/stable-diffusion-xl-base-1.0", subfolder="unet", dtype=torch.float16 ) ``` -也可使用[torch.Tensor.to](https://docs.pytorch.org/docs/stable/generated/torch.Tensor.to.html)方法即时转换精度，但会转换所有权重（不同于`torch_dtype`参数会保留`_keep_in_fp32_modules`中的层）。这对某些必须保持fp32精度的层尤为重要（参见[示例](https://github.com/huggingface/diffusers/blob/f864a9a352fa4a220d860bfdd1782e3e5af96382/src/diffusers/models/transformers/transformer_wan.py#L374)）。 +也可使用[torch.Tensor.to](https://docs.pytorch.org/docs/stable/generated/torch.Tensor.to.html)方法即时转换精度，但会转换所有权重（不同于`dtype`参数会保留`_keep_in_fp32_modules`中的层）。这对某些必须保持fp32精度的层尤为重要（参见[示例](https://github.com/huggingface/diffusers/blob/f864a9a352fa4a220d860bfdd1782e3e5af96382/src/diffusers/models/transformers/transformer_wan.py#L374)）。 diff --git a/examples/advanced_diffusion_training/README.md b/examples/advanced_diffusion_training/README.md index c9c3c1c50871..9d68bf9a825f 100644 --- a/examples/advanced_diffusion_training/README.md +++ b/examples/advanced_diffusion_training/README.md @@ -194,7 +194,7 @@ repo_id = f"{username}/3d-icon-SDXL-LoRA" pipe = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16, + dtype=torch.float16, variant="fp16", ).to("cuda") @@ -372,11 +372,11 @@ def lora_lora_unet_blocks(lora_path, alpha, target_blocks): filtered_state_dict = {k: v * alpha for k, v in state_dict.items() if is_belong_to_blocks(k, target_blocks)} return filtered_state_dict -vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16) +vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", dtype=torch.float16) pipeline = StableDiffusionXLPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", vae=vae, - torch_dtype=torch.float16, + dtype=torch.float16, ).to("cuda") # pick a blora for content/style (you can also set one to None) @@ -437,11 +437,11 @@ def lora_lora_unet_blocks(lora_path, alpha, target_blocks): filtered_state_dict = {k: v * alpha for k, v in state_dict.items() if is_belong_to_blocks(k, target_blocks)} return filtered_state_dict -vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16) +vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", dtype=torch.float16) pipeline = StableDiffusionXLPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", vae=vae, - torch_dtype=torch.float16, + dtype=torch.float16, ).to("cuda") lora_path = "lora-library/B-LoRA-pen_sketch" diff --git a/examples/advanced_diffusion_training/README_flux.md b/examples/advanced_diffusion_training/README_flux.md index 65e59ba6e7af..ae2b1d2c31b9 100644 --- a/examples/advanced_diffusion_training/README_flux.md +++ b/examples/advanced_diffusion_training/README_flux.md @@ -294,7 +294,7 @@ from safetensors.torch import load_file username = "linoyts" repo_id = f"{username}/3d-icon-Flux-LoRA" -pipe = AutoPipelineForText2Image.from_pretrained("black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16).to('cuda') +pipe = AutoPipelineForText2Image.from_pretrained("black-forest-labs/FLUX.1-dev", dtype=torch.bfloat16).to('cuda') pipe.load_lora_weights(repo_id, weight_name="pytorch_lora_weights.safetensors") @@ -342,7 +342,7 @@ from safetensors.torch import load_file username = "linoyts" repo_id = f"{username}/3d-icon-Flux-LoRA" -pipe = AutoPipelineForText2Image.from_pretrained("black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16).to('cuda') +pipe = AutoPipelineForText2Image.from_pretrained("black-forest-labs/FLUX.1-dev", dtype=torch.bfloat16).to('cuda') text_encoders = [pipe.text_encoder, pipe.text_encoder_2] tokenizers = [pipe.tokenizer, pipe.tokenizer_2] diff --git a/examples/cogvideo/README.md b/examples/cogvideo/README.md index ab0facc0a1e0..3d22b2bf9931 100644 --- a/examples/cogvideo/README.md +++ b/examples/cogvideo/README.md @@ -215,7 +215,7 @@ import torch from diffusers import CogVideoXPipeline from diffusers.utils import export_to_video -pipe = CogVideoXPipeline.from_pretrained("THUDM/CogVideoX-2b", torch_dtype=torch.float16) +pipe = CogVideoXPipeline.from_pretrained("THUDM/CogVideoX-2b", dtype=torch.float16) # pipe.load_lora_weights("/path/to/lora/weights", adapter_name="cogvideox-lora") # Or, pipe.load_lora_weights("my-awesome-hf-username/my-awesome-lora-name", adapter_name="cogvideox-lora") # If loading from the HF Hub pipe.to("cuda") diff --git a/examples/cogview4-control/README.md b/examples/cogview4-control/README.md index c73c5ed3ca14..531198b896c5 100644 --- a/examples/cogview4-control/README.md +++ b/examples/cogview4-control/README.md @@ -102,7 +102,7 @@ from PIL import Image import numpy as np import torch -pipe = CogView4ControlPipeline.from_pretrained("THUDM/CogView4-6B", torch_dtype=torch.bfloat16).to("cuda") +pipe = CogView4ControlPipeline.from_pretrained("THUDM/CogView4-6B", dtype=torch.bfloat16).to("cuda") pipe.load_lora_weights("...") # change this. open_pose = OpenposeDetector.from_pretrained("lllyasviel/Annotators") @@ -171,7 +171,7 @@ import torch transformer = CogView4Transformer2DModel.from_pretrained("...") # change this. pipe = CogView4ControlPipeline.from_pretrained( - "THUDM/CogView4-6B", transformer=transformer, torch_dtype=torch.bfloat16 + "THUDM/CogView4-6B", transformer=transformer, dtype=torch.bfloat16 ).to("cuda") open_pose = OpenposeDetector.from_pretrained("lllyasviel/Annotators") diff --git a/examples/community/README.md b/examples/community/README.md index 4ff9c4d77704..67d241c153ad 100644 --- a/examples/community/README.md +++ b/examples/community/README.md @@ -120,7 +120,7 @@ from pipeline_stg_mochi import MochiSTGPipeline from diffusers.utils import export_to_video # Load the pipeline -pipe = MochiSTGPipeline.from_pretrained("genmo/mochi-1-preview", variant="bf16", torch_dtype=torch.bfloat16) +pipe = MochiSTGPipeline.from_pretrained("genmo/mochi-1-preview", variant="bf16", dtype=torch.bfloat16) # Enable memory savings pipe = pipe.to("cuda") @@ -257,7 +257,7 @@ if __name__ == "__main__": pipeline = AdaptiveMaskInpaintPipeline.from_pretrained( "Uminosachi/realisticVisionV51_v51VAE-inpainting", scheduler=scheduler, - torch_dtype=torch.float16, + dtype=torch.float16, requires_safety_checker=False ).to(device) @@ -318,7 +318,7 @@ negative_prompt = "pink" # Load the diffusion pipeline pipeline = DiffusionPipeline.from_pretrained( model_name, - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, custom_pipeline="pipeline_flux_with_cfg" ) pipeline.enable_model_cpu_offload() @@ -363,7 +363,7 @@ from examples.community.pipeline_stable_diffusion_xl_differential_img2img import pipeline = StableDiffusionXLDifferentialImg2ImgPipeline.from_pretrained( - "SG161222/RealVisXL_V4.0", torch_dtype=torch.float16, variant="fp16" + "SG161222/RealVisXL_V4.0", dtype=torch.float16, variant="fp16" ).to("cuda") pipeline.scheduler = DPMSolverMultistepScheduler.from_config(pipeline.scheduler.config, use_karras_sigmas=True) @@ -474,16 +474,16 @@ from diffusers.utils import load_image pipe = DiffusionPipeline.from_pretrained( "prs-eth/marigold-v1-0", custom_pipeline="marigold_depth_estimation" - # torch_dtype=torch.float16, # (optional) Run with half-precision (16-bit float). - # variant="fp16", # (optional) Use with `torch_dtype=torch.float16`, to directly load fp16 checkpoint + # dtype=torch.float16, # (optional) Run with half-precision (16-bit float). + # variant="fp16", # (optional) Use with `dtype=torch.float16`, to directly load fp16 checkpoint ) # (New) LCM version (faster speed) pipe = DiffusionPipeline.from_pretrained( "prs-eth/marigold-depth-lcm-v1-0", custom_pipeline="marigold_depth_estimation" - # torch_dtype=torch.float16, # (optional) Run with half-precision (16-bit float). - # variant="fp16", # (optional) Use with `torch_dtype=torch.float16`, to directly load fp16 checkpoint + # dtype=torch.float16, # (optional) Run with half-precision (16-bit float). + # variant="fp16", # (optional) Use with `dtype=torch.float16`, to directly load fp16 checkpoint ) pipe.to("cuda") @@ -543,7 +543,7 @@ pipe = DiffusionPipeline.from_pretrained( "longlian/lmd_plus", custom_pipeline="llm_grounded_diffusion", custom_revision="main", - variant="fp16", torch_dtype=torch.float16 + variant="fp16", dtype=torch.float16 ) pipe.enable_model_cpu_offload() @@ -578,7 +578,7 @@ from diffusers import DiffusionPipeline pipe = DiffusionPipeline.from_pretrained( "longlian/lmd_plus", custom_pipeline="llm_grounded_diffusion", - variant="fp16", torch_dtype=torch.float16 + variant="fp16", dtype=torch.float16 ) pipe.enable_model_cpu_offload() @@ -615,7 +615,7 @@ import torch feature_extractor = CLIPImageProcessor.from_pretrained("laion/CLIP-ViT-B-32-laion2B-s34B-b79K") -clip_model = CLIPModel.from_pretrained("laion/CLIP-ViT-B-32-laion2B-s34B-b79K", torch_dtype=torch.float16) +clip_model = CLIPModel.from_pretrained("laion/CLIP-ViT-B-32-laion2B-s34B-b79K", dtype=torch.float16) guided_pipeline = DiffusionPipeline.from_pretrained( @@ -623,7 +623,7 @@ guided_pipeline = DiffusionPipeline.from_pretrained( custom_pipeline="clip_guided_stable_diffusion", clip_model=clip_model, feature_extractor=feature_extractor, - torch_dtype=torch.float16, + dtype=torch.float16, ) guided_pipeline.enable_attention_slicing() guided_pipeline = guided_pipeline.to("cuda") @@ -678,7 +678,7 @@ import torch pipe = DiffusionPipeline.from_pretrained( "CompVis/stable-diffusion-v1-4", variant='fp16', - torch_dtype=torch.float16, + dtype=torch.float16, safety_checker=None, # Very important for videos...lots of false positives while interpolating custom_pipeline="interpolate_stable_diffusion", ).to('cuda') @@ -718,7 +718,7 @@ def download_image(url): response = requests.get(url) return PIL.Image.open(BytesIO(response.content)).convert("RGB") -pipe = DiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", custom_pipeline="stable_diffusion_mega", torch_dtype=torch.float16, variant="fp16") +pipe = DiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", custom_pipeline="stable_diffusion_mega", dtype=torch.float16, variant="fp16") pipe.to("cuda") pipe.enable_attention_slicing() @@ -773,7 +773,7 @@ import torch pipe = DiffusionPipeline.from_pretrained( 'hakurei/waifu-diffusion', custom_pipeline="lpw_stable_diffusion", - torch_dtype=torch.float16 + dtype=torch.float16 ) pipe = pipe.to("cuda") @@ -837,7 +837,7 @@ diffuser_pipeline = DiffusionPipeline.from_pretrained( custom_pipeline="speech_to_image_diffusion", speech_model=model, speech_processor=processor, - torch_dtype=torch.float16, + dtype=torch.float16, ) diffuser_pipeline.enable_attention_slicing() @@ -896,7 +896,7 @@ import torch pipe = DiffusionPipeline.from_pretrained( "CompVis/stable-diffusion-v1-4", custom_pipeline="wildcard_stable_diffusion", - torch_dtype=torch.float16, + dtype=torch.float16, ) prompt = "__animal__ sitting on a __object__ wearing a __clothing__" out = pipe( @@ -1135,7 +1135,7 @@ diffuser_pipeline = DiffusionPipeline.from_pretrained( detection_pipeline=language_detection_pipeline, translation_model=trans_model, translation_tokenizer=trans_tokenizer, - torch_dtype=torch.float16, + dtype=torch.float16, ) diffuser_pipeline.enable_attention_slicing() @@ -1298,7 +1298,7 @@ mask_image = load_image(mask_url, mode="RGB") pipe = DiffusionPipeline.from_pretrained( "stable-diffusion-v1-5/stable-diffusion-inpainting", custom_pipeline="img2img_inpainting", - torch_dtype=torch.float16 + dtype=torch.float16 ) pipe = pipe.to("cuda") @@ -1587,7 +1587,7 @@ device = torch.device("cpu" if not torch.cuda.is_available() else "cuda") pipeline = DiffusionPipeline.from_pretrained( "kakaobrain/karlo-v1-alpha", - torch_dtype=torch.float16, + dtype=torch.float16, custom_pipeline="stable_unclip", decoder_pipe_kwargs=dict( image_encoder=None, @@ -1671,7 +1671,7 @@ device = torch.device("cpu" if not torch.cuda.is_available() else "cuda") pipe = DiffusionPipeline.from_pretrained( "kakaobrain/karlo-v1-alpha", - torch_dtype=torch.float16, + dtype=torch.float16, custom_pipeline="unclip_text_interpolation" ) pipe.to(device) @@ -1712,7 +1712,7 @@ dtype = torch.float16 if torch.cuda.is_available() else torch.bfloat16 pipe = DiffusionPipeline.from_pretrained( "kakaobrain/karlo-v1-alpha-image-variations", - torch_dtype=dtype, + dtype=dtype, custom_pipeline="unclip_image_interpolation" ) pipe.to(device) @@ -1817,7 +1817,7 @@ feature_extractor = CLIPImageProcessor.from_pretrained( "laion/CLIP-ViT-B-32-laion2B-s34B-b79K" ) clip_model = CLIPModel.from_pretrained( - "laion/CLIP-ViT-B-32-laion2B-s34B-b79K", torch_dtype=torch.float16 + "laion/CLIP-ViT-B-32-laion2B-s34B-b79K", dtype=torch.float16 ) # Load guided pipeline @@ -1826,7 +1826,7 @@ guided_pipeline = DiffusionPipeline.from_pretrained( custom_pipeline="clip_guided_stable_diffusion_img2img", clip_model=clip_model, feature_extractor=feature_extractor, - torch_dtype=torch.float16, + dtype=torch.float16, ) guided_pipeline.enable_attention_slicing() guided_pipeline = guided_pipeline.to("cuda") @@ -1884,7 +1884,7 @@ scheduler = DDIMScheduler.from_pretrained("stabilityai/stable-diffusion-2-1", su pipe = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-2-1", custom_pipeline="stable_diffusion_tensorrt_txt2img", variant='fp16', - torch_dtype=torch.float16, + dtype=torch.float16, scheduler=scheduler,) # re-use cached folder to save ONNX models and TensorRT Engines @@ -1923,7 +1923,7 @@ def center_crop_and_resize(im): return im.crop((left, upper, right, lower)).resize((512, 512)) -torch_dtype = torch.float16 +dtype = torch.float16 device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') # scheduler and text_encoder param values as in the paper @@ -1938,7 +1938,7 @@ scheduler = DDIMScheduler( text_encoder = CLIPTextModel.from_pretrained( pretrained_model_name_or_path="openai/clip-vit-large-patch14", - torch_dtype=torch_dtype, + dtype=dtype, ) # initialize pipeline @@ -1949,7 +1949,7 @@ pipeline = DiffusionPipeline.from_pretrained( scheduler=scheduler, text_encoder=text_encoder, leapfrog_steps=True, - torch_dtype=torch_dtype, + dtype=dtype, ).to(device) # download image @@ -2009,7 +2009,7 @@ init_image = download_image(img_url).resize((512, 512)) mask_image = download_image(mask_url).resize((512, 512)) mask_image = PIL.ImageOps.invert(mask_image) pipe = StableDiffusionPipeline.from_pretrained( - "CompVis/stable-diffusion-v1-4", torch_dtype=torch.float16, custom_pipeline="stable_diffusion_repaint", + "CompVis/stable-diffusion-v1-4", dtype=torch.float16, custom_pipeline="stable_diffusion_repaint", ) pipe.scheduler = RePaintScheduler.from_config(pipe.scheduler.config) pipe = pipe.to("cuda") @@ -2038,7 +2038,7 @@ scheduler = DDIMScheduler.from_pretrained("stabilityai/stable-diffusion-2-1", pipe = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-2-1", custom_pipeline="stable_diffusion_tensorrt_img2img", variant='fp16', - torch_dtype=torch.float16, + dtype=torch.float16, scheduler=scheduler,) # re-use cached folder to save ONNX models and TensorRT Engines @@ -2076,7 +2076,7 @@ def draw_box_with_text(img, boxes, names): pipe = StableDiffusionBoxDiffPipeline.from_pretrained( "stabilityai/stable-diffusion-2-1-base", - torch_dtype=torch.float16, + dtype=torch.float16, ) pipe.to("cuda") @@ -2134,7 +2134,7 @@ input_image = load_image("https://hf.co/datasets/huggingface/documentation-image pipe = StableDiffusionReferencePipeline.from_pretrained( "stable-diffusion-v1-5/stable-diffusion-v1-5", safety_checker=None, - torch_dtype=torch.float16 + dtype=torch.float16 ).to('cuda:0') pipe.scheduler = UniPCMultistepScheduler.from_config(pipe.scheduler.config) @@ -2187,12 +2187,12 @@ image = image[:, :, None] image = np.concatenate([image, image, image], axis=2) canny_image = Image.fromarray(image) -controlnet = ControlNetModel.from_pretrained("lllyasviel/sd-controlnet-canny", torch_dtype=torch.float16) +controlnet = ControlNetModel.from_pretrained("lllyasviel/sd-controlnet-canny", dtype=torch.float16) pipe = StableDiffusionControlNetReferencePipeline.from_pretrained( "stable-diffusion-v1-5/stable-diffusion-v1-5", controlnet=controlnet, safety_checker=None, - torch_dtype=torch.float16 + dtype=torch.float16 ).to('cuda:0') pipe.scheduler = UniPCMultistepScheduler.from_config(pipe.scheduler.config) @@ -2455,7 +2455,7 @@ feature_extractor = CLIPImageProcessor.from_pretrained( "laion/CLIP-ViT-B-32-laion2B-s34B-b79K" ) clip_model = CLIPModel.from_pretrained( - "laion/CLIP-ViT-B-32-laion2B-s34B-b79K", torch_dtype=torch.float16 + "laion/CLIP-ViT-B-32-laion2B-s34B-b79K", dtype=torch.float16 ) coca_model = open_clip.create_model('coca_ViT-L-14', pretrained='laion2B-s13B-b90k').to('cuda') coca_model.dtype = torch.float16 @@ -2476,7 +2476,7 @@ mixing_pipeline = DiffusionPipeline.from_pretrained( coca_model=coca_model, coca_tokenizer=coca_tokenizer, coca_transform=coca_transform, - torch_dtype=torch.float16, + dtype=torch.float16, ) mixing_pipeline.enable_attention_slicing() mixing_pipeline = mixing_pipeline.to("cuda") @@ -2525,7 +2525,7 @@ import torch pipe = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0" - , torch_dtype = torch.float16 + , dtype = torch.float16 , use_safetensors = True , variant = "fp16" , custom_pipeline = "lpw_stable_diffusion_xl", @@ -2655,7 +2655,7 @@ device="cuda" # Load fixed vae (optional) vae = AutoencoderKL.from_pretrained( - "madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16 + "madebyollin/sdxl-vae-fp16-fix", dtype=torch.float16 ).to(device) # Create scheduler and model (similar to StableDiffusionPipeline) @@ -2663,7 +2663,7 @@ model_id="stablediffusionapi/yamermix-v8-vae" scheduler = DPMSolverMultistepScheduler(beta_start=0.00085, beta_end=0.012, beta_schedule="scaled_linear", num_train_timesteps=1000) pipe = DiffusionPipeline.from_pretrained( model_id, - torch_dtype=torch.float16, + dtype=torch.float16, vae=vae, custom_pipeline="mixture_tiling_sdxl", scheduler=scheduler, @@ -2713,14 +2713,14 @@ device = "cuda" # Initialize the models and pipeline controlnet = ControlNetUnionModel.from_pretrained( - "brad-twinkl/controlnet-union-sdxl-1.0-promax", torch_dtype=torch.float16 + "brad-twinkl/controlnet-union-sdxl-1.0-promax", dtype=torch.float16 ).to(device=device) -vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16).to(device=device) +vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", dtype=torch.float16).to(device=device) model_id = "SG161222/RealVisXL_V5.0" pipe = DiffusionPipeline.from_pretrained( model_id, - torch_dtype=torch.float16, + dtype=torch.float16, vae=vae, controlnet=controlnet, custom_pipeline="mod_controlnet_tile_sr_sdxl", @@ -2814,7 +2814,7 @@ scheduler = PNDMScheduler.from_pretrained("stabilityai/stable-diffusion-2-inpain pipe = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-2-inpainting", custom_pipeline="stable_diffusion_tensorrt_inpaint", variant='fp16', - torch_dtype=torch.float16, + dtype=torch.float16, scheduler=scheduler, ) @@ -2904,7 +2904,7 @@ from diffusers.utils import load_image model_id = "kxic/zero123-165000" # zero123-105000, zero123-165000, zero123-xl -pipe = Zero1to3StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16) +pipe = Zero1to3StableDiffusionPipeline.from_pretrained(model_id, dtype=torch.float16) pipe.enable_xformers_memory_efficient_attention() pipe.enable_vae_tiling() @@ -2984,13 +2984,13 @@ input_image = load_image("https://huggingface.co/datasets/huggingface/documentat # pipe = DiffusionPipeline.from_pretrained( # "stabilityai/stable-diffusion-xl-base-1.0", # custom_pipeline="stable_diffusion_xl_reference", -# torch_dtype=torch.float16, +# dtype=torch.float16, # use_safetensors=True, # variant="fp16").to('cuda:0') pipe = StableDiffusionXLReferencePipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16, + dtype=torch.float16, use_safetensors=True, variant="fp16").to('cuda:0') @@ -3062,11 +3062,11 @@ ref_image = load_image( # initialize the models and pipeline controlnet_conditioning_scale = 0.5 # recommended for good generalization controlnet = ControlNetModel.from_pretrained( - "diffusers/controlnet-canny-sdxl-1.0", torch_dtype=torch.float16 + "diffusers/controlnet-canny-sdxl-1.0", dtype=torch.float16 ) -vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16) +vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", dtype=torch.float16) pipe = StableDiffusionXLControlNetReferencePipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-base-1.0", controlnet=controlnet, vae=vae, torch_dtype=torch.float16 + "stabilityai/stable-diffusion-xl-base-1.0", controlnet=controlnet, vae=vae, dtype=torch.float16 ).to("cuda:0") pipe.scheduler = UniPCMultistepScheduler.from_config(pipe.scheduler.config) @@ -3134,7 +3134,7 @@ from diffusers import DiffusionPipeline # make sure you're logged in with `hf auth login` model_id_or_path = "stable-diffusion-v1-5/stable-diffusion-v1-5" # can also be used with dreamlike-art/dreamlike-photoreal-2.0 -pipe = DiffusionPipeline.from_pretrained(model_id_or_path, torch_dtype=torch.float16, custom_pipeline="pipeline_fabric").to("cuda") +pipe = DiffusionPipeline.from_pretrained(model_id_or_path, dtype=torch.float16, custom_pipeline="pipeline_fabric").to("cuda") # let's specify a prompt prompt = "An astronaut riding an elephant" @@ -3376,7 +3376,7 @@ import torch pipe = DiffusionPipeline.from_pretrained("SimianLuo/LCM_Dreamshaper_v7", custom_pipeline="latent_consistency_txt2img", custom_revision="main") # To save GPU memory, torch.float16 can be used, but it may compromise image quality. -pipe.to(torch_device="cuda", torch_dtype=torch.float32) +pipe.to(torch_device="cuda", dtype=torch.float32) ``` - 2. Run inference with as little as 4 steps: @@ -3405,7 +3405,7 @@ import torch pipe = DiffusionPipeline.from_pretrained("SimianLuo/LCM_Dreamshaper_v7", custom_pipeline="latent_consistency_img2img") # To save GPU memory, torch.float16 can be used, but it may compromise image quality. -pipe.to(torch_device="cuda", torch_dtype=torch.float32) +pipe.to(torch_device="cuda", dtype=torch.float32) ``` - 2. Run inference with as little as 4 steps: @@ -3437,7 +3437,7 @@ from diffusers import DiffusionPipeline pipe = DiffusionPipeline.from_pretrained("SimianLuo/LCM_Dreamshaper_v7", custom_pipeline="latent_consistency_interpolate") # To save GPU memory, torch.float16 can be used, but it may compromise image quality. -pipe.to(torch_device="cuda", torch_dtype=torch.float32) +pipe.to(torch_device="cuda", dtype=torch.float32) prompts = [ "Self-portrait oil painting, a beautiful cyborg with golden hair, Margot Robbie, 8k", @@ -3534,14 +3534,14 @@ from examples.community.pipeline_stable_diffusion_xl_controlnet_adapter import ( controlnet_depth = ControlNetModel.from_pretrained( "diffusers/controlnet-depth-sdxl-1.0", - torch_dtype=torch.float16, + dtype=torch.float16, variant="fp16", use_safetensors=True ) adapter_depth = T2IAdapter.from_pretrained( - "TencentARC/t2i-adapter-depth-midas-sdxl-1.0", torch_dtype=torch.float16, variant="fp16", use_safetensors=True + "TencentARC/t2i-adapter-depth-midas-sdxl-1.0", dtype=torch.float16, variant="fp16", use_safetensors=True ) -vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16, use_safetensors=True) +vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", dtype=torch.float16, use_safetensors=True) pipe = StableDiffusionXLControlNetAdapterPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", @@ -3550,7 +3550,7 @@ pipe = StableDiffusionXLControlNetAdapterPipeline.from_pretrained( vae=vae, variant="fp16", use_safetensors=True, - torch_dtype=torch.float16, + dtype=torch.float16, ) pipe = pipe.to("cuda") pipe.enable_xformers_memory_efficient_attention() @@ -3599,14 +3599,14 @@ from examples.community.pipeline_stable_diffusion_xl_controlnet_adapter_inpaint controlnet_depth = ControlNetModel.from_pretrained( "diffusers/controlnet-depth-sdxl-1.0", - torch_dtype=torch.float16, + dtype=torch.float16, variant="fp16", use_safetensors=True ) adapter_depth = T2IAdapter.from_pretrained( - "TencentARC/t2i-adapter-depth-midas-sdxl-1.0", torch_dtype=torch.float16, variant="fp16", use_safetensors=True + "TencentARC/t2i-adapter-depth-midas-sdxl-1.0", dtype=torch.float16, variant="fp16", use_safetensors=True ) -vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16, use_safetensors=True) +vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", dtype=torch.float16, use_safetensors=True) pipe = StableDiffusionXLControlNetAdapterInpaintPipeline.from_pretrained( "diffusers/stable-diffusion-xl-1.0-inpainting-0.1", @@ -3615,7 +3615,7 @@ pipe = StableDiffusionXLControlNetAdapterInpaintPipeline.from_pretrained( vae=vae, variant="fp16", use_safetensors=True, - torch_dtype=torch.float16, + dtype=torch.float16, ) pipe = pipe.to("cuda") pipe.enable_xformers_memory_efficient_attention() @@ -4042,8 +4042,8 @@ from PIL import Image motion_id = "guoyww/animatediff-motion-adapter-v1-5-2" adapter = MotionAdapter.from_pretrained(motion_id) -controlnet = ControlNetModel.from_pretrained("lllyasviel/control_v11p_sd15_openpose", torch_dtype=torch.float16) -vae = AutoencoderKL.from_pretrained("stabilityai/sd-vae-ft-mse", torch_dtype=torch.float16) +controlnet = ControlNetModel.from_pretrained("lllyasviel/control_v11p_sd15_openpose", dtype=torch.float16) +vae = AutoencoderKL.from_pretrained("stabilityai/sd-vae-ft-mse", dtype=torch.float16) model_id = "SG161222/Realistic_Vision_V5.1_noVAE" pipe = DiffusionPipeline.from_pretrained( @@ -4052,7 +4052,7 @@ pipe = DiffusionPipeline.from_pretrained( controlnet=controlnet, vae=vae, custom_pipeline="pipeline_animatediff_controlnet", - torch_dtype=torch.float16, + dtype=torch.float16, ).to(device="cuda") pipe.scheduler = DPMSolverMultistepScheduler.from_pretrained( model_id, subfolder="scheduler", beta_schedule="linear", clip_sample=False, timestep_spacing="linspace", steps_offset=1 @@ -4104,9 +4104,9 @@ from PIL import Image motion_id = "guoyww/animatediff-motion-adapter-v1-5-2" adapter = MotionAdapter.from_pretrained(motion_id) -controlnet1 = ControlNetModel.from_pretrained("lllyasviel/control_v11p_sd15_openpose", torch_dtype=torch.float16) -controlnet2 = ControlNetModel.from_pretrained("lllyasviel/sd-controlnet-canny", torch_dtype=torch.float16) -vae = AutoencoderKL.from_pretrained("stabilityai/sd-vae-ft-mse", torch_dtype=torch.float16) +controlnet1 = ControlNetModel.from_pretrained("lllyasviel/control_v11p_sd15_openpose", dtype=torch.float16) +controlnet2 = ControlNetModel.from_pretrained("lllyasviel/sd-controlnet-canny", dtype=torch.float16) +vae = AutoencoderKL.from_pretrained("stabilityai/sd-vae-ft-mse", dtype=torch.float16) model_id = "SG161222/Realistic_Vision_V5.1_noVAE" pipe = DiffusionPipeline.from_pretrained( @@ -4115,7 +4115,7 @@ pipe = DiffusionPipeline.from_pretrained( controlnet=[controlnet1, controlnet2], vae=vae, custom_pipeline="pipeline_animatediff_controlnet", - torch_dtype=torch.float16, + dtype=torch.float16, ).to(device="cuda") pipe.scheduler = DPMSolverMultistepScheduler.from_pretrained( model_id, subfolder="scheduler", clip_sample=False, timestep_spacing="linspace", steps_offset=1, beta_schedule="linear", @@ -4203,7 +4203,7 @@ pipe = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", custom_pipeline="pipeline_demofusion_sdxl", custom_revision="main", - torch_dtype=torch.float16, + dtype=torch.float16, ) pipe = pipe.to("cuda") @@ -4359,7 +4359,7 @@ from diffusers import DiffusionPipeline import torch -pipe = DiffusionPipeline.from_pretrained("XCLIU/instaflow_0_9B_from_sd_1_5", torch_dtype=torch.float16, custom_pipeline="instaflow_one_step") +pipe = DiffusionPipeline.from_pretrained("XCLIU/instaflow_0_9B_from_sd_1_5", dtype=torch.float16, custom_pipeline="instaflow_one_step") pipe.to("cuda") ### if GPU is not available, comment this line prompt = "A hyper-realistic photo of a cute cat." @@ -4379,7 +4379,7 @@ import torch device = torch.device("cuda" if torch.cuda.is_available() else "cpu") -pipe = DiffusionPipeline.from_pretrained("XCLIU/instaflow_0_9B_from_sd_1_5", torch_dtype=torch.float16, custom_pipeline="instaflow_one_step") +pipe = DiffusionPipeline.from_pretrained("XCLIU/instaflow_0_9B_from_sd_1_5", dtype=torch.float16, custom_pipeline="instaflow_one_step") pipe.to(device) ### if GPU is not available, comment this line pipe.load_lora_weights("artificialguybr/logo-redmond-1-5v-logo-lora-for-liberteredmond-sd-1-5") prompt = "logo, A logo for a fitness app, dynamic running figure, energetic colors (red, orange) ),LogoRedAF ," @@ -4424,7 +4424,7 @@ prompt = "A lying dog" # Float32 is essential to a well optimization model_path = "stable-diffusion-v1-5/stable-diffusion-v1-5" scheduler = DDIMScheduler(num_train_timesteps=1000, beta_start=0.00085, beta_end=0.0120, beta_schedule="scaled_linear") -pipeline = NullTextPipeline.from_pretrained(model_path, scheduler=scheduler, torch_dtype=torch.float32).to(device) +pipeline = NullTextPipeline.from_pretrained(model_path, scheduler=scheduler, dtype=torch.float32).to(device) # Saves the inverted_latent to save time inverted_latent, uncond = pipeline.invert(input_image, invert_prompt, num_inner_steps=10, early_stop_epsilon=1e-5, num_inference_steps=steps) @@ -4530,7 +4530,7 @@ from diffusers import DiffusionPipeline from PIL import Image model_id = "a-r-r-o-w/dreamshaper-xl-turbo" -pipe = DiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16, variant="fp16", custom_pipeline="pipeline_sdxl_style_aligned") +pipe = DiffusionPipeline.from_pretrained(model_id, dtype=torch.float16, variant="fp16", custom_pipeline="pipeline_sdxl_style_aligned") pipe = pipe.to("cuda") # Enable memory saving techniques @@ -4629,7 +4629,7 @@ noise_scheduler = DDIMScheduler( vae = AutoencoderKL.from_pretrained("stabilityai/sd-vae-ft-mse").to(dtype=torch.float16) pipeline = DiffusionPipeline.from_pretrained( "SG161222/Realistic_Vision_V4.0_noVAE", - torch_dtype=torch.float16, + dtype=torch.float16, scheduler=noise_scheduler, vae=vae, custom_pipeline="ip_adapter_face_id" @@ -4694,13 +4694,13 @@ face_adapter = './checkpoints/ip-adapter.bin' controlnet_path = './checkpoints/ControlNetModel' # load IdentityNet -controlnet = ControlNetModel.from_pretrained(controlnet_path, torch_dtype=torch.float16) +controlnet = ControlNetModel.from_pretrained(controlnet_path, dtype=torch.float16) base_model = 'wangqixun/YamerMIX_v8' pipe = StableDiffusionXLInstantIDPipeline.from_pretrained( base_model, controlnet=controlnet, - torch_dtype=torch.float16 + dtype=torch.float16 ) pipe.to("cuda") @@ -4744,7 +4744,7 @@ from scheduling_ufogen import UFOGenScheduler ufogen_model_id_or_path = "/path/to/ufogen/model" pipe = StableDiffusionPipeline( ufogen_model_id_or_path, - torch_dtype=torch.float16, + dtype=torch.float16, ) # You can initialize a UFOGenScheduler as follows: @@ -4871,7 +4871,7 @@ python -m pip install intel_extension_for_pytorch== -f https://dev 2. After pipeline initialization, `prepare_for_ipex()` should be called to enable IPEX acceleration. Supported inference datatypes are Float32 and BFloat16. ```python -pipe = AnimateDiffPipelineIpex.from_pretrained(base, motion_adapter=adapter, torch_dtype=dtype).to(device) +pipe = AnimateDiffPipelineIpex.from_pretrained(base, motion_adapter=adapter, dtype=dtype).to(device) # For Float32 pipe.prepare_for_ipex(torch.float32, prompt="A girl smiling") # For BFloat16 @@ -4924,12 +4924,12 @@ def elapsed_time(pipeline, nb_pass=3, num_inference_steps=1): ############## bf16 inference performance ############### # 1. IPEX Pipeline initialization -pipe = AnimateDiffPipelineIpex.from_pretrained(base, motion_adapter=adapter, torch_dtype=dtype).to(device) +pipe = AnimateDiffPipelineIpex.from_pretrained(base, motion_adapter=adapter, dtype=dtype).to(device) pipe.scheduler = EulerDiscreteScheduler.from_config(pipe.scheduler.config, timestep_spacing="trailing", beta_schedule="linear") pipe.prepare_for_ipex(torch.bfloat16, prompt = prompt) # 2. Original Pipeline initialization -pipe2 = AnimateDiffPipeline.from_pretrained(base, motion_adapter=adapter, torch_dtype=dtype).to(device) +pipe2 = AnimateDiffPipeline.from_pretrained(base, motion_adapter=adapter, dtype=dtype).to(device) pipe2.scheduler = EulerDiscreteScheduler.from_config(pipe2.scheduler.config, timestep_spacing="trailing", beta_schedule="linear") # 3. Compare performance between Original Pipeline and IPEX Pipeline @@ -4942,12 +4942,12 @@ with torch.cpu.amp.autocast(enabled=True, dtype=torch.bfloat16): ############## fp32 inference performance ############### # 1. IPEX Pipeline initialization -pipe3 = AnimateDiffPipelineIpex.from_pretrained(base, motion_adapter=adapter, torch_dtype=dtype).to(device) +pipe3 = AnimateDiffPipelineIpex.from_pretrained(base, motion_adapter=adapter, dtype=dtype).to(device) pipe3.scheduler = EulerDiscreteScheduler.from_config(pipe3.scheduler.config, timestep_spacing="trailing", beta_schedule="linear") pipe3.prepare_for_ipex(torch.float32, prompt = prompt) # 2. Original Pipeline initialization -pipe4 = AnimateDiffPipeline.from_pretrained(base, motion_adapter=adapter, torch_dtype=dtype).to(device) +pipe4 = AnimateDiffPipeline.from_pretrained(base, motion_adapter=adapter, dtype=dtype).to(device) pipe4.scheduler = EulerDiscreteScheduler.from_config(pipe4.scheduler.config, timestep_spacing="trailing", beta_schedule="linear") # 3. Compare performance between Original Pipeline and IPEX Pipeline @@ -4973,7 +4973,7 @@ from pipeline_hunyuandit_differential_img2img import ( pipe = HunyuanDiTDifferentialImg2ImgPipeline.from_pretrained( - "Tencent-Hunyuan/HunyuanDiT-Diffusers", torch_dtype=torch.float16 + "Tencent-Hunyuan/HunyuanDiT-Diffusers", dtype=torch.float16 ).to("cuda") @@ -5077,7 +5077,7 @@ pipeline = DiffusionPipeline.from_pretrained( scheduler=scheduler, variant="fp16", use_safetensors=True, - torch_dtype=dtype, + dtype=dtype, ).to(device) @@ -5155,7 +5155,7 @@ from diffusers.utils.torch_utils import randn_tensor pipe = StableDiffusionPipeline.from_pretrained( "stable-diffusion-v1-5/stable-diffusion-v1-5", custom_pipeline="hyoungwoncho/sd_perturbed_attention_guidance", - torch_dtype=torch.float16 + dtype=torch.float16 ) device = "cuda" @@ -5250,14 +5250,14 @@ torch.manual_seed(0) # load controlnet controlnet = PixArtControlNetAdapterModel.from_pretrained( controlnet_repo_id, - torch_dtype=weight_dtype, + dtype=weight_dtype, use_safetensors=True, ).to(device) pipe = PixArtAlphaControlnetPipeline.from_pretrained( "PixArt-alpha/PixArt-XL-2-1024-MS", controlnet=controlnet, - torch_dtype=weight_dtype, + dtype=weight_dtype, use_safetensors=True, ).to(device) @@ -5317,7 +5317,7 @@ from examples.community.cogvideox_ddim_inversion import CogVideoXPipelineForDDIM # Load pretrained pipeline pipeline = CogVideoXPipelineForDDIMInversion.from_pretrained( "THUDM/CogVideoX1.5-5B", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, ).to("cuda") # Run DDIM inversion, and the videos will be generated in the output_path @@ -5369,12 +5369,12 @@ model_file = hf_hub_download( ) # Initialize the models and pipeline -vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=dtype) +vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", dtype=dtype) model_id = "SG161222/RealVisXL_V4.0" pipe = DiffusionPipeline.from_pretrained( model_id, - torch_dtype=dtype, + dtype=dtype, vae=vae, unet=None, #<- Do not load with original model. custom_pipeline="pipeline_faithdiff_stable_diffusion_xl", @@ -5459,9 +5459,9 @@ edit_instruction = "Turn sky into a sunny one" pipe = DiffusionPipeline.from_pretrained( - "stabilityai/stable-diffusion-3-medium-diffusers", custom_pipeline="pipeline_stable_diffusion_3_instruct_pix2pix", torch_dtype=torch.float16).to('cuda') + "stabilityai/stable-diffusion-3-medium-diffusers", custom_pipeline="pipeline_stable_diffusion_3_instruct_pix2pix", dtype=torch.float16).to('cuda') -pipe.transformer = SD3Transformer2DModel.from_pretrained("CaptainZZZ/sd3-instructpix2pix",torch_dtype=torch.float16).to('cuda') +pipe.transformer = SD3Transformer2DModel.from_pretrained("CaptainZZZ/sd3-instructpix2pix",dtype=torch.float16).to('cuda') edited_image = pipe( prompt=edit_instruction, @@ -5503,7 +5503,7 @@ from diffusers.utils import load_image pipe = FluxKontextPipeline.from_pretrained( "black-forest-labs/FLUX.1-Kontext-dev", - torch_dtype=torch.bfloat16, + dtype=torch.bfloat16, custom_pipeline="pipeline_flux_kontext_multiple_images", ) pipe.to("cuda") @@ -5559,14 +5559,14 @@ prior_model = "black-forest-labs/FLUX.1-Redux-dev" # Load ControlNet controlnet = FluxControlNetModel.from_pretrained( controlnet_model, - torch_dtype=dtype, + dtype=dtype, ) # Load Fill + ControlNet Pipeline fill_pipe = FluxControlNetFillInpaintPipeline.from_pretrained( base_model, controlnet=controlnet, - torch_dtype=dtype, + dtype=dtype, ).to(device) # OPTIONAL FP8 @@ -5578,7 +5578,7 @@ fill_pipe = FluxControlNetFillInpaintPipeline.from_pretrained( # OPTIONAL Prior Redux #pipe_prior_redux = FluxPriorReduxPipeline.from_pretrained( # prior_model, -# torch_dtype=dtype, +# dtype=dtype, #).to(device) # Inputs @@ -5629,4 +5629,3 @@ from datetime import datetime timestamp = datetime.now().strftime("%Y%m%d_%H%M%S") result.images[0].save(f"flux_fill_controlnet_inpaint_depth{timestamp}.jpg") ``` - diff --git a/examples/community/README_community_scripts.md b/examples/community/README_community_scripts.md index d790b67a44c5..0c6c5ec69669 100644 --- a/examples/community/README_community_scripts.md +++ b/examples/community/README_community_scripts.md @@ -123,12 +123,12 @@ def prepare_ip_adapter_image_embeds( vae = AutoencoderKL.from_pretrained( "madebyollin/sdxl-vae-fp16-fix", - torch_dtype=torch.float16, + dtype=torch.float16, ).to("cuda") pipeline = StableDiffusionXLPipeline.from_pretrained( "RunDiffusion/Juggernaut-XL-v9", - torch_dtype=torch.float16, + dtype=torch.float16, vae=vae, variant="fp16", ).to("cuda") @@ -210,7 +210,7 @@ def seamless_tiling(pipeline, x_axis, y_axis): layer._conv_forward = asymmetric_conv2d_convforward.__get__(layer, torch.nn.Conv2d) return pipeline -pipeline = StableDiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16, use_safetensors=True) +pipeline = StableDiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16, use_safetensors=True) pipeline.enable_model_cpu_offload() prompt = ["texture of a red brick wall"] seed = 123456 @@ -284,7 +284,7 @@ class SDPromptSchedulingCallback(PipelineCallback): pipeline: StableDiffusionPipeline = StableDiffusionPipeline.from_pretrained( "stable-diffusion-v1-5/stable-diffusion-v1-5", - torch_dtype=torch.float16, + dtype=torch.float16, variant="fp16", use_safetensors=True, ).to("cuda") @@ -379,7 +379,7 @@ class SDXLPromptSchedulingCallback(PipelineCallback): pipeline: StableDiffusionXLPipeline = StableDiffusionXLPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", - torch_dtype=torch.float16, + dtype=torch.float16, variant="fp16", use_safetensors=True, ).to("cuda") diff --git a/examples/controlnet/README.md b/examples/controlnet/README.md index 388368a8778f..474167609951 100644 --- a/examples/controlnet/README.md +++ b/examples/controlnet/README.md @@ -267,9 +267,9 @@ import torch base_model_path = "path to model" controlnet_path = "path to controlnet" -controlnet = ControlNetModel.from_pretrained(controlnet_path, torch_dtype=torch.float16) +controlnet = ControlNetModel.from_pretrained(controlnet_path, dtype=torch.float16) pipe = StableDiffusionControlNetPipeline.from_pretrained( - base_model_path, controlnet=controlnet, torch_dtype=torch.float16 + base_model_path, controlnet=controlnet, dtype=torch.float16 ) # speed up diffusion process with faster scheduler and memory optimization diff --git a/examples/controlnet/README_flux.md b/examples/controlnet/README_flux.md index fefe0148a5be..dca0183c86f3 100644 --- a/examples/controlnet/README_flux.md +++ b/examples/controlnet/README_flux.md @@ -138,11 +138,11 @@ from diffusers.models.controlnet_flux import FluxControlNetModel base_model = 'black-forest-labs/FLUX.1-dev' controlnet_model = 'promeai/FLUX.1-controlnet-lineart-promeai' -controlnet = FluxControlNetModel.from_pretrained(controlnet_model, torch_dtype=torch.bfloat16) +controlnet = FluxControlNetModel.from_pretrained(controlnet_model, dtype=torch.bfloat16) pipe = FluxControlNetPipeline.from_pretrained( base_model, controlnet=controlnet, - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) # enable memory optimizations pipe.enable_model_cpu_offload() diff --git a/examples/controlnet/README_sd3.md b/examples/controlnet/README_sd3.md index 9c2d6aaac3c8..86267a0a220c 100644 --- a/examples/controlnet/README_sd3.md +++ b/examples/controlnet/README_sd3.md @@ -108,7 +108,7 @@ import torch base_model_path = "stabilityai/stable-diffusion-3-medium-diffusers" controlnet_path = "DavyMorgan/sd3-controlnet-out" -controlnet = SD3ControlNetModel.from_pretrained(controlnet_path, torch_dtype=torch.float16) +controlnet = SD3ControlNetModel.from_pretrained(controlnet_path, dtype=torch.float16) pipe = StableDiffusion3ControlNetPipeline.from_pretrained( base_model_path, controlnet=controlnet ) diff --git a/examples/controlnet/README_sdxl.md b/examples/controlnet/README_sdxl.md index 442cfd386a3a..44a7bdf6f20a 100644 --- a/examples/controlnet/README_sdxl.md +++ b/examples/controlnet/README_sdxl.md @@ -101,9 +101,9 @@ import torch base_model_path = "stabilityai/stable-diffusion-xl-base-1.0" controlnet_path = "path to controlnet" -controlnet = ControlNetModel.from_pretrained(controlnet_path, torch_dtype=torch.float16) +controlnet = ControlNetModel.from_pretrained(controlnet_path, dtype=torch.float16) pipe = StableDiffusionXLControlNetPipeline.from_pretrained( - base_model_path, controlnet=controlnet, torch_dtype=torch.float16 + base_model_path, controlnet=controlnet, dtype=torch.float16 ) # speed up diffusion process with faster scheduler and memory optimization @@ -133,9 +133,9 @@ SDXL's VAE is known to suffer from numerical instability issues. This is why we If you're using this VAE during training, you need to ensure you're using it during inference too. You do so by: ```diff -+ vae = AutoencoderKL.from_pretrained(vae_path_or_repo_id, torch_dtype=torch.float16) -controlnet = ControlNetModel.from_pretrained(controlnet_path, torch_dtype=torch.float16) ++ vae = AutoencoderKL.from_pretrained(vae_path_or_repo_id, dtype=torch.float16) +controlnet = ControlNetModel.from_pretrained(controlnet_path, dtype=torch.float16) pipe = StableDiffusionXLControlNetPipeline.from_pretrained( - base_model_path, controlnet=controlnet, torch_dtype=torch.float16, + base_model_path, controlnet=controlnet, dtype=torch.float16, + vae=vae, ) diff --git a/examples/cosmos3/README.md b/examples/cosmos3/README.md index c9b3517ce243..f86a528abc3e 100644 --- a/examples/cosmos3/README.md +++ b/examples/cosmos3/README.md @@ -224,7 +224,7 @@ torchrun --nproc_per_node 4 examples/cosmos3/inference_cosmos3.py \ The CLI uses `Cosmos3OmniModularPipeline`, not the legacy task pipeline. Its distributed setup order is important: 1. Construct it with `Cosmos3OmniModularPipeline.from_pretrained(...)` and call - `pipe.load_components(torch_dtype=torch.bfloat16)` while it is still on CPU. Do not use + `pipe.load_components(dtype=torch.bfloat16)` while it is still on CPU. Do not use `device_map`. 2. Initialize the NCCL process group and call `torch.cuda.set_device(local_rank)` before building the device mesh or applying TP. If needed, replace the scheduler with diff --git a/examples/custom_diffusion/README.md b/examples/custom_diffusion/README.md index 8dcba85622cb..337f91ea2195 100644 --- a/examples/custom_diffusion/README.md +++ b/examples/custom_diffusion/README.md @@ -196,7 +196,7 @@ import torch from diffusers import DiffusionPipeline pipe = DiffusionPipeline.from_pretrained( - "CompVis/stable-diffusion-v1-4", torch_dtype=torch.float16 + "CompVis/stable-diffusion-v1-4", dtype=torch.float16 ).to("cuda") pipe.unet.load_attn_procs( "path-to-save-model", weight_name="pytorch_custom_diffusion_weights.bin" @@ -223,7 +223,7 @@ model_id = "sayakpaul/custom-diffusion-cat" card = RepoCard.load(model_id) base_model_id = card.data.to_dict()["base_model"] -pipe = DiffusionPipeline.from_pretrained(base_model_id, torch_dtype=torch.float16).to( +pipe = DiffusionPipeline.from_pretrained(base_model_id, dtype=torch.float16).to( "cuda") pipe.unet.load_attn_procs(model_id, weight_name="pytorch_custom_diffusion_weights.bin") pipe.load_textual_inversion(model_id, weight_name=".bin") @@ -248,7 +248,7 @@ model_id = "sayakpaul/custom-diffusion-cat-wooden-pot" card = RepoCard.load(model_id) base_model_id = card.data.to_dict()["base_model"] -pipe = DiffusionPipeline.from_pretrained(base_model_id, torch_dtype=torch.float16).to( +pipe = DiffusionPipeline.from_pretrained(base_model_id, dtype=torch.float16).to( "cuda") pipe.unet.load_attn_procs(model_id, weight_name="pytorch_custom_diffusion_weights.bin") pipe.load_textual_inversion(model_id, weight_name=".bin") diff --git a/examples/dreambooth/README.md b/examples/dreambooth/README.md index 73ea8238953f..a5e0e317b568 100644 --- a/examples/dreambooth/README.md +++ b/examples/dreambooth/README.md @@ -283,7 +283,7 @@ from diffusers import StableDiffusionPipeline import torch model_id = "path-to-your-trained-model" -pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16).to("cuda") +pipe = StableDiffusionPipeline.from_pretrained(model_id, dtype=torch.float16).to("cuda") prompt = "A photo of sks dog in a bucket" image = pipe(prompt, num_inference_steps=50, guidance_scale=7.5).images[0] @@ -404,7 +404,7 @@ lora_model_id = "patrickvonplaten/lora_dreambooth_dog_example" card = RepoCard.load(lora_model_id) base_model_id = card.data.to_dict()["base_model"] -pipe = StableDiffusionPipeline.from_pretrained(base_model_id, torch_dtype=torch.float16) +pipe = StableDiffusionPipeline.from_pretrained(base_model_id, dtype=torch.float16) ... ``` @@ -420,7 +420,7 @@ lora_model_id = "sayakpaul/dreambooth-text-encoder-test" card = RepoCard.load(lora_model_id) base_model_id = card.data.to_dict()["base_model"] -pipe = StableDiffusionPipeline.from_pretrained(base_model_id, torch_dtype=torch.float16) +pipe = StableDiffusionPipeline.from_pretrained(base_model_id, dtype=torch.float16) pipe = pipe.to("cuda") pipe.load_lora_weights(lora_model_id) image = pipe("A picture of a sks dog in a bucket", num_inference_steps=25).images[0] diff --git a/examples/dreambooth/README_ideogram4.md b/examples/dreambooth/README_ideogram4.md index 4fda51a4c092..f2810ca85c24 100644 --- a/examples/dreambooth/README_ideogram4.md +++ b/examples/dreambooth/README_ideogram4.md @@ -158,7 +158,7 @@ After training, load the base pipeline and your LoRA: import torch from diffusers import Ideogram4Pipeline -pipeline = Ideogram4Pipeline.from_pretrained("ideogram-ai/ideogram-4-nf4-diffusers", torch_dtype=torch.bfloat16) +pipeline = Ideogram4Pipeline.from_pretrained("ideogram-ai/ideogram-4-nf4-diffusers", dtype=torch.bfloat16) pipeline.to("cuda") pipeline.load_lora_weights("trained-ideogram4-lora", weight_name="pytorch_lora_weights.safetensors") diff --git a/examples/dreambooth/README_krea2.md b/examples/dreambooth/README_krea2.md index 646217d9f4f8..fea414c3126d 100644 --- a/examples/dreambooth/README_krea2.md +++ b/examples/dreambooth/README_krea2.md @@ -191,7 +191,7 @@ Train on RAW, then load your LoRA into **Turbo** for fast, high-quality generati import torch from diffusers import Krea2Pipeline -pipe = Krea2Pipeline.from_pretrained("krea/Krea-2-Turbo", torch_dtype=torch.bfloat16) +pipe = Krea2Pipeline.from_pretrained("krea/Krea-2-Turbo", dtype=torch.bfloat16) pipe.to("cuda") # Load your trained LoRA (trained on Krea 2 RAW) diff --git a/examples/dreambooth/README_sdxl.md b/examples/dreambooth/README_sdxl.md index c033d1e641bc..48f6c7e39532 100644 --- a/examples/dreambooth/README_sdxl.md +++ b/examples/dreambooth/README_sdxl.md @@ -135,7 +135,7 @@ lora_model_id = <"lora-sdxl-dreambooth-id"> card = RepoCard.load(lora_model_id) base_model_id = card.data.to_dict()["base_model"] -pipe = DiffusionPipeline.from_pretrained(base_model_id, torch_dtype=torch.float16) +pipe = DiffusionPipeline.from_pretrained(base_model_id, dtype=torch.float16) pipe = pipe.to("cuda") pipe.load_lora_weights(lora_model_id) image = pipe("A picture of a sks dog in a bucket", num_inference_steps=25).images[0] @@ -154,13 +154,13 @@ card = RepoCard.load(lora_model_id) base_model_id = card.data.to_dict()["base_model"] # Load the base pipeline and load the LoRA parameters into it. -pipe = DiffusionPipeline.from_pretrained(base_model_id, torch_dtype=torch.float16) +pipe = DiffusionPipeline.from_pretrained(base_model_id, dtype=torch.float16) pipe = pipe.to("cuda") pipe.load_lora_weights(lora_model_id) # Load the refiner. refiner = StableDiffusionXLImg2ImgPipeline.from_pretrained( - "stabilityai/stable-diffusion-xl-refiner-1.0", torch_dtype=torch.float16, use_safetensors=True, variant="fp16" + "stabilityai/stable-diffusion-xl-refiner-1.0", dtype=torch.float16, use_safetensors=True, variant="fp16" ) refiner.to("cuda") diff --git a/examples/dreambooth/README_z_image.md b/examples/dreambooth/README_z_image.md index 92d8c180bf65..598759cdc30c 100644 --- a/examples/dreambooth/README_z_image.md +++ b/examples/dreambooth/README_z_image.md @@ -326,7 +326,7 @@ Once you have trained a LoRA, you can load it for inference: import torch from diffusers import ZImagePipeline -pipe = ZImagePipeline.from_pretrained("Tongyi-MAI/Z-Image", torch_dtype=torch.bfloat16) +pipe = ZImagePipeline.from_pretrained("Tongyi-MAI/Z-Image", dtype=torch.bfloat16) pipe.to("cuda") # Load your trained LoRA diff --git a/examples/flux-control/README.md b/examples/flux-control/README.md index 5463fc155247..74f637d4f35d 100644 --- a/examples/flux-control/README.md +++ b/examples/flux-control/README.md @@ -105,7 +105,7 @@ from PIL import Image import numpy as np import torch -pipe = FluxControlPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16).to("cuda") +pipe = FluxControlPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", dtype=torch.bfloat16).to("cuda") pipe.load_lora_weights("...") # change this. open_pose = OpenposeDetector.from_pretrained("lllyasviel/Annotators") @@ -174,7 +174,7 @@ import torch transformer = FluxTransformer2DModel.from_pretrained("...") # change this. pipe = FluxControlPipeline.from_pretrained( - "black-forest-labs/FLUX.1-dev", transformer=transformer, torch_dtype=torch.bfloat16 + "black-forest-labs/FLUX.1-dev", transformer=transformer, dtype=torch.bfloat16 ).to("cuda") open_pose = OpenposeDetector.from_pretrained("lllyasviel/Annotators") diff --git a/examples/instruct_pix2pix/README.md b/examples/instruct_pix2pix/README.md index 9df6b46ee99b..b18f954d57e5 100644 --- a/examples/instruct_pix2pix/README.md +++ b/examples/instruct_pix2pix/README.md @@ -148,7 +148,7 @@ import torch from diffusers import StableDiffusionInstructPix2PixPipeline model_id = "your_model_id" # <- replace this -pipe = StableDiffusionInstructPix2PixPipeline.from_pretrained(model_id, torch_dtype=torch.float16).to("cuda") +pipe = StableDiffusionInstructPix2PixPipeline.from_pretrained(model_id, dtype=torch.float16).to("cuda") generator = torch.Generator("cuda").manual_seed(0) url = "https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/test_pix2pix_4.png" diff --git a/examples/instruct_pix2pix/README_sdxl.md b/examples/instruct_pix2pix/README_sdxl.md index dcf828e80ba8..83d0eebdeba1 100644 --- a/examples/instruct_pix2pix/README_sdxl.md +++ b/examples/instruct_pix2pix/README_sdxl.md @@ -110,7 +110,7 @@ import torch from diffusers import StableDiffusionXLInstructPix2PixPipeline model_id = "your_model_id" # <- replace this -pipe = StableDiffusionXLInstructPix2PixPipeline.from_pretrained(model_id, torch_dtype=torch.float16).to("cuda") +pipe = StableDiffusionXLInstructPix2PixPipeline.from_pretrained(model_id, dtype=torch.float16).to("cuda") generator = torch.Generator("cuda").manual_seed(0) url = "https://datasets-server.huggingface.co/assets/fusing/instructpix2pix-1000-samples/--/fusing--instructpix2pix-1000-samples/train/23/input_image/image.jpg" diff --git a/examples/kandinsky2_2/text_to_image/README.md b/examples/kandinsky2_2/text_to_image/README.md index c6afca868970..358893717dfb 100644 --- a/examples/kandinsky2_2/text_to_image/README.md +++ b/examples/kandinsky2_2/text_to_image/README.md @@ -108,7 +108,7 @@ Once the training is finished the model will be saved in the `output_dir` specif from diffusers import AutoPipelineForText2Image import torch -pipe = AutoPipelineForText2Image.from_pretrained(output_dir, torch_dtype=torch.float16) +pipe = AutoPipelineForText2Image.from_pretrained(output_dir, dtype=torch.float16) pipe.enable_model_cpu_offload() prompt='A robot naruto, 4k photo' @@ -124,7 +124,7 @@ model_path = "path_to_saved_model" unet = UNet2DConditionModel.from_pretrained(model_path + "/checkpoint-/unet") -pipe = AutoPipelineForText2Image.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", unet=unet, torch_dtype=torch.float16) +pipe = AutoPipelineForText2Image.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", unet=unet, dtype=torch.float16) pipe.enable_model_cpu_offload() image = pipe(prompt="A robot naruto, 4k photo").images[0] @@ -165,9 +165,9 @@ To perform inference with the fine-tuned prior model, you will need to first cre from diffusers import AutoPipelineForText2Image, DiffusionPipeline import torch -pipe_prior = DiffusionPipeline.from_pretrained(output_dir, torch_dtype=torch.float16) +pipe_prior = DiffusionPipeline.from_pretrained(output_dir, dtype=torch.float16) prior_components = {"prior_" + k: v for k,v in pipe_prior.components.items()} -pipe = AutoPipelineForText2Image.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", **prior_components, torch_dtype=torch.float16) +pipe = AutoPipelineForText2Image.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", **prior_components, dtype=torch.float16) pipe.enable_model_cpu_offload() prompt='A robot naruto, 4k photo' @@ -281,7 +281,7 @@ Once you have trained a Kandinsky decoder model using the above command, inferen from diffusers import AutoPipelineForText2Image import torch -pipe = AutoPipelineForText2Image.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", torch_dtype=torch.float16) +pipe = AutoPipelineForText2Image.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", dtype=torch.float16) pipe.unet.load_attn_procs(output_dir) pipe.enable_model_cpu_offload() @@ -296,7 +296,7 @@ image.save("robot_naruto.png") from diffusers import AutoPipelineForText2Image import torch -pipe = AutoPipelineForText2Image.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", torch_dtype=torch.float16) +pipe = AutoPipelineForText2Image.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", dtype=torch.float16) pipe.prior_prior.load_attn_procs(output_dir) pipe.enable_model_cpu_offload() diff --git a/examples/model_search/README.md b/examples/model_search/README.md index da7fb3358728..a553b25c4e10 100644 --- a/examples/model_search/README.md +++ b/examples/model_search/README.md @@ -101,7 +101,7 @@ pipeline.auto_load_textual_inversion("EasyNegative", token="EasyNegative") | search_word | string, Path | ー | The search query string. Can be a keyword, Civitai URL, local directory or file path. | | model_type | string | `Checkpoint` | The type of model to search for.
(for example `Checkpoint`, `TextualInversion`, `Controlnet`, `LORA`, `Hypernetwork`, `AestheticGradient`, `Poses`) | | base_model | string | None | Trained model tag (for example `SD 1.5`, `SD 3.5`, `SDXL 1.0`) | -| torch_dtype | string, torch.dtype | None | Override the default `torch.dtype` and load the model with another dtype. | +| dtype | string, torch.dtype | None | Override the default `torch.dtype` and load the model with another dtype. | | force_download | bool | False | Whether or not to force the (re-)download of the model weights and configuration files, overriding the cached versions if they exist. | | cache_dir | string, Path | None | Path to the folder where cached files are stored. | | resume | bool | False | Whether to resume an incomplete download. | @@ -134,7 +134,7 @@ pipeline.auto_load_textual_inversion("EasyNegative", token="EasyNegative") |:---------------------:|:-------------------:|:--------------:|:----------------------------------------------------------------:| | search_word | string, Path | ー | The search query string. Can be a keyword, Hugging Face URL, local directory or file path, or a Hugging Face path (`/`). | | checkpoint_format | string | `single_file` | The format of the model checkpoint.
● `single_file` to search for `single file checkpoint`
●`diffusers` to search for `multifolder diffusers format checkpoint` | -| torch_dtype | string, torch.dtype | None | Override the default `torch.dtype` and load the model with another dtype. | +| dtype | string, torch.dtype | None | Override the default `torch.dtype` and load the model with another dtype. | | force_download | bool | False | Whether or not to force the (re-)download of the model weights and configuration files, overriding the cached versions if they exist. | | cache_dir | string, Path | None | Path to a directory where a downloaded pretrained model configuration is cached if the standard cache is not used. | | token | string, bool | None | The token to use as HTTP bearer authorization for remote files. | diff --git a/examples/profiling/README.md b/examples/profiling/README.md index 38b35772d03d..b38979577b60 100644 --- a/examples/profiling/README.md +++ b/examples/profiling/README.md @@ -95,7 +95,7 @@ This is non-invasive — it monkey-patches bound methods without modifying sourc **Pipeline config registry** — each entry specifies: -- `pipeline_cls`, `pretrained_model_name_or_path`, `torch_dtype` +- `pipeline_cls`, `pretrained_model_name_or_path`, `dtype` - `call_kwargs` with pipeline-specific defaults: | Pipeline | Resolution | Frames | Steps | Extra | diff --git a/examples/research_projects/anytext/README.md b/examples/research_projects/anytext/README.md index ddd2f7c6f406..97abf4261d64 100644 --- a/examples/research_projects/anytext/README.md +++ b/examples/research_projects/anytext/README.md @@ -23,10 +23,10 @@ from anytext_controlnet import AnyTextControlNetModel from diffusers.utils import load_image -anytext_controlnet = AnyTextControlNetModel.from_pretrained("tolgacangoz/anytext-controlnet", torch_dtype=torch.float16, +anytext_controlnet = AnyTextControlNetModel.from_pretrained("tolgacangoz/anytext-controlnet", dtype=torch.float16, variant="fp16",) pipe = DiffusionPipeline.from_pretrained("tolgacangoz/anytext", font_path="arial-unicode-ms.ttf", - controlnet=anytext_controlnet, torch_dtype=torch.float16, + controlnet=anytext_controlnet, dtype=torch.float16, trust_remote_code=False, # One needs to give permission to run this pipeline's code ).to("cuda") diff --git a/examples/research_projects/colossalai/README.md b/examples/research_projects/colossalai/README.md index bae223ec290c..5b3df0b4f095 100644 --- a/examples/research_projects/colossalai/README.md +++ b/examples/research_projects/colossalai/README.md @@ -102,7 +102,7 @@ from diffusers import StableDiffusionPipeline import torch model_id = "path-to-save-model" -pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16).to("cuda") +pipe = StableDiffusionPipeline.from_pretrained(model_id, dtype=torch.float16).to("cuda") prompt = "A photo of sks dog in a bucket" image = pipe(prompt, num_inference_steps=50, guidance_scale=7.5).images[0] diff --git a/examples/research_projects/flux_lora_quantization/README.md b/examples/research_projects/flux_lora_quantization/README.md index 71ed28520a8c..aa2d42a6bb00 100644 --- a/examples/research_projects/flux_lora_quantization/README.md +++ b/examples/research_projects/flux_lora_quantization/README.md @@ -122,7 +122,7 @@ import torch ckpt_id = "black-forest-labs/FLUX.1-dev" pipeline = FluxPipeline.from_pretrained( - ckpt_id, text_encoder=None, text_encoder_2=None, torch_dtype=torch.float16 + ckpt_id, text_encoder=None, text_encoder_2=None, dtype=torch.float16 ) pipeline.load_lora_weights("yarn_art_lora_flux_nf4", weight_name="pytorch_lora_weights.safetensors") pipeline.fuse_lora() @@ -147,10 +147,10 @@ nf4_config = BitsAndBytesConfig( transformer = FluxTransformer2DModel.from_pretrained( "fused_transformer", quantization_config=nf4_config, - torch_dtype=bnb_4bit_compute_dtype, + dtype=bnb_4bit_compute_dtype, ) pipeline = AutoPipelineForText2Image.from_pretrained( - ckpt_id, transformer=transformer, torch_dtype=bnb_4bit_compute_dtype + ckpt_id, transformer=transformer, dtype=bnb_4bit_compute_dtype ) pipeline.enable_model_cpu_offload() diff --git a/examples/research_projects/lora/README.md b/examples/research_projects/lora/README.md index 55b870b0bc03..8d6492790f7c 100644 --- a/examples/research_projects/lora/README.md +++ b/examples/research_projects/lora/README.md @@ -73,7 +73,7 @@ from diffusers import StableDiffusionPipeline import torch model_path = "sayakpaul/sd-model-finetuned-lora-t4" -pipe = StableDiffusionPipeline.from_pretrained("CompVis/stable-diffusion-v1-4", torch_dtype=torch.float16) +pipe = StableDiffusionPipeline.from_pretrained("CompVis/stable-diffusion-v1-4", dtype=torch.float16) pipe.unet.load_attn_procs(model_path) pipe.to("cuda") diff --git a/examples/research_projects/multi_subject_dreambooth/README.md b/examples/research_projects/multi_subject_dreambooth/README.md index 3415a63c1063..eeb06719ce5f 100644 --- a/examples/research_projects/multi_subject_dreambooth/README.md +++ b/examples/research_projects/multi_subject_dreambooth/README.md @@ -142,7 +142,7 @@ from diffusers import StableDiffusionPipeline import torch model_id = "path-to-your-trained-model" -pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16).to("cuda") +pipe = StableDiffusionPipeline.from_pretrained(model_id, dtype=torch.float16).to("cuda") prompt = "A photo of a t@y person petting an sks dog" image = pipe(prompt, num_inference_steps=200, guidance_scale=7.5).images[0] diff --git a/examples/research_projects/multi_token_textual_inversion/README.md b/examples/research_projects/multi_token_textual_inversion/README.md index 5e5d4991cb5b..18e733512657 100644 --- a/examples/research_projects/multi_token_textual_inversion/README.md +++ b/examples/research_projects/multi_token_textual_inversion/README.md @@ -102,7 +102,7 @@ Once you have trained a model using above command, the inference can be done sim from diffusers import StableDiffusionPipeline model_id = "path-to-your-trained-model" -pipe = StableDiffusionPipeline.from_pretrained(model_id,torch_dtype=torch.float16).to("cuda") +pipe = StableDiffusionPipeline.from_pretrained(model_id,dtype=torch.float16).to("cuda") prompt = "A backpack" diff --git a/examples/research_projects/promptdiffusion/README.md b/examples/research_projects/promptdiffusion/README.md index 3df04eb7a361..9be97b52ffd2 100644 --- a/examples/research_projects/promptdiffusion/README.md +++ b/examples/research_projects/promptdiffusion/README.md @@ -34,9 +34,9 @@ query = ImageOps.invert(load_image("https://github.com/Zhendong-Wang/Prompt-Diff # load prompt diffusion controlnet and prompt diffusion -controlnet = PromptDiffusionControlNetModel.from_pretrained("iczaw/prompt-diffusion-diffusers", subfolder="controlnet", torch_dtype=torch.float16) +controlnet = PromptDiffusionControlNetModel.from_pretrained("iczaw/prompt-diffusion-diffusers", subfolder="controlnet", dtype=torch.float16) model_id = "path-to-model" -pipe = PromptDiffusionPipeline.from_pretrained("iczaw/prompt-diffusion-diffusers", subfolder="base", controlnet=controlnet, torch_dtype=torch.float16, variant="fp16") +pipe = PromptDiffusionPipeline.from_pretrained("iczaw/prompt-diffusion-diffusers", subfolder="base", controlnet=controlnet, dtype=torch.float16, variant="fp16") # speed up diffusion process with faster scheduler and memory optimization pipe.scheduler = UniPCMultistepScheduler.from_config(pipe.scheduler.config) diff --git a/examples/research_projects/pytorch_xla/training/text_to_image/README.md b/examples/research_projects/pytorch_xla/training/text_to_image/README.md index f99ab124864e..e4419936cfa8 100644 --- a/examples/research_projects/pytorch_xla/training/text_to_image/README.md +++ b/examples/research_projects/pytorch_xla/training/text_to_image/README.md @@ -141,7 +141,7 @@ def main(): model_path = "jffacevedo/pxla_trained_model" pipe = StableDiffusionPipeline.from_pretrained( model_path, - torch_dtype=torch.bfloat16 + dtype=torch.bfloat16 ) pipe.to(device) prompt = ["A naruto with green eyes and red legs."] diff --git a/examples/t2i_adapter/README_sdxl.md b/examples/t2i_adapter/README_sdxl.md index 0a3b5e33d46a..6097b612fda4 100644 --- a/examples/t2i_adapter/README_sdxl.md +++ b/examples/t2i_adapter/README_sdxl.md @@ -101,9 +101,9 @@ import torch base_model_path = "stabilityai/stable-diffusion-xl-base-1.0" adapter_path = "path to adapter" -adapter = T2IAdapter.from_pretrained(adapter_path, torch_dtype=torch.float16) +adapter = T2IAdapter.from_pretrained(adapter_path, dtype=torch.float16) pipe = StableDiffusionXLAdapterPipeline.from_pretrained( - base_model_path, adapter=adapter, torch_dtype=torch.float16 + base_model_path, adapter=adapter, dtype=torch.float16 ) # speed up diffusion process with faster scheduler and memory optimization diff --git a/examples/text_to_image/README.md b/examples/text_to_image/README.md index 4718d59fcfcb..d51a04414137 100644 --- a/examples/text_to_image/README.md +++ b/examples/text_to_image/README.md @@ -106,7 +106,7 @@ import torch from diffusers import StableDiffusionPipeline model_path = "path_to_saved_model" -pipe = StableDiffusionPipeline.from_pretrained(model_path, torch_dtype=torch.float16) +pipe = StableDiffusionPipeline.from_pretrained(model_path, dtype=torch.float16) pipe.to("cuda") image = pipe(prompt="yoda").images[0] @@ -120,9 +120,9 @@ import torch from diffusers import StableDiffusionPipeline, UNet2DConditionModel model_path = "path_to_saved_model" -unet = UNet2DConditionModel.from_pretrained(model_path + "/checkpoint-/unet", torch_dtype=torch.float16) +unet = UNet2DConditionModel.from_pretrained(model_path + "/checkpoint-/unet", dtype=torch.float16) -pipe = StableDiffusionPipeline.from_pretrained("", unet=unet, torch_dtype=torch.float16) +pipe = StableDiffusionPipeline.from_pretrained("", unet=unet, dtype=torch.float16) pipe.to("cuda") image = pipe(prompt="yoda").images[0] @@ -251,7 +251,7 @@ from diffusers import StableDiffusionPipeline import torch model_path = "sayakpaul/sd-model-finetuned-lora-t4" -pipe = StableDiffusionPipeline.from_pretrained("CompVis/stable-diffusion-v1-4", torch_dtype=torch.float16) +pipe = StableDiffusionPipeline.from_pretrained("CompVis/stable-diffusion-v1-4", dtype=torch.float16) pipe.unet.load_attn_procs(model_path) pipe.to("cuda") @@ -271,7 +271,7 @@ lora_model_id = "sayakpaul/sd-model-finetuned-lora-t4" card = RepoCard.load(lora_model_id) base_model_id = card.data.to_dict()["base_model"] -pipe = StableDiffusionPipeline.from_pretrained(base_model_id, torch_dtype=torch.float16) +pipe = StableDiffusionPipeline.from_pretrained(base_model_id, dtype=torch.float16) ... ``` diff --git a/examples/text_to_image/README_sdxl.md b/examples/text_to_image/README_sdxl.md index 6fb10ec9e1b3..0e8e31fa2b51 100644 --- a/examples/text_to_image/README_sdxl.md +++ b/examples/text_to_image/README_sdxl.md @@ -88,7 +88,7 @@ from diffusers import DiffusionPipeline import torch model_path = "you-model-id-goes-here" # <-- change this -pipe = DiffusionPipeline.from_pretrained(model_path, torch_dtype=torch.float16) +pipe = DiffusionPipeline.from_pretrained(model_path, dtype=torch.float16) pipe.to("cuda") prompt = "A naruto with green eyes and red legs." @@ -275,7 +275,7 @@ from diffusers import DiffusionPipeline import torch model_path = "takuoko/sd-naruto-model-lora-sdxl" -pipe = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16) +pipe = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.float16) pipe.to("cuda") pipe.load_lora_weights(model_path) diff --git a/examples/textual_inversion/README.md b/examples/textual_inversion/README.md index 528c7962e38f..ddca93dee37f 100644 --- a/examples/textual_inversion/README.md +++ b/examples/textual_inversion/README.md @@ -107,7 +107,7 @@ from diffusers import StableDiffusionPipeline import torch model_id = "path-to-your-trained-model" -pipe = StableDiffusionPipeline.from_pretrained(model_id,torch_dtype=torch.float16).to("cuda") +pipe = StableDiffusionPipeline.from_pretrained(model_id,dtype=torch.float16).to("cuda") repo_id_embeds = "path-to-your-learned-embeds" pipe.load_textual_inversion(repo_id_embeds) diff --git a/examples/textual_inversion/README_sdxl.md b/examples/textual_inversion/README_sdxl.md index c971ee2a0f39..d891273e4865 100644 --- a/examples/textual_inversion/README_sdxl.md +++ b/examples/textual_inversion/README_sdxl.md @@ -35,7 +35,7 @@ from diffusers import StableDiffusionXLPipeline import torch model_id = "./textual_inversion_cat_sdxl" -pipe = StableDiffusionXLPipeline.from_pretrained(model_id,torch_dtype=torch.float16).to("cuda") +pipe = StableDiffusionXLPipeline.from_pretrained(model_id,dtype=torch.float16).to("cuda") prompt = "A backpack" From 93625ca10be6057843505ea1b13ad3fa813aca10 Mon Sep 17 00:00:00 2001 From: stevhliu Date: Tue, 28 Jul 2026 18:17:26 -0700 Subject: [PATCH 3/3] Revert "docs" This reverts commit 04ca80543f1ec30eb1cd02a15b10e17e081aaad6. --- .ai/AGENTS.md | 2 -- docs/source/en/conceptual/contribution.md | 15 ++++++++++----- 2 files changed, 10 insertions(+), 7 deletions(-) diff --git a/.ai/AGENTS.md b/.ai/AGENTS.md index ae611feb4bb6..18c9d0215fd0 100644 --- a/.ai/AGENTS.md +++ b/.ai/AGENTS.md @@ -37,8 +37,6 @@ Strive to write code as simple and explicit as possible. Task-specific guides live in `.ai/skills/` and are loaded on demand by AI agents. Available skills include: - [model-integration](./skills/model-integration/SKILL.md) (adding/converting pipelines) -- [custom-blocks](./skills/custom-blocks/SKILL.md) (packaging a `ModularPipelineBlocks` subclass for the Hub) -- [diffusers-cli](./skills/diffusers-cli/SKILL.md) (running pipelines, inspecting schemas, and using the Diffusers CLI) - [self-review](./skills/self-review/SKILL.md) (pre-PR self-review against the project rules) ## Self-review before a PR diff --git a/docs/source/en/conceptual/contribution.md b/docs/source/en/conceptual/contribution.md index f9d377907e76..7f7d8bd77f29 100644 --- a/docs/source/en/conceptual/contribution.md +++ b/docs/source/en/conceptual/contribution.md @@ -589,8 +589,13 @@ The repository keeps AI-agent configuration in [`.ai/`](https://github.com/huggi ### AI-assisted and agentic contributions -AI agents are welcome tools for contributing to Diffusers. We encourage you to set up your agent with the Diffusers [agent guide](https://github.com/huggingface/diffusers/blob/main/.ai/AGENTS.md) and use the relevant task-specific skills, such as `model-integration` and `self-review`. Run `make codex` or `make claude` to use the skills with your agent, then follow the guide to scope, implement, test, and review your contribution. You remain responsible for understanding, testing, and maintaining the changes in your PR. - -If you are a model author or part of a team that officially maintains a model, we encourage you to use agents for a new model integration. Follow the repository's [recommended setup](https://github.com/huggingface/diffusers/blob/main/.ai/AGENTS.md) and use the [`model-integration`](https://github.com/huggingface/diffusers/blob/main/.ai/skills/model-integration/SKILL.md) skill. Coordinate the scope with maintainers before opening a PR. - -If you are contributing a model to Diffusers for the first time as a community contributor, we generally recommend starting with a custom implementation that loads code from the Hub. This gives users access to the model while its integration into the core library is evaluated. See the [custom models](../using-diffusers/automodel) and [custom modular blocks](../modular_diffusers/custom_blocks) guides for supported patterns. +AI-assisted contributions are welcome, but they must be coordinated, scoped, and verified to keep review load manageable. PRs that do not follow these guidelines may be closed without detailed review. + +- **Coordinate before opening a PR.** Find or open an issue, review similar PRs (open and recently closed), and wait for an explicit acknowledgment from a maintainer on that issue before opening a PR. This gives us a chance to discuss scope, avoid duplicate work, and confirm the approach. +- **Fix patterns, not one-offs.** If you spot an recurring issue, search the codebase for similar instances and open a *single* issue with a clear, systematic scope (e.g. "fix mutable defaults across all schedulers") rather than many issues or PRs for individual instances. +- **Self-review before opening.** Run the [`self-review`](https://github.com/huggingface/diffusers/blob/main/.ai/skills/self-review/SKILL.md) skill — it reviews your diff against [`.ai/review-rules.md`](https://github.com/huggingface/diffusers/blob/main/.ai/review-rules.md), the same rubric the `@claude` CI reviewer uses — and address what it reports — it's a helper, not authoritative, and can be wrong. Focus on the blocking issues that make sense to you, and clean up dead/unused code as much as possible. If you disagree with a suggestion, it's fine to leave it for the reviewer to discuss after the PR is opened — the notes you share (see below) tell the reviewer it was a deliberate call. +- **Share your self-review notes.** Please post the final self-review report — the round that reflects the diff you're submitting — on the PR, in the description or as a comment, including findings you intentionally did not fix and why. It helps the reviewer see what has already been checked and which calls were deliberate, and usually saves a few rounds of back-and-forth. +- **Include in the PR description:** + - A **coordination link** to the issue or discussion where a maintainer acknowledged the work. + - The **test commands you ran** and their results (paste relevant output, not just "tests pass"). + - Your **self-review notes** (or a link to the PR comment containing them), as described above.