chore: support python 3.12, torch 2.6.0, clean up build/deps

[psychedelicious]

Summary

This PR includes all changes from #7847. I cherry-picked the changes from that PR's fork on to the origin so that I could do some test PyPI builds.

I cannot get MPS to generate - get an max recursion depth error when trying to load models.

very large traceback

I get the same error on python 3.12 and 3.11.

Related Issues / Discussions

QA Instructions

Merge Plan

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Tests added / updated (if applicable)
• Documentation added / updated (if applicable)
• Updated What's New copy (if doing a release after this PR)

[psychedelicious]

I think I’ve narrowed down the issue. It’s not related to the python or torch version, it’s transformers.

Starting with transformers==4.50.0 and through to latest 4.50.3, diffusers-format SD 1.5 & SDXL models fail to load text encoders. We use transformers for this. The issue occurs on main if you update transformers to an affected version.

This happens in the compel and sdxl_compel_prompt nodes.

Other models (e.g. unet) may also fail to load, but I can’t get past this to try loading them easily. I’m also running into unexpected OOMs, but that also happens on main so I don’t think it’s related. Might just be my macbook.

I suspect we will need to raise the root issue w/ transformers, so I did a bisect-kinda thing on the transformers dependency and identified the range of commits that is causing the problem.

Last working commit: huggingface/transformers@981c276

The next commit introduces an python module import error which prevents the app from loading, so I'm not sure if the model loading error will occur.

• Commit: huggingface/transformers@1603018
• From PR #36335: Update form pretrained to make TP a first class citizen

The import error appears to be related to an interaction with transformers and diffusers.

Then, 10 or so commits later, this commit fixes the import error, and I can run the app. It is the first commit where the app loads and the model loading issue occurs. It's not clear if this specific commit introduced the model loading error.

• Commit: huggingface/transformers@a40f1ac
• From PR #36453: Fix couples of issues from #36335

Odds are we will need to take this up with transformers to get it fixed so we can continue to keep the package updated. I'll make a minimal repro tomorrow and raise the issue w/ transformers.

But for now, I believe pinning transformers==4.49.0 fixes the issue.

[hipsterusername]

Is it that there's a change that needs to be made in the compel/loading logic based on something updated in Transformers? Or do you think this was an unintentional regression?

[Vargol]

I've upgraded my Diffuser venv to the latest transformers and run an old SDXL with compel script and it at least gets as far as denoising. I've got other stuff using the GPU up so haven't run it to completion.

$ pip show torch
Name: torch
Version: 2.6.0

$ pip show transformers
Name: transformers
Version: 4.50.3

$ python sdxl_compel.py
Loading pipeline components...: 100%|█████████████████████████████████████████████████████| 3/3 [00:10<00:00, 3.63s/it]
3%|██▎ | 1/36 [00:20<11:44, 20.12s/it

Here's the script so you can see if there's anything different to they way you load the encoder.

sdxl_compel.txt

[psychedelicious]

Found the problem:

InvokeAI/invokeai/backend/util/mps_fixes.py

Line 7 in ae45fc8

torch.empty = torch.zeros

So, question is, do we need this still... and, for that matter, do we need any of this file any more?

[gogurtenjoyer]

I can verify that we do not need the MPS fixes with Torch 2.6.0 - these fixes were so that black images aren't produced on MPS when using float16 or bfloat16. I just checked, and the black images only appear when using attention_type: sliced which isn't the default anymore anyhow (I believe it's torch-sdp, which works great).

[psychedelicious]

Changes:

• Add pins.json which records the requisite python version and PyPI torch index URLs to the repo. The launcher will fetch these items and use them during installation. This allows the repo to take responsibility for stuff, and handle bumping versions. We do not need to update the launcher to handle changes to python/torch versions.
• Restore the setuptools configuration - not sure why this was removed, but it prevents the wheel from building.
• Remove all the OG controlnet processor nodes. These have been deprecated since Sept 2024 (~7 months) and hidden from the UI since then. Workflows from before that time will break but it's a worthy price to pay to get rid of the controlnet_aux, which has caused many hours of anguish.
• Remove a crapload of unused dependencies. I carefully reviewed each dependency's documentation and tested related functionality. Everything working.

[psychedelicious]

I'm on a spring cleaning rampage.

I am merging #7881 into this PR. It removes the installer scripts from the repo and updates our CI accordingly. It's been manually e2e tested.