changes to get dream.py working on M1

[toffaletti]

• use MPS specific fork of k_diffusion
• move all device init logic to T2I.init
• handle m1 specific edge case with autocast device type
• check torch.cuda.is_available before using cuda

Besides these changes, there is still an open question about what to do will the requirement to use --full_precision for M1. See discussion here for details: #237 This is now addressed in the second commit.

[junukwon7]

Thanks! Confirmed it works for me. (Clean install, M1 Max MBP, miniforge environment)

[magnusviri]

@Birch-san's k-diffusion prevents conda env create from finishing on my computer. I've got miniconda. I know others have gotten it to work so I'll have to debug it.

dream.py runs w/o crashing but produces black images, probably because of k-difusion.

[Birch-san]

@magnusviri

@Birch-san's k-diffusion prevents conda env create from finishing on my computer. I've got miniconda.

Try this:
#63 (comment)

[cvakiitho]

I confirm this works out of the box, for some reason I could make it work by just running conda env update -f environment-mac.yaml - server then just hanged on image generation with no error.
But after a real clean conda env it works flawlessly on my m1 max.

[magnusviri]

I have it working now. I'll add some information to the README-Mac to address the hang. The seed isn't repeatable but I didn't see a quick and easy fix and so I say we just commit this. I'm afraid if it sticks around for long that, at least for me, it will get even more confusing because everything is moving so fast. I'm not sure how to get it merged though.

[magnusviri]

I get to leave a review. I should've put that comment here. So I will copy-paste it.

I have it working now. I'll add some information to the README-Mac to address the hang. The seed isn't repeatable but I didn't see a quick and easy fix and so I say we just commit this. I'm afraid if it sticks around for long that, at least for me, it will get even more confusing because everything is moving so fast. I'm not sure how to get it merged though.

[lstein]

Great job everyone. This was a bear!

[lstein]

Crashes on CUDA systems during image generation with "expected scalar type Half but found Float". The problem is this line in choose_autocast_device():

    if device_type != 'cuda' or device_type != 'cpu':
        return 'cpu'

Since the device can't be both cuda and cpu, this will always return "cpu". The fix is trivial, and I'll take care of it, but the take-home message is to always run the thing to make sure it is working! (I don't always follow that advice myself, so I know how it is).

[magnusviri]

@lstein I only have a 2GB CUDA capable GPU and it's currently not installed in anything. It was easier for me to use my Mac :)

However, I've actually been thinking of setting up GitHub actions. I don't think they provide GPU's though. Maybe they will because this is such a cool project.

[toffaletti]

Crashes on CUDA systems during image generation with "expected scalar type Half but found Float". The problem is this line in choose_autocast_device():

    if device_type != 'cuda' or device_type != 'cpu':
        return 'cpu'

Since the device can't be both cuda and cpu, this will always return "cpu". The fix is trivial, and I'll take care of it, but the take-home message is to always run the thing to make sure it is working! (I don't always follow that advice myself, so I know how it is).

doh! should have been and. this is such a common mistake pattern that its a linter error in lots of languages...