DeepFloyd?

[nam37]

This might be a silly question since I haven't looked in to the technologies involved, but is it possible that DeepFloyd couple be configured to use the AUTOMATIC1111 SD webui?

[gitmylo]

It could be implemented as a plugin, but I don't see why it would be in the main repository.

[gitmylo]

Do note that deepfloyd uses a lot of VRAM, I did get it running on 12gb. But it was using 100% of it. And if you want to not unload models between every run, you're going to need a lot of RAM.

[Fetzii]

@gitmylo
Did you do anything special to make it run on 12GB of VRAM? I would like to try it, but I also got only 12GB of VRAM

[gitmylo]

I used the diffusers example
on pytorch 2 with cuda 117, skip the parts with enable_xformers_memory_efficient_attention() (i know pytorch 2 will run on 12gb vram like this though)
make sure you call the enable_model_cpu_offload(). otherwise you won't be able to fit the models in your gpu. this way it will move them to cpu. (Note: this uses a lot of cpu ram. i upgraded to 48gb because my pc had stutters on 16gb due to running out of ram. however, it did work still)

just follow the steps in the example, skip stage 3, not needed.

one image i generated:
(A robot holding a sign that says: "Diffusion")

Side note

If you really want it as a webui extension, i can do that. But it will be the bare minimum to get it running. So don't expect a lot of features.

[Fetzii]

Thx for the hints!

It would be real great to have it as an extension. I think it will make DeepFloyd much more available to the masses and therefore it might get more attention, which is always good for an amazing open source project like DeepFloyd :)

[djo267]

DeepFloyd IF is now out. Would love to see this as a plugin. https://deepfloyd.ai/deepfloyd-if

[clemenslindner]

how to run it on the webui tho?

[sekkit]

Any information about the extension?

[darkzbaron]

Same here!

[Ainaemaet]

With the speed at which Automatic1111 and extensions are developed, I'm really surprised to see there is no DeepFloyd IF support yet

[dhwz]

Well the it's named "stable diffusion webui" not "deepfloyd if webui" :-)

[Ainaemaet]

Not sure if you're new to Automatic1111, but in the last year it's become MUCH more than just a 'stable diffusion webui' - and is somewhat of a Swiss Army Knife of ai image generation; and I'm far from the only person that speculated about the possibility of it being included as a plugin, if not in the main repo during that time.

Also, you're replying to a nearly 60-day-old comment lol.

(not trying to be rude, I know you're own comment was light-hearted so please see mine the same :) ).

[DiamondGlassDrill]

So no one working on it?