[Feature request] Integrate Deforum

[HanJammer]

It would be extremally cool if Deforum would be integrated as a method of creating txt2vid https://github.com/deforum/stable-diffusion

hlky's webui (https://github.com/sd-webui/stable-diffusion-webui) already has txt2vid integrated in the streamlit version of the UI although it's not nearly as good as deforum's implementation.

[ClashSAN]

Would these custom scripts do the same thing?
https://github.com/yownas/seed_travel (same prompt, shows transition variations between 2 seeds)
https://github.com/memes-forever/Stable-diffusion-webui-video (repeated img2img)
https://pastebin.com/QZc3dnPx (untested) #792
Other video possibilities: steps, img2img real video frames, cfg, prompt travel (https://github.com/nateraw/stable-diffusion-videos)

[HanJammer]

Would these custom scripts do the same thing? https://github.com/yownas/seed_travel (same prompt, shows transition variations between 2 seeds) https://github.com/memes-forever/Stable-diffusion-webui-video (repeated img2img) https://pastebin.com/QZc3dnPx (untested) #792 Other video possibilities: steps, img2img real video frames, cfg, prompt travel (https://github.com/nateraw/stable-diffusion-videos)

Check these example Deforum videos:
https://www.youtube.com/watch?v=YdK4Wo5zWSc
https://www.youtube.com/watch?v=qXghXywioeM
also this if you have FB: https://www.facebook.com/groups/stablediffusionuniverse/permalink/785172502715602

You will see that it's way ahead of simple img2img (which Deforum also has), prompt travel or seed transitions other implementations offer...

[cmp-nct]

Would these custom scripts do the same thing?
Looks both very interesting.
Seed travel uses a python library to make a movie from images, the second one uses ffmpeg as a process execution which is kinda ugly.

So if we wish to generate movies the method of seed travel looks clean.

[HanJammer]

img2img real video frames -> it's useful but not for fully "imaginary" videos...

I will test memes-forever and yownas although I already can see it's hardly as comprehensive as Deforum which allows you to zoom-pan-rotate even in pseudo-3D space with variable speed skip between various prompts and so on. But it's GUI is build on jupyter notebook running in the Collab session and only way to run it locally is by connecting Collab to your local machine which is sub-optimal... Also problem with Deforum is that it's hard to configure the way it's GUI is built (but I think that this could be remediated by the Gradio).

[HanJammer]

With the additional scripts released since I posted it I think I can close this as done (it's still not deforum-grade functionality, but it's close).

[ameen-roayan]

Side question kind of, How do we update automatic1111`s version is it automated with every launch or there is something that needs to be done ?

[Slamsneider]

if you have git then rightclick the folder and choose "git sync... and then 'pull'" " if no git you have to re-download.

[jet3004]

With the additional scripts released since I posted it I think I can close this as done (it's still not deforum-grade functionality, but it's close).

Which ones have come close in your eyes?

[ClashSAN]

@jet3004 He probably is referring to animator
https://github.com/Animator-Anon/Animator

[everyoneishappy]

There is also a local version of deforum here: https://github.com/HelixNGC7293/DeforumStableDiffusionLocal

[HanJammer]

Yes, It would be super nice to have a GUI for it, configuring everything in the text files is pretty hard. Also the scripts available for 1111 GUI only partially cover what's possible with Deforum (ie. 3D "camera" movement or transitions between prompts are not available yet) so I guess this feature request should still remain open after all...

[scorbon]

They've made deforum as an add on script for this distro now. Should it be merged? https://github.com/deforum-art/deforum-for-automatic1111-webui

[HanJammer]

Excellent. I can't wait to test it. So I guess we can safely assume this feature request is fulfilled now.