Promt-to-Prompt implementation

[DrR0bot]

No more prompt engineering.
You can now edit images by editing their captions. No more prompt engineering or thousands tryouts.

https://twitter.com/AbermanKfir/status/1581553590472097794?t=jVmYVITAGEfyruy1cpg9oQ&s=19

[Kosinkadink]

Gonna try to make a script that incorporates this, fingers crossed.

[kamyker]

https://github.com/bloc97/CrossAttentionControl

[illtellyoulater]

any news on this?

[jtkelm2]

Can anyone say whether this prompt-to-prompt/attention control business is really any better than txt2mask inpainting or the "img2img alternative" scripts from a few weeks ago? From what I've seen of the example pics, it doesn't really look like it.

[Kosinkadink]

https://arxiv.org/abs/2208.01626

This is the research paper this is based on. It is a fundamentally different technique where the observation that the attention map can be used to automatically be used to mask an area of interest is exploited to use the attention map of one prompt as part of a mask for a different one, looping that modified attention map forward step by step. As a consequence, it essentially generates two images at the same time, and the research paper suggests implementing them as a batch to minimize performance loss.

The paper drew my attention about a week ago, but there was only one project on GitHub that tried to implement it for Stable Diffusion. The code linked here should help me implement a more general solution that the first piece of code did not.

[chyld]

Link to the Google repo

https://github.com/google/prompt-to-prompt

[apolinario]

Since this is implemented using diffusers, I wonder if #2577 would make sense here too

[ZeroCool22]

This would be awesome!