PLMS with Highres. Fix Enabled

[amarand]

I've read through all of the Issues relating to PLMS with "Highres. Fix" enabled. Decided not to create another issue because this has been reported so many times.

I realize the problem is directly related to the way "Highres. Fix" feature works: it feeds the initially created image back into img2img to fix the faces. Cool.

I've seen a few folks say that "PLMS is not supported in img2img" however, when I look at the upstream (CompVis), it looks like perhaps PLMS is supported in img2img as of at least "Latest commit [2ff270f] on Aug 10"

https://github.com/CompVis/stable-diffusion/blob/main/scripts/img2img.py

from ldm.models.diffusion.plms import PLMSSampler
z_enc = sampler.stochastic_encode(init_latent, torch.tensor([t_enc]*batch_size).to(device))

The error we're seeing is:

AttributeError: 'PLMSSampler' object has no attribute 'stochastic_encode'

But I'm seeing the attribute listed in img2img.py upstream. Is this code that hasn't been reintegrated into this branch yet?

Or...is this just a bug in img2img? Like, it should work, but it's not working. Or are we operating on a version older than this?

Sorry if I'm confused. I've just seen a lot of the issues closed with various tags saying img2img doesn't work with PLMS, but it seems like it should (at least it appears to do so in the code), and even the attribute "stochastic_encode" is visible in the .py code upstream.

Thoughts?

[amarand]

I may have answered my own question....

There are a few copies of img2img.py in the auto1111 git repo:

modules/img2img.py
scripts/img2imgalt.py
a pipeline copy

And then...

img2img.py

under repositories/stable-diffusion

I've noticed that the modules and scripts copies do not have the PLMS but the repositories/stable-diffusion copy contains it.

So auto1111 includes the current upstream version of img2img.py, but I'm assuming that's as a reference?

I guess the better question is (and, again, please correct me if I'm wrong): when will PLMS support be integrated into img2img under auto1111?

I think a lot of folks use the "Highres. Fix" feature, because it allows you to search the seed space randomly for fun stuff, at lower resolution, but then, when you want to upscale it properly, without Highres. Fix feature, you're generating an entirely different (wildly different) image with the same seed.