Fixed noise_pred_text referenced before assignment.

[LagPixelLOL]

Fixed local variable noise_pred_text referenced before assignment when using PAG with guidance scale and guidance rescale at the same time.

@yiyixuxu @asomoza

I'm not sure if allowing guidance rescale with PAG + guidance scale is actually a thing, but from my tests it does work with PAG + guidance scale. If it's not allowed then it can be fixed with adding a condition or raising an error when all PAG scale, guidance scale, and guidance rescale are passed at the same time.

Before this fix it currently does raise an error when passing all 3 at the same time, but it's an unbounded local variable error, which is extremely confusing.

[LagPixelLOL]

@sayakpaul

[sayakpaul]

Thanks for your PR, could you provide a reproducible snippet that raises the error?

[sayakpaul]

I am also going to cc @sunovivid (PAG first author) to brainstorm if allowing guidance rescale with PAG + guidance scale makes sense.

[LagPixelLOL]

import torch
import diffusers

pipeline = diffusers.AutoPipelineForText2Image.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.bfloat16, enable_pag=True)
pipeline.to("cuda")
pipeline(
    prompt="test",
    guidance_scale=5,
    guidance_rescale=0.5,
    pag_scale=3,
    num_inference_steps=35,
)

produces the below error

Traceback (most recent call last):
  File "/root/differr.py", line 6, in <module>
    pipeline(
  File "/usr/local/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/diffusers/pipelines/pag/pipeline_pag_sd_xl.py", line 1250, in __call__
    noise_pred = rescale_noise_cfg(noise_pred, noise_pred_text, guidance_rescale=self.guidance_rescale)
                                               ^^^^^^^^^^^^^^^
UnboundLocalError: cannot access local variable 'noise_pred_text' where it is not associated with a value

[yiyixuxu]

same issue reported here #9220

fix looks good to me!

[yiyixuxu]

can you run make style and make fix-copies?

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[sunovivid]

Thank you for the contributions and experiments!

I am also going to cc @sunovivid (PAG first author) to brainstorm if allowing guidance rescale with PAG + guidance scale makes sense.

It looks reasonable to use guidance rescale with PAG + guidance scale. Noise rescale modifies the scale of the 'guided' prediction to the original text-conditioned prediction to cut off extra scale from guidance extrapolation. So rescaling PAG + CFG guided prediction to the original prediction looks reasonable. After the cut-off, the noise rescale function mixes the rescaled noise with the original 'text' prediction. It seems also reasonable because our main component is 'text' prediction, both in CFG and PAG. I think it is a very good fix!

cc @sayakpaul

[LagPixelLOL]

The style is fixed now.

[yiyixuxu]

seems like we broke the tests; I looked up the import of PAGMixin on github, not many but some so let's try to not making a breaking change here

we can add a argument return_pred_text; default to False but set True when calling from our pipelines.

[LagPixelLOL]

Should be fixed now.

[yiyixuxu]

@LagPixelLOL merged! thanks for your PR!