Skip to content

Update sdxl reference pipeline to latest sdxl pipeline #9938

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Nov 28, 2024
+518 −139
Merged

Update sdxl reference pipeline to latest sdxl pipeline #9938

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
1f2f154
Update sdxl reference community pipeline
dimitribarbot Nov 16, 2024
b90405c
Update README.md
dimitribarbot Nov 16, 2024
62c2cae
Style & quality
dimitribarbot Nov 16, 2024
a819287
Use example images from huggingface documentation-images repository
dimitribarbot Nov 16, 2024
18d8795
Merge branch 'main' into update-sdxl-reference
sayakpaul Nov 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 11 additions & 10 deletions examples/community/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2619,16 +2619,17 @@ for obj in range(bs):

### Stable Diffusion XL Reference

This pipeline uses the Reference. Refer to the [stable_diffusion_reference](https://github.com/huggingface/diffusers/blob/main/examples/community/README.md#stable-diffusion-reference).
This pipeline uses the Reference. Refer to the [Stable Diffusion Reference](https://github.com/huggingface/diffusers/blob/main/examples/community/README.md#stable-diffusion-reference) section for more information.

```py
import torch
from PIL import Image
# from diffusers import DiffusionPipeline
from diffusers.utils import load_image
from diffusers import DiffusionPipeline
from diffusers.schedulers import UniPCMultistepScheduler

input_image = load_image("https://hf.co/datasets/huggingface/documentation-images/resolve/main/diffusers/input_image_vermeer.png")
from .stable_diffusion_xl_reference import StableDiffusionXLReferencePipeline

input_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/sdxl_reference_input_cat.jpg")

# pipe = DiffusionPipeline.from_pretrained(
# "stabilityai/stable-diffusion-xl-base-1.0",
Expand All @@ -2646,22 +2647,22 @@ pipe = StableDiffusionXLReferencePipeline.from_pretrained(
pipe.scheduler = UniPCMultistepScheduler.from_config(pipe.scheduler.config)

result_img = pipe(ref_image=input_image,
prompt="1girl",
prompt="a dog",
num_inference_steps=20,
reference_attn=True,
reference_adain=True).images[0]
```

Reference Image

![reference_image](https://hf.co/datasets/huggingface/documentation-images/resolve/main/diffusers/input_image_vermeer.png)
![reference_image](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/sdxl_reference_input_cat.jpg)

Output Image

`prompt: 1 girl`
`prompt: a dog`

`reference_attn=True, reference_adain=True, num_inference_steps=20`
![Output_image](https://github.com/zideliu/diffusers/assets/34944964/743848da-a215-48f9-ae39-b5e2ae49fb13)
`reference_attn=False, reference_adain=True, num_inference_steps=20`
![Output_image](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/sdxl_reference_adain_dog.png)

Reference Image
![reference_image](https://github.com/huggingface/diffusers/assets/34944964/449bdab6-e744-4fb2-9620-d4068d9a741b)
Expand Down Expand Up @@ -4696,4 +4697,4 @@ with torch.no_grad():
```

In the folder examples/pixart there is also a script that can be used to train new models.
Please check the script `train_controlnet_hf_diffusers.sh` on how to start the training.
Please check the script `train_controlnet_hf_diffusers.sh` on how to start the training.
Loading
Loading