IP-Adapter for StableDiffusion3Img2ImgPipeline

[guiyrt]

What does this PR do?

Extends SD3 IP-Adapter implementation to StableDiffusion3Img2ImgPipeline, part of #9966. Some outputs for reference:

Inference code

import torch

from diffusers import StableDiffusion3Img2ImgPipeline
from diffusers.utils import load_image
from transformers import SiglipVisionModel, SiglipImageProcessor

model_id = "stabilityai/stable-diffusion-3.5-large"
image_encoder_id = "google/siglip-so400m-patch14-384"
ip_adapter_id = "guiyrt/InstantX-SD3.5-Large-IP-Adapter-diffusers"

feature_extractor = SiglipImageProcessor.from_pretrained(
    image_encoder_id, torch_dtype=torch.float16
)

image_encoder = SiglipVisionModel.from_pretrained(
    image_encoder_id, torch_dtype=torch.float16
)

pipe = StableDiffusion3Img2ImgPipeline.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    feature_extractor=feature_extractor,
    image_encoder=image_encoder,
)

# Load IP Adapter
pipe.load_ip_adapter(ip_adapter_id)
pipe.set_ip_adapter_scale(0.75)
pipe._exclude_from_cpu_offload.append("image_encoder")
pipe.enable_sequential_cpu_offload()

# Input
source_image = load_image("source.jpg")
ip_adapter_img = load_image("ip_adapter_img.png")

# please note that SD3.5 Large is sensitive to highres generation like 1536x1536
images = pipe(
    image=source_image,
    width=1024,
    height=1024,
    prompt="an astronaut",
    negative_prompt="lowres, low quality, worst quality",
    num_images_per_prompt=4,
    generator=torch.manual_seed(42),
    ip_adapter_image=ip_adapter_img,
    guidance_scale=6,
    num_inference_steps=60,
    strength=0.65,
).images

for i, image in enumerate(images):
    image.save(f"result_{i}.jpg")

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline?
• Did you read our philosophy doc (important for complex PRs)?
• Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

@hlky @yiyixuxu

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[hlky]

Thanks @guiyrt