Add support for ControlNet in Flux controlnet img2img #9979

[hhsparthipan]

Hi!
This PR brings support of Xlabs Controlnets in the flux img2img pipeline

The request: #9979

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.
@yiyixuxu

How to use

Here is the example of code:

`import numpy as np
import torch
import cv2
from PIL import Image
from diffusers.utils import load_image
from diffusers import FluxControlNetImg2ImgPipeline, FluxControlNetPipeline
from diffusers import FluxControlNetModel
from controlnet_aux import HEDdetector

base_model = "black-forest-labs/FLUX.1-dev"
controlnet_model = "Xlabs-AI/flux-controlnet-hed-diffusers"
controlnet = FluxControlNetModel.from_pretrained(
controlnet_model,
torch_dtype=torch.bfloat16,
use_safetensors=True,
)
pipe = FluxControlNetImg2ImgPipeline.from_pretrained(
base_model, controlnet=controlnet, torch_dtype=torch.bfloat16
)
pipe.load_lora_weights("strangerzonehf/Flux-Midjourney-Mix2-LoRA")

pipe.enable_sequential_cpu_offload()

hed = HEDdetector.from_pretrained("lllyasviel/Annotators")

image_source = load_image("bird.png")
control_image = hed(image_source)
control_image = control_image.resize(image_source.size)
if control_image.mode != "RGB":
control_image = control_image.convert("RGB")
control_image.save(f"hed.png")

prompt = "bird, cool, futuristic"
image = pipe(
prompt,
image=image_source,
control_image=control_image,
control_guidance_start=0.1,
control_guidance_end=0.9,
controlnet_conditioning_scale=0.9,
num_inference_steps=100,
guidance_scale=8,
).images[0]
image.save("flux.png")

Examples:

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[yiyixuxu]

thanks for the PR!
can we run make style first? it's bit hard to see what are actual changes you made here:)