Fix pipeline_flux_controlnet.py (#11095) #4483

Job	Run time
Setup Torch Pipelines CUDA Slow Tests Matrix	1m 47s
Torch CUDA Tests (models)	7m 30s
Torch CUDA Tests (schedulers)	5m 28s
Torch CUDA Tests (lora)	9m 42s
Flax TPU Tests	3m 13s
Torch CUDA Tests (others)	42s
ONNX CUDA Tests	4m 24s
Torch CUDA Tests (single_file)	5m 24s
Examples PyTorch CUDA tests on Ubuntu	35m 35s
PyTorch Compile CUDA tests	2m 42s
PyTorch xformers CUDA tests	6m 3s
Torch Pipelines CUDA Tests (cogvideo)	4m 8s
Torch Pipelines CUDA Tests (controlnet)	11m 38s
Torch Pipelines CUDA Tests (controlnet_flux)	1m 15s
Torch Pipelines CUDA Tests (controlnet_sd3)	3m 10s
Torch Pipelines CUDA Tests (ddpm)	2m 19s
Torch Pipelines CUDA Tests (deepfloyd_if)	3m 19s
Torch Pipelines CUDA Tests (flux)	4m 43s
Torch Pipelines CUDA Tests (hunyuan_video)	4m 17s
Torch Pipelines CUDA Tests (hunyuandit)	57s
Torch Pipelines CUDA Tests (i2vgen_xl)	1m 15s
Torch Pipelines CUDA Tests (ip_adapters)	32s
Torch Pipelines CUDA Tests (kandinsky2_2)	3m 45s
Torch Pipelines CUDA Tests (kandinsky3)	2m 3s
Torch Pipelines CUDA Tests (latent_consistency_models)	1m 22s
Torch Pipelines CUDA Tests (ltx)	1m 45s
Torch Pipelines CUDA Tests (marigold)	3m 30s
Torch Pipelines CUDA Tests (pixart_alpha)	53s
Torch Pipelines CUDA Tests (sana)	1m 37s
Torch Pipelines CUDA Tests (stable_cascade)	1m 32s
Torch Pipelines CUDA Tests (stable_diffusion)	4m 10s
Torch Pipelines CUDA Tests (stable_diffusion_2)	5m 20s
Torch Pipelines CUDA Tests (stable_diffusion_3)	1m 35s
Torch Pipelines CUDA Tests (stable_diffusion_xl)	5m 19s
Torch Pipelines CUDA Tests (stable_video_diffusion)	53s
	2h 33m 47s

Provide feedback