fix issue that training flux controlnet was unstable and validation r…#11373

yiyixuxu merged 3 commits intohuggingface:mainhuggingface/diffusers:mainfrom

PromeAIpro:flux-controlnet-trainPromeAIpro/diffusers:flux-controlnet-trainCopy head branch name to clipboard