how to add controlnet in sd3!

[appleyang123]

I currently use inpainting controlnet in sdxl because it uses unet to easily support controlnet. And I am curious about how to add controlnet in sd3 with transforms model structure.

[haofanwang]

Check this PR.

[jadechip]

@haofanwang do guys also have the training code for sd3 control net available somewhere? Thank you for your hard work 🙏

[appleyang123]

Yes, i also need the training code. Very thanks~

[iariav]

Would also really appreciate a training script, same as for SD 1.5 and sdxl

[jeromepatel]

Any updates on training script of SD3 Controlnet ?

[DavyMorgan]

Hi everyone, I’ve implemented a training script at #9249. I would really appreciate it if someone could review it when they have time. Thanks!