Flux with Inpaiting and ControlNet currently yields bad result with the base model.
To echo this comment, using de-distilled models could potentially help getting better outputs.
Currently the Flux ControlNet pipelines do not support models like https://huggingface.co/nyanko7/flux-dev-de-distill