SDXL Lora trained with Diffusers never used during inference on A1111 WebUI

[DubailSimon]

Hi ! I've trained a Lora on SDXL with the diffusers script, got an output in .safetensors, which works really to the perfection with the diffuser inference code, but I can't find a way to make it work on AUTO1111 webui.

Other people seems to have trouble loading the model because for them the output is not even in the .safetensors format, but it's not my case.

• It loads, meaning that I can retrieve the Lora in the built in extension (as well as in the additional model extension btw)
• I can "call" it in the prompt
• But the generation NEVER uses the Lora, as if it just didn't existed. I don't get any error during the generation, the image is there, but just nothing like I wanted, with the same result when I don't call the Lora.

I've looked into a lot of options, such as trying to merge the Lora with the SDXL checkpoint to be able to load the model as "one" and bypass this issue, but kohya-ss merge scripts doesn't seem to work for SDXL (looks like it is only up to 2.x max) - I get errors when running it. SuperMerger extension also failed (it's the same code anyway...)

I've also found some "conversion" codes supposed to convert diffuser format into A1111 format like this gist that apparently solved some people's issue, but it doesn't change anything for me. The Lora still isn't taken into account during inference...

Any idea where this could come from ? Any tips ?

Thank you 🙏

[DubailSimon]

SOLVED : For anyone finding this issue, HF solved this 2 weeks ago by pushing a conversion script exactly for this : https://github.com/huggingface/diffusers/blob/main/scripts/convert_diffusers_sdxl_lora_to_webui.py