How to use a Hugging Face trained Lora on Auto1111?

[Mousewrites]

I suspect I need to convert it or something, as there's no .safetensors or .pt file to download.

I know how to use Lora in general, just... can't figure out how to get my lora out of HF and into Auto1111

I've tried downloading that .bin, but the UI doesn't recognize the bin (also, it's quite small, only 3 mb, not sure that's the actual lora?)

I've asked on HF, on Civitai, on the civitai discord and other discords, and nobody seems to have any idea. There's a TON of lora suddenly popping up on Civitai, I figure I'm just missing something.

(yes, i'm trying to get local training of lora going, but I'm having issues with it, unrelated to not knowing how to get my lora out of HF. 😁 )

[CurtisDS]

I looked at your HF account and dont see this repo. A link would probably help. By the looks of it though... youll probably find what youre looking for in those folders that say "checkpoint". Just a guess. Lora can be in a .saftensors, .pt, or .ckpt format.

https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/c81b52ffbd6252842b3473a7aa8eb7ffc88ee7d1/extensions-builtin/Lora/lora.py#L190-#L193

    candidates = \
        glob.glob(os.path.join(shared.cmd_opts.lora_dir, '**/*.pt'), recursive=True) + \
        glob.glob(os.path.join(shared.cmd_opts.lora_dir, '**/*.safetensors'), recursive=True) + \
        glob.glob(os.path.join(shared.cmd_opts.lora_dir, '**/*.ckpt'), recursive=True)

[Mousewrites]

Whoops, it was set to private, i've reset it
https://huggingface.co/Mousewrites/charturnerhn

Checkpoints don't have the files expected, as far as I can tell.

[CurtisDS]

Like i said, it was just a guess. There are .pt files in there... the scaler.pt file... but doubt thats the lora model though. I know lora models can be small.... but not that small lol

[Mousewrites]

Yup, agreed.

I know there's got to be a way to get a usable file OUT but... I can't figure it out. Hopefully someone will shine a light on what I'm missing.

[ducklinus]

I think the correct file is the pytorch_lora_weights.bin in the root folder. However I don't know how to convert it to a pt file, the extension that auto1111 uses. I will follow this discussion for further clarifications.

[ignacfetser]

I have just made a small script that converts the key names to ones auto1111 seems to like better.
Just put the script it in the output folder where the 'checkpoint-xxxx' files are, it parses them and converts the 'custom_checkpoint_0.pkl' in each dir to safetensors format and saves them in the same dir where the script runs. Then move the safetensor files to the auto1111/models/lora folder.
I have no idea how long it will work but it does for me now.

import os;
import re;
import torch;
from safetensors.torch import save_file;

loraName = os.path.split(os.getcwd())[1];

for root, dirs, files in os.walk('.'):
  for dir in dirs:
    ckptIndex = re.search('^checkpoint\-(\d+)$', dir);
    if ckptIndex:
      newDict = dict();
      checkpoint = torch.load(os.path.join(dir, 'custom_checkpoint_0.pkl'));
      for idx, key in enumerate(checkpoint):

        newKey = re.sub('\.processor\.', '_', key);
        newKey = re.sub('mid_block\.', 'mid_block_', newKey);
        newKey = re.sub('_lora.up.', '.lora_up.', newKey);
        newKey = re.sub('_lora.down.', '.lora_down.', newKey);
        newKey = re.sub('\.(\d+)\.', '_\\1_', newKey);
        newKey = re.sub('to_out', 'to_out_0', newKey);
        newKey = 'lora_unet_'+newKey;

        newDict[newKey] = checkpoint[key];

      newLoraName = loraName + '-' + ckptIndex.group(1) + '.safetensors';
      print("Saving " + newLoraName);
      save_file(newDict, newLoraName);

[cfbauer]

Thank you! I added it as convert.py to the repo on huggingface. Is that all I need to do for it to work?

[cfbauer]

I figured it out, I cloned my huggingface repo locally. Here's the output when I ran it:

  File "/path/to/convert.py", line 13, in <module>
    checkpoint = torch.load(os.path.join(dir, 'custom_checkpoint_0.pkl'));
  File "/home/chris/Documents/miniconda/lib/python3.9/site-packages/torch/serialization.py", line 795, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "/home/chris/Documents/miniconda/lib/python3.9/site-packages/torch/serialization.py", line 1002, in _legacy_load
    magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, 'v'.

[ducklinus]

Thank you so much. This worked for me. Finally I can use my Loras in Auto1111.

[Nacurutu]

@Mousewrites

Maybe you can give a try to this TI:

https://civitai.com/models/3036/charturner-character-turnaround-helper-for-15-and-21

This LoRa:

https://civitai.com/models/7252/charturnerbeta-lora-experimental

and/or this Model:

https://civitai.com/models/4118/spybgs-toolkit-for-digital-artists

[cfbauer]

Found this discussion on guggingface with some additional details:

huggingface/diffusers#2326

[harrywang]

I suspect I need to convert it or something, as there's no .safetensors or .pt file to download.

I know how to use Lora in general, just... can't figure out how to get my lora out of HF and into Auto1111

I've tried downloading that .bin, but the UI doesn't recognize the bin (also, it's quite small, only 3 mb, not sure that's the actual lora?)

I've asked on HF, on Civitai, on the civitai discord and other discords, and nobody seems to have any idea. There's a TON of lora suddenly popping up on Civitai, I figure I'm just missing something.

(yes, i'm trying to get local training of lora going, but I'm having issues with it, unrelated to not knowing how to get my lora out of HF. 😁 )

I also want to know how to do this. Another question is why diffuser Lora is only 3m while civitai Lora is around 150m? Thanks a lot!

[catboxanon]

LoRA model size is dependent on the dim/rank, and most people training and uploading these to civitai have been using a rank of 128 due to a training script that spread from 4chan, which yields around 144-150mb, compared to what was originally proposed in the cloneofsimo repo which I believe uses a rank of 1.

[Nacurutu]

LoRA model size is dependent on the dim/rank, and most people training and uploading these to civitai have been using a rank of 128 due to a training script that spread from 4chan, which yields around 144-150mb, compared to what was originally proposed in the cloneofsimo repo which I believe uses a rank of 1.

Any difference regarding the used rank?

[elen07zz]

https://github.com/harrywang/finetune-sd/blob/main/convert-to-safetensors.py

maybe it could work.

[mats4d]

https://huggingface.co/spaces/safetensors/convert

[DubailSimon]

For anyone finding this issue, HF solved this 2 weeks ago by pushing a conversion script exactly for this : https://github.com/huggingface/diffusers/blob/main/scripts/convert_diffusers_sdxl_lora_to_webui.py