How do you deal with multiple programs that use the same model files? What if we had a tool for that...?

[Vendaciousness]

'I wish there was an easy way of sharing huge model files between different software...' --me

Do you use more than just Auto1111 for your Stable Diffusing? Perhaps you've installed ComfyUI and had to edit your own YAML file, in order to use your A1111 model files (without making copies).

I currently have 3 versions of ComfyUI, 3 of Auto1111, 2 of SDNext and a few others, all of which use the same SD model files, but want them in different places.

The solution I used at first was to change my model directory location settings. This can be done in A1111 easily from the 'Paths for Saving' tab in settings, though I remember when you used to have to specify your directories from the command line.

But some programs force you to use the directories that they specify and don't allow changes. I won't name names, but I use two right now. For these, I learned you can use Symlinks and save hundreds of GB of space.

Symlink is short for 'Symbolic Link' and it's like a shortcut that software can't tell from the real thing. So now, I have a single directory for my SDXL checkpoints, with a bunch of symlinks pointing to it in all the places model folders would be, such as a1111's '/models/Stable-diffusion' or ComfyUI's '/models/checkpoints'. The only thing to watch out for are renaming or deleting your source directories, or performing 'automatic upgrades', which might screw things up by mistaking the symlinks for real directories. For the most part there's no issue, as long as you're careful. For situation where you need a different folder name, you can rename the symlink to anything you like once it is created.

I think a lot of the people who use symlinks do it with the command line 'mklink', but personally, I hate doing crap like this in a terminal, since you get one character wrong, like a '/' instead of '' and the whole thing breaks. Also, which one goes first, the location of the source folder, or the target directory? That kinda stuff makes my eyes cross. Instead, you can use a handy tool called Symlink Creator. There are two different programs by that name and both work, but this one is open source, more full-featured and a bit easier to understand:
https://github.com/arnobpl/SymlinkCreator/releases

Watch the dev's video to get a better understanding of what symlinks are and what they're useful for:
https://www.youtube.com/watch?v=1Ad2Qymz5hI

AH!--But what if there was a tool JUST for managing all those different AI models you love to horde... The problem of managing all these huge files can't be solved from within A1111. It would need to be independent of all software and be able to work with them all. I'd make one, but I'm literally not smart enough to code. But maybe one of the brilliant and talented coders in this awesome community could do it... Imagine making a tool that was as indispensable as the safetensors format... You would be a hero...

I envision a WebUI (gradio) tool in which you first specify the permanent location for your model files of each type, like checkpoints, VAEs, embeddings, etc., then you could create and manage settings for each SD app you have, with all of it's directories.

For example, if you were adding a new A1111 installation to your setup, you'd choose Add New > SD > Automatic1111, then just browse to your install directory and click OK and all of the different model subdirectories would be populated automatically and stored in a csv or json file.

The program would basically manage all your Symlinks and whenever you installed a new program, you could add it and never worry about moving hundreds of gigabytes in and out of folders every time you have to reinstall, as I used to do every time my copy of A1111 got corrupted or I misconfigured it.

You could even have it watch or at least check the symlinks and origin files for changes/renaming, too. You could support model directories, too, for LLMs where there's a whole folder full of files for each model.

If anyone gets inspired and decides to make something, hit me up!

-V

[Chokaphi1]

I have used a tool called link shell extension for years. That adds the ability to generate the various link types by drag and drop as well as handles copy and moving folders with links.

Just select all the files/folders and then right click and drag to the folder I want the links to be created at.

[Vendaciousness]

That's pretty cool, but I think I'll stay with Symlink Creator, as I don't foresee needing to make Symlinks much, now that my system is set up. If I did this often at all though, I'd definitely add it to my right-click.

And I would still rather like to have a tool for organizing and symlinking AI models. Probably though, people would be more likely to make a tool for classifying all their models before they'd make one for linking, like I suggested.

Thanks for the link!