How to reduce model switching time

[Hamzajani153]

I use multiple model when i try to switch the model it take 50 to 60 second to switch model on another. How to reduce the time of model when i try to switch it. I'm using aws ec2 g5xlarge instance for this. If is use api which api i use to reduce the model switching time.

[w-e-w]

if there's only a couple of models that you are "switching between" and the easiest way is to just not unload them and keeping them in RAM or VRAM

[Hamzajani153]

This is change from the webui i make my own ui so i integrate it by using api. Tell me which api is responsible for this.

[ibrainventures]

i think it is that endpoint .. : /sdapi/v1/options

[Hamzajani153]

This sdapi/v1/options endpoint is use for load a single model in memory but i try to load the multiple model at a time. So when i switching the model it doesnot take time.

[ibrainventures]

That endpoint is to set the config by api inside your Automatic111 "instance".
Look at the bottom of the added image, there are the config options for your needed setup (sd_checkpoints_limit .. etc.)
If you want to have multiple models loaded / (keept after loaded) you wil set this config to your needs.
(like the w-e-w added a image with the automatic1111 config). The models are not pre-loaded, they will stored
after "used". --> always the last .. so .. you must warmup your instance with the models and after that you
have a under 1 sec switching time between models (running here with 5 models cached..)