Change models without restarting.

[dreemur99]

It would be nice to be able to swap models would without having to restart the program, either by list from a folder or a dialog box/context window or w/e. Being able to start it without having loaded a model first and loading it after the program or endpoint or whatever is loaded would be neat too.

I'd say this is a small thing but i know next to nothing in coding so im sure the coding involved would be very involved but still.

[LostRuins]

This is probably not going to be a good idea due to potential resource leaks, since I doubt everything is being properly cleaned up when a model is unloaded.

[sinni800]

Seeing this, make it a hack: Koboldcpp restarts itself with the same parameters, plus the new model name optionally detaches the new process from the current one (Linux), then dies. One could implement the API to switch models here as well, giving KoboldCPP a directory to choose from.

This would be a hack, sure, but one that is comparably easy to implement and adds a lot of usability.

[LostRuins]

Yeah I suppose you can write a shell script or python script to do that

[sinni800]

Thanks for the functionality, BTW. It works perfectly fine the way it's implemented now where it restarts itself completely with a new config file.

[zeorion974]

I agree about this idea as this regenerates the tunnel each times too (for tunnel connection)

[LostRuins]

it has been added in the latest version.

[LostRuins]

this is now added

[TFWol]

@dreemur99 Don't forget to mark an answer 😃 (if you're still around)