Expand CLI and GUI configuration

[mervn]

This is grounds for a discussion, rather than an issue, since this may be philosophical. The CLI and GUI (launcher) should be able to configure:

• llama.cpp (i.e "--temp" is passthrough)
• webui

Not being able to configure llama.cpp is particularly problematic; as variables like "temperature", "top_k", and "top_p" cannot be changed in CLI only mode. It is also documented that the webui overrides some configurations (i.e --defaultgenamt), which means a custom webui cannot be created non-interactively.

Is there interest in implementing this? Or something like this? For example this, default config locations (i.e $XDG_CONFIG_HOME/koboldcpp/webui.json) and falling back to the command-line or built-in config, would be useful.

The end goal of these changes is to make it faster to spin-up a session, which goes a very very long way for desktop users transitioning from remote AI services.

[LostRuins]

Hi, sorry for the delay in response.

Yeah, it's true the cli mode in KoboldCpp is somewhat limited - KoboldCpp is primarily intended to be used over API via the web UI.

Regarding spinning up a session, KoboldCpp has this feature called --preloadstory which allows you to setup a session exactly how you want it. Simply configure all settings (e.g. instruct tags, temperature, top_k etc) and then save it as a JSON save file, then launch your koboldcpp with this set in --preloadstory. All settings will be automatically loaded.

Does that help?