How do I use the OllamaApi part of KoboldCPP?

[TFWol]

Enabled APIs: KoboldCppApi OpenAiApi OllamaApi
Starting Kobold API on port 5001 at http://127.0.0.1:5001/api/
Starting OpenAI Compatible API on port 5001 at http://127.0.0.1:5001/v1/

Under the 3 enabled API's, I see entries correlating to only 2 of them. Kobold and OpenAI.

Which is the one for Ollama?

[LostRuins]

Ollama uses the /api/chat endpoint - generally this is set automatically by whatever software you are using.

You only need to point the tool to port 5001 and it should work.

Alternatively, run KoboldCpp at port 11434 and it should work out of the box with your tool (that is the ollama default port)

[TFWol]

Awesome. Thanks for clarifying!

[TFWol]

I'm guessing Open WebUI isn't compatible with that particular API implementation then, just the OpenAI one.

EDIT:
Yep. I see OWUI log:

aiohttp.client_exceptions.ContentTypeError: 404, message='Attempt to decode JSON with unexpected mimetype: text/html', url='http://192.168.1.3:5001/api/version'

An Ollama instance would give back something like {"version":"0.7.0"}
while koboldcpp outputs an error.

Should I open up a feature request to emulate that URL?

[LostRuins]

Ah, okay, that would be good. Can you include a sample of a full response from the newest Ollama version so I can emulate it.

[LostRuins]

Alright nevermind, I got it

[TFWol]

Alright nevermind, I got it

Sorry about that! I fell asleep at my desk. My sleep schedule is all over the place. 😅