Example of Ollama with OpenAI proxy server

[jxnl]

Is there an example of using ollama with the openai proxy server to host mistral?

[krrishdholakia]

Yep right here - https://docs.litellm.ai/docs/proxy/quick_start#supported-llms

$ litellm --model ollama/<ollama-model-name>

e.g.

$ litellm --model ollama/codellama

to call ollama's codellama model (by default this will assume it's on port 11434)

If you want to change the api base, just do

$ litellm --model ollama/<ollama-model-name> --api_base <my-hosted-endpoint>

[krrishdholakia]

@jxnl let me know if this solves your problem