[Question]: Custom AI Endpoints Ollama API key: Required but ignored Errors when running the Ollama chat models

[getpoweredbyai]

What is your question?

LibreChat | 2024-09-01 21:43:28 error: Failed to fetch models from Ollama API. If you are not using Ollama directly, and instead, through some aggregator or reverse proxy that handles fetching via OpenAI spec, ensure the name of the endpoint doesn't start with ollama (case-insensitive). Cannot read properties of undefined (reading 'status')
LibreChat | 2024-09-01 21:43:29 error: Failed to fetch models from Mistral API
LibreChat | The request either timed out or was unsuccessful. Error message:
LibreChat | Cannot read properties of undefined (reading 'status')

I am trying to use Ollama Local, but I would also like to run in produciton also with Ollama, any assistance will be greatly appreciated.

Trying to get your new Artifacts feature running with local models.

More Details

LibreChat | 2024-09-01 21:43:28 error: Failed to fetch models from Ollama API. If you are not using Ollama directly, and instead, through some aggregator or reverse proxy that handles fetching via OpenAI spec, ensure the name of the endpoint doesn't start with ollama (case-insensitive). Cannot read properties of undefined (reading 'status')
LibreChat | 2024-09-01 21:43:29 error: Failed to fetch models from Mistral API
LibreChat | The request either timed out or was unsuccessful. Error message:
LibreChat | Cannot read properties of undefined (reading 'status')

What is the main subject of your question?

Endpoints

Screenshots

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

What's your config look like? did you see this?

https://www.librechat.ai/docs/configuration/librechat_yaml/ai_endpoints/ollama

[getpoweredbyai]

I tried a few variations of what is on that link, each time I got the same errors. Here is a screenshot, maybe I am placing it in the wrong place? I am so new to coding.

[getpoweredbyai]

[getpoweredbyai]

My next plan was to run this on a server. will i have the same issues? can ollama models be added another way or already included like the other models?

[getpoweredbyai]

Figured it out, thanks!

[aawaheed]

May I know how you fixed it?
I am facing the same issue. Trying with Ollama locally and have set it up using librechat.yml however the response is empty.

[miche1an9e10]

How did you fix this? I am also facing the same issue. Having all set up according to the Librechat setup docs.

error: Failed to fetch models from Ollama API. If you are not using Ollama directly, and instead, through some aggregator or reverse proxy that handles fetching via OpenAI spec, ensure the name of the endpoint doesn't start with ollama (case-insensitive). Cannot read properties of undefined (reading 'status')

[boydthomson]

Yup, same problem. Feels like I've tried everything... but obviously have not.

[aawaheed]

my librechat.yaml looks like this for ollama

custom:
   # ollama
   - name: "Ollama"
     apiKey: "ollama"
     baseURL: "http://host.docker.internal:11434/v1/" 
     models:
       default: [
         "llama3.1",
         "HR"
       ]
       fetch: false # fetching list of models is not supported
     titleConvo: true
     titleModel: "current_model"
     summarize: false
     summaryModel: "current_model"
     forcePrompt: false
     modelDisplayLabel: "Ollama"
     addParams:
       "stop": [
           "<|start_header_id|>",
           "<|end_header_id|>",
           "<|eot_id|>",
           "<|reserved_special_token"
       ]

[MalchuL]

I found solution for myself. In case if you use LibreChat in docker, but install Ollama manually.
TLDR;

# Stop Ollama service
export OLLAMA_HOST=0.0.0.0:11434
ollama serve

in librechat.yaml:

      baseURL: "http://host.docker.internal:11434/v1/chat/completions"  #To make container access to your host

By default Ollama uses default allowed hosts: 127.0.0.1 or localhost. But from docker host became other 172.21.0.1 (in my case for example) and Ollama not allow this. Try to change Ollama host to 0.0.0.0, that means allow all connections.

[abdullahalamodi]

I found solution for myself. In case if you use LibreChat in docker, but install Ollama manually. TLDR;

# Stop Ollama service
export OLLAMA_HOST=0.0.0.0:11434
ollama serve

in librechat.yaml:

      baseURL: "http://host.docker.internal:11434/v1/chat/completions"  #To make container access to your host

By default Ollama uses default allowed hosts: 127.0.0.1 or localhost. But from docker host became other 172.21.0.1 (in my case for example) and Ollama not allow this. Try to change Ollama host to 0.0.0.0, that means allow all connections.

it's work, thank you