LibreChat Can't Connect to Ollama

[F1zzyD]

What happened?

Even through making the appropriate changes to the config.yaml, LibreChat does not seem to see Ollama, even though Ollama is set up correctly and reports models being ready. I have made all of the changes that have been suggested to LibreChat but nothing changes. It still won't see it.

Version Information

latest

Steps to Reproduce

1. Run LibreChat through docker compose up
2. Attempt to add Ollama
3. LibreChat doesn't show Ollama

What browsers are you seeing the problem on?

Firefox

Relevant log output

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

You didn't share your config so I can't help you troubleshoot very well.

Please review: https://www.librechat.ai/docs/quick_start/custom_endpoints
And try this config:

endpoints:
  custom:
    - name: "Ollama"
      apiKey: "ollama"
      baseURL: "http://host.docker.internal:11434/v1/"
      models:
        default: [
          # some defaults in case fetching the list fails
          "llama3.3",
          "llama3.2",
          "llama3.2-vision",
          "qwen2.5",
          "qwen2.5-coder",
          "command-r",
          "mixtral",
          "phi3",
          "phi4",
          ]
        fetch: true
      titleConvo: true
      # titleModel: "phi3"
      titleModel: "current_model" # uses the currently selected model
      modelDisplayLabel: "Ollama"

[F1zzyD]

I did this already by following the docs. Adding this to the librechat.yaml does not allow for connecting to Ollama, making this do nothing.

[danny-avila]

Did you read the Ollama faq? it's likely not exposed to the docker network, these are the relevant parts:

• https://github.com/ollama/ollama/blob/main/docs/faq.md#setting-environment-variables-on-linux
• https://github.com/ollama/ollama/blob/main/docs/faq.md#how-do-i-configure-ollama-server

[F1zzyD]

Note this will depend on how/where Ollama is running.

Take a look at the Ollama faq as this is a frequently experienced problem with Ollama across many different applications

https://github.com/ollama/ollama/blob/main/docs/faq.md

Ollama is already set to be seen by the docker stack, specifically Environment="OLLAMA_HOST=0.0.0.0:11434" that is referenced in your link. Ollama is set and shows the models that are ready. The config is set exactly how you said it should be, yet LibreChat still refuses or can't see Ollama, even if it's in the same docker-compose stack. I have also adjusted the baseURL with no success.

[danny-avila]

Share your librechat.yaml and any logs from the LibreChat container, especially those you see on startup.

The logs are also written to files, you can find the instructions on where to find these logs here: https://github.com/danny-avila/LibreChat/issues/new?template=BUG-REPORT.yml

[nirvdrum]

I ran into what I thought was the same issue. It turned out I had the baseURL key wrong. I had written it as baseUrl. LibreChat dutifully loads the YAML just fine. My Docker logs shows the custom endpoint being read in, but it just didn't work. Additionally, the YAML Validator said there was nothing wrong with my YAML, so I assumed the config was correct. It wasn't until I copied & pasted the config from @danny-avila's comment above that I caught the issue.

I don't know how likely it is that others run into the same issue, but hopefully my experience can save others some time. I think it'd be helpful if LibreChat validated custom endpoints and errored out if an invalid key is found or if a required key is missing.