Can't chat with ollama models

[AzizZayed]

What happened?

I getht he following error when chating with a loaded ollama model:

Something went wrong. Here's the specific error message we encountered: An error occurred while processing the request: 404 page not found

In the error logs:

{"error":"404 page not found","level":"error","message":"[api/server/controllers/agents/client.js #sendCompletion] Operation aborted 404 page not found","name":"ResponseError","pregelTaskId":"5bdc4549-682e-5ff3-8900-5f07119f23b8","stack":"ResponseError: 404 page not found\n    at checkOk (/app/node_modules/ollama/dist/browser.cjs:77:9)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async post (/app/node_modules/ollama/dist/browser.cjs:141:3)\n    at async Ollama.processStreamableRequest (/app/node_modules/ollama/dist/browser.cjs:252:25)\n    at async ChatOllama._streamResponseChunks (/app/node_modules/@langchain/ollama/dist/chat_models.cjs:737:24)\n    at async ChatOllama._generateUncached (/app/node_modules/@langchain/core/dist/language_models/chat_models.cjs:215:34)\n    at async ChatOllama.invoke (/app/node_modules/@langchain/core/dist/language_models/chat_models.cjs:92:24)\n    at async RunnableCallable.func (/app/node_modules/@librechat/agents/dist/cjs/graphs/Graph.cjs:383:39)\n    at async RunnableCallable.invoke (/app/node_modules/@langchain/langgraph/dist/utils.cjs:88:27)\n    at async RunnableSequence.invoke (/app/node_modules/@langchain/core/dist/runnables/base.cjs:1305:33)","status_code":404}
{"error":"404 page not found","level":"error","message":"[api/server/controllers/agents/client.js #sendCompletion] Unhandled error type 404 page not found","name":"ResponseError","pregelTaskId":"5bdc4549-682e-5ff3-8900-5f07119f23b8","stack":"ResponseError: 404 page not found\n    at checkOk (/app/node_modules/ollama/dist/browser.cjs:77:9)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async post (/app/node_modules/ollama/dist/browser.cjs:141:3)\n    at async Ollama.processStreamableRequest (/app/node_modules/ollama/dist/browser.cjs:252:25)\n    at async ChatOllama._streamResponseChunks (/app/node_modules/@langchain/ollama/dist/chat_models.cjs:737:24)\n    at async ChatOllama._generateUncached (/app/node_modules/@langchain/core/dist/language_models/chat_models.cjs:215:34)\n    at async ChatOllama.invoke (/app/node_modules/@langchain/core/dist/language_models/chat_models.cjs:92:24)\n    at async RunnableCallable.func (/app/node_modules/@librechat/agents/dist/cjs/graphs/Graph.cjs:383:39)\n    at async RunnableCallable.invoke (/app/node_modules/@langchain/langgraph/dist/utils.cjs:88:27)\n    at async RunnableSequence.invoke (/app/node_modules/@langchain/core/dist/runnables/base.cjs:1305:33)","status_code":404}

Version Information

ghcr.io/danny-avila/librechat-rag-api-dev latest 64a31a1f9483 6 days ago 7.87GB
ghcr.io/danny-avila/librechat latest 3d0aff522ece 2 weeks ago 1.11GB

Steps to Reproduce

docker-compose.override.yml:

services:
  api:
    image: ghcr.io/danny-avila/librechat:latest
    restart: unless-stopped
    volumes:
      - type: bind
        source: ./librechat.yaml
        target: /app/librechat.yaml
      - ./librechat_data/images:/app/client/public/images
      - ./librechat_data/uploads:/app/uploads
      - ./librechat_data/logs:/app/api/logs

  mongodb:
    restart: unless-stopped
    volumes:
      - ./librechat_data/data-node:/data/db

  meilisearch:
    restart: unless-stopped
    volumes:
      - ./librechat_data/meili_data_v1.12:/meili_data

  vectordb:
    restart: unless-stopped
    volumes:
      - pgdata2:/var/lib/postgresql/data

# USE RAG API IMAGE WITH LOCAL EMBEDDINGS SUPPORT
  rag_api:
    image: ghcr.io/danny-avila/librechat-rag-api-dev:latest
    restart: unless-stopped
    extra_hosts:
      - "host.docker.internal:host-gateway"

# ADD OLLAMA
  ollama:
    image: ollama/ollama:latest
    restart: unless-stopped
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities:
                - gpu
    ports:
      - "11434:11434"
    volumes:
      - ./librechat_data/ollama:/root/.ollama
      - ./librechat_data/modelfiles:/modelfiles
    environment:
      - OLLAMA_NUM_PARALLEL=4
      - OLLAMA_MAX_LOADED_MODELS=3

librechat.yaml:

# For more information, see the Configuration Guide:
# https://www.librechat.ai/docs/configuration/librechat_yaml

# Configuration version (required)
version: 1.2.1

# Cache settings: Set to true to enable caching
cache: true

# File strategy s3/firebase
# fileStrategy: "s3"

# Custom interface configuration
interface:
  customWelcome: "Welcome to LibreChat! Enjoy your experience."
  # MCP Servers UI configuration
  mcpServers:
    placeholder: 'MCP Servers'
  endpointsMenu: true
  modelSelect: true
  parameters: true
  sidePanel: true
  presets: true
  prompts: true
  bookmarks: true
  multiConvo: true
  agents: true
 
# Example Actions Object Structure
actions:
  allowedDomains:
    - "swapi.dev"
    - "librechat.ai"
    - "google.com"

# Definition of custom endpoints
endpoints:
  custom:
    - name: "Ollama"
      apiKey: "ollama"
      # use 'host.docker.internal' instead of localhost if running LibreChat in a docker container
      baseURL: "http://ollama:11434/api/"
      models:
        default: [
          "llama2",
          "mistral",
          "codellama",
          "dolphin-mixtral",
          "mistral-openorca"
          ]
        # fetching list of models is supported but the `name` field must start
        # with `ollama` (case-insensitive), as it does in this example.
        fetch: true
      titleConvo: true
      titleModel: "current_model"
      summarize: false
      summaryModel: "current_model"
      forcePrompt: false
      modelDisplayLabel: "Ollama"

    - name: 'groq'
      apiKey: '${GROQ_API_KEY}'
      baseURL: 'https://api.groq.com/openai/v1/'
      models:
        default:
          [
            'llama-3.3-70b-versatile',
            'llama-3.1-8b-instant',
            'llama3-70b-8192',
            'llama3-8b-8192',
            'gemma2-9b-it',
          ]
        fetch: true
      titleConvo: true
      titleModel: 'llama-3.3-70b-versatile'
      modelDisplayLabel: 'groq'

In the docs, it says to use baseURL: "http://host.docker.internal:11434/v1/" but that did not work. I changed it to baseURL: "http://ollama:11434/api/" and the model fetching worked but chatting still does not work.

What browsers are you seeing the problem on?

Microsoft Edge

Relevant log output

2025-07-17T04:16:50.009Z debug: [BaseClient] Loading history:
{
  conversationId: "3cce30ba-e3ba-4af1-b66f-6a36ce77cc11",
  parentMessageId: "c00433d5-51ef-4417-9123-905845ddcd20____",
}
2025-07-17T04:16:50.010Z debug: [BaseClient] Context Count (1/2)
{
  remainingContextTokens: 3583,
  maxContextTokens: 3600,
}
2025-07-17T04:16:50.011Z info: Custom config file loaded:
2025-07-17T04:16:50.010Z debug: [BaseClient] Context Count (2/2)
{
  remainingContextTokens: 3583,
  maxContextTokens: 3600,
}
2025-07-17T04:16:50.010Z debug: [BaseClient] tokenCountMap:
{
  2034a364-25a7-4d28-a74c-13e0b7c79003: 5,
  c00433d5-51ef-4417-9123-905845ddcd20____: 4,
  357fd540-604b-4c25-9b33-37cbbdfe9580: 5,
}
2025-07-17T04:16:50.010Z debug: [BaseClient]
{
  promptTokens: 17,
  remainingContextTokens: 3583,
  payloadSize: 3,
  maxContextTokens: 3600,
}
2025-07-17T04:16:50.011Z debug: [BaseClient] tokenCountMap
{
  2034a364-25a7-4d28-a74c-13e0b7c79003: 5,
  c00433d5-51ef-4417-9123-905845ddcd20____: 4,
  357fd540-604b-4c25-9b33-37cbbdfe9580: 5,
}
2025-07-17T04:16:50.011Z debug: [BaseClient] userMessage
{
  messageId: "357fd540-604b-4c25-9b33-37cbbdfe9580",
  parentMessageId: "c00433d5-51ef-4417-9123-905845ddcd20____",
  conversationId: "3cce30ba-e3ba-4af1-b66f-6a36ce77cc11",
  sender: "User",
  text: "hi",
  isCreatedByUser: true,
  tokenCount: 5,
}
2025-07-17T04:16:50.011Z info: {
  "version": "1.2.1",
  "cache": true,
  "interface": {
    "customWelcome": "Welcome to LibreChat! Enjoy your experience.",
    "mcpServers": {
   ... [truncated]
2025-07-17T04:16:50.011Z debug: Custom config:
{
  version: "1.2.1",
  cache: true,
    interface.customWelcome: "Welcome to LibreChat! Enjoy your experience.",
    mcpServers.placeholder: "MCP Servers",
    interface.endpointsMenu: true,
    interface.modelSelect: true,
    interface.parameters: true,
    interface.sidePanel: true,
    interface.presets: true,
    interface.prompts: true,
    interface.bookmarks: true,
    interface.multiConvo: true,
    interface.agents: true,
    // 7 socialLogin(s)
    registration.socialLogins: ["github","google","discord","openid","facebook","apple","saml"],
    // 3 allowedDomain(s)
    actions.allowedDomains: ["swapi.dev","librechat.ai","google.com"],
    // 2 custom(s)
    endpoints.custom: [{"name":"Ollama","apiKey":"ollama","baseURL":"http://ollama:11434/api/","models":{"default":["llama2... [truncated],{"name":"groq","apiKey":"${GROQ_API_KEY}","baseURL":"https://api.groq.com/openai/v1/","models":{"def... [truncated]],
}
2025-07-17T04:16:50.018Z debug: [saveConvo] api/app/clients/BaseClient.js - saveMessageToDatabase #saveConvo
2025-07-17T04:16:50.019Z error: [api/server/controllers/agents/client.js #sendCompletion] Operation aborted 404 page not found
2025-07-17T04:16:50.019Z error: [api/server/controllers/agents/client.js #sendCompletion] Unhandled error type 404 page not found
2025-07-17T04:16:50.020Z info: Custom config file loaded:
2025-07-17T04:16:50.020Z info: {
  "version": "1.2.1",
  "cache": true,
  "interface": {
    "customWelcome": "Welcome to LibreChat! Enjoy your experience.",
    "mcpServers": {
   ... [truncated]
2025-07-17T04:16:50.021Z debug: Custom config:
{
  version: "1.2.1",
  cache: true,
    interface.customWelcome: "Welcome to LibreChat! Enjoy your experience.",
    mcpServers.placeholder: "MCP Servers",
    interface.endpointsMenu: true,
    interface.modelSelect: true,
    interface.parameters: true,
    interface.sidePanel: true,
    interface.presets: true,
    interface.prompts: true,
    interface.bookmarks: true,
    interface.multiConvo: true,
    interface.agents: true,
    // 7 socialLogin(s)
    registration.socialLogins: ["github","google","discord","openid","facebook","apple","saml"],
    // 3 allowedDomain(s)
    actions.allowedDomains: ["swapi.dev","librechat.ai","google.com"],
    // 2 custom(s)
    endpoints.custom: [{"name":"Ollama","apiKey":"ollama","baseURL":"http://ollama:11434/api/","models":{"default":["llama2... [truncated],{"name":"groq","apiKey":"${GROQ_API_KEY}","baseURL":"https://api.groq.com/openai/v1/","models":{"def... [truncated]],
}

Screenshots

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

Ollama is working on my end.

Please change:

baseURL: "http://ollama:11434/api/"

to:

baseURL: "http://ollama:11434/v1/"

More info:

• https://www.librechat.ai/docs/configuration/librechat_yaml/ai_endpoints/ollama
• https://github.com/ollama/ollama/blob/main/docs/faq.md#how-can-i-expose-ollama-on-my-network

[AzizZayed]

This worked, thank you!

Just an FYI, baseURL: "http://host.docker.internal:11434/v1/" did not work because host.docker.internal is not assocaited to any IP address in the librechat container (I checked with dig host.docker.internal and it returned no answer section).

[AzizZayed]

Also, the ollama documentation shows /api/ as the way to get to the endpoints, but mentions nothing about /v1/.