MCP tools not called when using litellm and ollama

[snowstopxt]

I am using litellm proxy with these configs

and my librechat with these configs

I am trying to use the mcp tools defined in my fastapi_mcp server but am always met with an incorrect response and tools not being run correctly. Agents are also met with the same problem.

I am running everything in deploy-compose (including my mcp server) Any help will be appreciated, as I have been working on this for hours and am still not able to get it to work.

What I have tried:

• Using ollama_chat -> returns error: json: cannot unmarshal array into Go struct field .tools.function.parameters.properties.type of type string
• Changing mcp server to the litellm exposed mcp url -> error: Connection failed

[fbnlbl-hws]

I have the exact same issue, also using LibreChat with LiteLLM and Ollama, trying to call a SearXNG MCP server directly.
The LLM answers with the correct syntax for calling the MCP server but LibreChat is not invoking it.
The SearXNG MCP is showing up in the chat tool dropdown and is selected for the attached demo Request / Response data.
I can use the MCP server just fine using Open-WebUI or MCP Inspector.

librechat.yaml

version: 1.2.5
cache: true

interface:
  endpointsMenu: true
  modelSelect: true
  parameters: true  # This is crucial - without it, the agent builder won't appear
  sidePanel: true
  agentBuilder: true

endpoints:
  assistants:
    disableBuilder: false
    capabilities: ["tools", "actions", "retrieval", "code_interpreter", "image_vision"]
  agents:
    disableBuilder: false
    capabilities: ["execute_code", "file_search", "actions", "tools", "ocr"]
    actions:
      allowedDomains:
        - "litellm.ai-lab.de"
        - "ai-lab.de"
  custom:
    - name: "LiteLLM"
      apiKey: "sk-..."
      baseURL: "https://litellm.ai-lab.de/v1"
      models:
        default: ["llama3.1:8b"]
        fetch: true 
      titleConvo: true
      titleModel: "current_model"
      modelDisplayLabel: "LiteLLM"
      capabilities: ["tools", "actions"]
      agentOptions:
        capabilities: ["tools", "actions"]
      params:
        tools: true

mcpServers:
  searxng:
    url: https://searxngmcp.ai-lab.de/sse

litellm config.yml:

model_list:
  - model_name: "llama3.1:8b"
    litellm_params:
      model: "ollama/llama3.1:8b"
      api_base: "https://ollama.ai-lab.de"
      temperature: 0.7
      max_tokens: 4096
      drop_params: true
  - model_name: "gemma3:27b"
    litellm_params:
      model: "ollama/gemma3:27b"
      api_base: "https://ollama.ai-lab.de"
      temperature: 0.7
      max_tokens: 32768  # or up to 128k if supported
      drop_params: true
  - model_name: "qwen3:32b"
    litellm_params:
      model: "ollama/qwen3:32b"
      api_base: "https://ollama.ai-lab.de"
      temperature: 0.7
      max_tokens: 32768  # Adjust based on your Qwen3 model
      drop_params: true

litellm_settings:
  set_verbose: false
  success_callback: ["langfuse"]
  callbacks: ["langfuse"]
  redact_user_api_key_info: true
  prompt_dir: "./prompts"
  global_disable_no_log_param: false 

mcp_servers:
  searxng_mcp:
    url: "https://searxngmcp.ai-lab.de/sse"

general_settings:
  store_model_in_db: true
  store_prompts_in_spend_logs: true

LiteLLM log of the user prompt and LLM answer, when trying to use MCP in qwen3 chat:

Request:

{
  "user": "6821e947597a91903d7e2c4e",
  "model": "qwen3:32b",
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "search_mcp_searxng",
        "parameters": {
          "type": "object",
          "$schema": "http://json-schema.org/draft-07/schema#",
          "required": [
            "q"
          ],
          "properties": {
            "q": {
              "type": "string"
            },
           ...,
          "additionalProperties": false
        },
        "description": "\n    Perform a search using SearXNG with all supported parameters.\n    "
      }
    }
  ],
  "stream": true,
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "text": "Who is the current pope?",
          "type": "text"
        }
      ]
    }
  ]
}

LLM Response:

{
  "id": "chatcmpl-e69b49ef-2fbe-4b7d-8bfb-8ce63e0a293f",
  "model": "qwen3:32b",
  "usage": {
    "total_tokens": 705,
    "prompt_tokens": 681,
    "completion_tokens": 24,
    "prompt_tokens_details": null,
    "completion_tokens_details": {
      "audio_tokens": null,
      "reasoning_tokens": 0,
      "accepted_prediction_tokens": null,
      "rejected_prediction_tokens": null
    }
  },
  "object": "chat.completion",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "{\"name\": \"search_mcp_searxng\", \"arguments\": {\"q\": \"current pope\"}}",
        "tool_calls": null,
        "function_call": null
      },
      "finish_reason": "stop"
    }
  ],
  "created": 1748844601,
  "system_fingerprint": null
}

I hope this provides some additional infos that might help fix this issue.

[snowstopxt]

I think I have found a solution which was also mentioned in #2215. Changing the model to openai and adding /v1 in the litellm config file worked. I have added a dummy api key in my env file for OPENAI_API_KEY as well.

[fbnlbl-hws]

So my error is connected to LiteLLM changing the response coming from Ollama, moving the tool call instructions to the content field, causing it to simply be displayed without execution.
Directly using Ollama models in the Librechat configuration fixes this issue, but makes debugging LLM input / output difficult.
So this issue is not really connected to LibreChat.

When using curl to query Ollama directly, you can see the output shows the function call in the tool_calls section of the json.

Query:

curl -X POST https://ollama.ai-lab.de/v1/chat/completions -H "Content-Type: application/json" -d '{"user":"6821e947597a91903d7e2c4e","model":"mistral:7b-instruct","tools":[{"type":"function","function":{"name":"search_mcp_searxng","parameters":{"type":"object","$schema":"http://json-schema.org/draft-07/schema#","required":["q"],"properties":{"q":{"type":"string"},"page":{"anyOf":[{"anyOf":[{"not":{}},{}]},{"type":"null"}]},"theme":{"anyOf":[{"anyOf":[{"not":{}},{"anyOf":[{"type":"string"},{}]}]},{"type":"null"}]},"format":{"anyOf":[{"anyOf":[{"not":{}},{"type":"string"}]},{"type":"null"}]},"engines":{"anyOf":[{"anyOf":[{"not":{}},{"anyOf":[{"type":"string"},{}]}]},{"type":"null"}]},"language":{"anyOf":[{"anyOf":[{"not":{}},{"anyOf":[{"type":"string"},{}]}]},{"type":"null"}]},"categories":{"anyOf":[{"anyOf":[{"not":{}},{"anyOf":[{"type":"string"},{}]}]},{"type":"null"}]},"safesearch":{"anyOf":[{"anyOf":[{"not":{}},{"anyOf":[{},{}]}]},{"type":"null"}]},"time_range":{"anyOf":[{"anyOf":[{"not":{}},{"anyOf":[{"type":"string"},{}]}]},{"type":"null"}]},"image_proxy":{"anyOf":[{"anyOf":[{"not":{}},{"anyOf":[{"type":"boolean"},{}]}]},{"type":"null"}]},"autocomplete":{"anyOf":[{"anyOf":[{"not":{}},{"anyOf":[{"type":"string"},{}]}]},{"type":"null"}]},"enabled_engines":{"anyOf":[{"anyOf":[{"not":{}},{"anyOf":[{"type":"string"},{}]}]},{"type":"null"}]},"enabled_plugins":{"anyOf":[{"anyOf":[{"not":{}},{"anyOf":[{"type":"string"},{}]}]},{"type":"null"}]},"disabled_engines":{"anyOf":[{"anyOf":[{"not":{}},{"anyOf":[{"type":"string"},{}]}]},{"type":"null"}]},"disabled_plugins":{"anyOf":[{"anyOf":[{"not":{}},{"anyOf":[{"type":"string"},{}]}]},{"type":"null"}]},"results_on_new_tab":{"anyOf":[{"anyOf":[{"not":{}},{"anyOf":[{},{}]}]},{"type":"null"}]}},"additionalProperties":false},"description":"\n    Perform a search using SearXNG with all supported parameters.\n    "}}],"stream":true,"messages":[{"role":"user","content":[{"text":"Who is the current pope?","type":"text"}]}]}'

Output:

{"id":"chatcmpl-584","object":"chat.completion.chunk","created":1748856838,"model":"mistral:7b-instruct","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"","tool_calls":[{"id":"call_yvuprbfu","index":0,"type":"function","function":{"name":"search_mcp_searxng","arguments":"{\"q\":\"current pope\"}"}}]},"finish_reason":null}]}

{"id":"chatcmpl-584","object":"chat.completion.chunk","created":1748856838,"model":"mistral:7b-instruct","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":"tool_calls"}]}

[fbnlbl-hws]

Got it working using the "openai" trick mentioned by @snowstopxt, here is a sample LiteLLM config entry for reference:

- model_name: "mistral:7b-instruct"
    litellm_params:
      model: "openai/mistral:7b-instruct" # Important add openai/
      api_base: "https://ollama.ai-lab.de/v1" # Change to your ollama instance
      api_key: "dummy" # Important
      temperature: 0.7
      max_tokens: 32768  # Adjust based on your model
      drop_params: true
    model_info:
      supports_function_calling: true  
      supports_parallel_function_calling: false 

[notdefine]

I have the same issue, but I use the configuration dialog from litellm to configure endpoints.

When I add /v1 to the api_base and test the connection, it fails:

Connection Test Results
Connection to ollama/qwen3:latest failed

Did you reconfigure your ollama instance to work on URL /v1 @fbnlbl-hws ?

And when I am changing the model name to ollama//qwen3:latest I got this error:

OllamaException - {"error":"model 'openai/qwen3:latest' not found"}