MCP issue with Llama cpp docker models

[NotYourAverageAl]

What happened?

I have Librechat setup with custom endpoint with llama-swap.
I have setup a couple of mcp servers but I get an error message when I try to call an mcp error: [api/server/controllers/agents/client.js #sendCompletion] Unhandled error type 500 JSON schema conversion failed: Unrecognized schema: {"not":{}}

Any thoughts?

Version Information

ce3b00be875a

Steps to Reproduce

1. Setup llama-swap custom endpoint
2. Setup mcp servers with librechat
3. Activate mcps on a llama-swap model and try calling a tool

What browsers are you seeing the problem on?

Microsoft Edge

Relevant log output

2025-07-29 14:27:05 info: Cleaning up FlowStateManager intervals...

> [email protected] backend

> cross-env NODE_ENV=production node api/server/index.js

2025-07-29 14:27:08 info: Connected to MongoDB

2025-07-29 14:27:08 info: [indexSync] Starting index synchronization check...

2025-07-29 14:27:08 info: Custom config file loaded:

2025-07-29 14:27:08 info: {

  "version": "1.2.8",

  "cache": true,

  "interface": {

    "privacyPolicy": {

      "externalUrl": "https://librechat.ai/privacy-policy",

      "openNewTab": true

    },

    "termsOfService": {

      "externalUrl": "https://librechat.ai/tos",

      "openNewTab": true

    }

  },

  "registration": {

    "socialLogins": [

      "discord",

      "facebook",

      "github",

      "google",

      "openid"

    ]

  },

  "endpoints": {

    "custom": [

      {

        "name": "Llamaswap",

        "apiKey": "swap",

        "baseURL": "http://host.docker.internal:65000/v1/",

        "models": {

          "default": [

            "Mistral-Small-3.2-24B-Instruct-2506",

            "Qwen3-30B-A3B",

            "ERNIE-4.5-21B-A3B",

            "EXAONE-4.0-32B",

            "Jan-nano-128k",

            "Qwen3-32B",

            "gemma-3-27b-it-qat",

            "llama3.2-1b",

            "nomic-embed-text-v1.5"

          ],

          "fetch": true

        },

        "titleConvo": true,

        "titleModel": "current_model",

        "summarize": false,

        "summaryModel": "current_model",

        "forcePrompt": false,

        "modelDisplayLabel": "Llamaswap"

      },

      {

        "name": "Ollama",

        "apiKey": "ollama",

        "baseURL": "http://host.docker.internal:11434/v1/",

        "models": {

          "default": [

            ""

          ],

          "fetch": true

        },

        "titleConvo": true,

        "titleModel": "current_model",

        "summarize": false,

        "summaryModel": "current_model",

        "forcePrompt": false,

        "modelDisplayLabel": "Ollama"

      }

    ]

  },

  "mcpServers": {

    "filesystem": {

      "type": "stdio",

      "command": "npx",

      "args": [

        "-y",

        "@modelcontextprotocol/server-filesystem",

        "/app/librechat_mcp_filesystem"

      ],

      "timeout": 30000,

      "initTimeout": 10000

    },

    "memory": {

      "type": "stdio",

      "command": "npx",

      "args": [

        "-y",

        "@modelcontextprotocol/server-memory"

      ],

      "timeout": 30000,

      "initTimeout": 10000,

      "env": {

        "MEMORY_FILE_PATH": "/app/librechat_mcp_memory/memory.json"

      }

    },

    "time": {

      "type": "stdio",

      "command": "uvx",

      "args": [

        "mcp-server-time",

        "--local-timezone=Australia/Melbourne"

      ],

      "timeout": 30000,

      "initTimeout": 10000

    },

    "web_search": {

      "type": "stdio",

      "command": "uvx",

      "args": [

        "duckduckgo-mcp-server"

      ],

      "timeout": 30000,

      "initTimeout": 10000

    }

  }

}

2025-07-29 14:27:08 info: RAG API is running and reachable at http://host.docker.internal:8008.

2025-07-29 14:27:08 info: No changes needed for 'USER' role permissions

2025-07-29 14:27:08 info: No changes needed for 'ADMIN' role permissions

2025-07-29 14:27:08 info: Turnstile is DISABLED (no siteKey provided).

2025-07-29 14:27:08 info: Configuring social logins...

2025-07-29 14:27:08 info: Server listening on all interfaces at port 3080. Use http://localhost:3080 to access it

2025-07-29 14:27:08 info: Initializing MCP servers...

2025-07-29 14:27:09 info: [indexSync] Messages are fully synced: 96/96

2025-07-29 14:27:09 info: [indexSync] Conversations are fully synced: 15/15

2025-07-29 14:27:09 info: [MCP][filesystem] Access token missing

2025-07-29 14:27:09 info: [MCP][memory] Access token missing

2025-07-29 14:27:09 info: [MCP][time] Access token missing

2025-07-29 14:27:09 info: [MCP][filesystem] Access token missing and no refresh token available

2025-07-29 14:27:09 info: [MCP][filesystem] Setting up OAuth event listener

2025-07-29 14:27:09 info: [MCP][memory] Access token missing and no refresh token available

2025-07-29 14:27:09 info: [MCP][memory] Setting up OAuth event listener

2025-07-29 14:27:09 info: [MCP][web_search] Access token missing

2025-07-29 14:27:09 info: [MCP][time] Access token missing and no refresh token available

2025-07-29 14:27:09 info: [MCP][time] Setting up OAuth event listener

2025-07-29 14:27:09 info: [MCP][web_search] Access token missing and no refresh token available

2025-07-29 14:27:09 info: [MCP][web_search] Setting up OAuth event listener

realpath: --: No such file or directory

realpath: --: No such file or directory

2025-07-29 14:27:09 info: [MCP][time] Instructions not included (serverInstructions not configured)

2025-07-29 14:27:09 info: [MCP][time] Capabilities: {"experimental":{},"tools":{"listChanged":false}}

2025-07-29 14:27:09 info: [MCP][time] Available tools: get_current_time, convert_time

2025-07-29 14:27:09 info: [MCP][time] ✓ Initialized

[07/29/25 14:27:10] INFO     Processing request of type            server.py:625

                             PingRequest                                        

                    INFO     Processing request of type            server.py:625

                             PingRequest                                        

                    INFO     Processing request of type            server.py:625

                             PingRequest                                        

2025-07-29 14:27:10 info: [MCP][web_search] Instructions not included (serverInstructions not configured)

2025-07-29 14:27:10 info: [MCP][web_search] Capabilities: {"experimental":{},"prompts":{"listChanged":false},"resources":{"subscribe":false,"listChanged":false},"tools":{"listChanged":false}}

                    INFO     Processing request of type            server.py:625

                             ListToolsRequest                                   

2025-07-29 14:27:10 info: [MCP][web_search] Available tools: search, fetch_content

2025-07-29 14:27:10 info: [MCP][web_search] ✓ Initialized

Knowledge Graph MCP Server running on stdio

2025-07-29 14:27:12 info: [MCP][memory] Instructions not included (serverInstructions not configured)

2025-07-29 14:27:12 info: [MCP][memory] Capabilities: {"tools":{}}

2025-07-29 14:27:12 info: [MCP][memory] Available tools: create_entities, create_relations, add_observations, delete_entities, delete_observations, delete_relations, read_graph, search_nodes, open_nodes

2025-07-29 14:27:12 info: [MCP][memory] ✓ Initialized

Secure MCP Filesystem Server running on stdio

Allowed directories: [ '/app/librechat_mcp_filesystem' ]

2025-07-29 14:27:12 info: [MCP][filesystem] Instructions not included (serverInstructions not configured)

2025-07-29 14:27:12 info: [MCP][filesystem] Capabilities: {"tools":{}}

2025-07-29 14:27:12 info: [MCP][filesystem] Available tools: read_file, read_multiple_files, write_file, edit_file, create_directory, list_directory, list_directory_with_sizes, directory_tree, move_file, search_files, get_file_info, list_allowed_directories

2025-07-29 14:27:12 info: [MCP][filesystem] ✓ Initialized

2025-07-29 14:27:12 info: [MCP] Initialized 4/4 app-level server(s)

2025-07-29 14:27:12 info: [MCP][filesystem] ✓ Initialized

2025-07-29 14:27:12 info: [MCP][memory] ✓ Initialized

2025-07-29 14:27:12 info: [MCP][time] ✓ Initialized

2025-07-29 14:27:12 info: [MCP][web_search] ✓ Initialized

2025-07-29 14:27:12 info: [MCP] All app-level servers initialized successfully

[07/29/25 14:27:12] INFO     Processing request of type            server.py:625

                             PingRequest                                        

                    INFO     Processing request of type            server.py:625

                             ListToolsRequest                                   

2025-07-29 14:27:12 info: MCP servers initialized successfully

2025-07-29 14:27:48 info: [Login] [Login successful] [Username: [email protected]] [Request-IP: 172.18.0.1]

2025-07-29 14:27:49 error: Failed to load service key from file: /app/api/data/auth.json ENOENT: no such file or directory, open '/app/api/data/auth.json'

2025-07-29 14:27:50 error: Failed to fetch models from Llamaswap API The server responded with status 404: Request failed with status code 404

[07/29/25 14:27:51] INFO     Processing request of type            server.py:625

                             PingRequest                                        

                    INFO     Processing request of type            server.py:625

                             ListToolsRequest                                   

2025-07-29 14:30:13 error: [api/server/controllers/agents/client.js #sendCompletion] Operation aborted 500 JSON schema conversion failed:

Unrecognized schema: {"not":{}}

Unrecognized schema: {"not":{}}

Unrecognized schema: {"not":{}}

Unrecognized schema: {"not":{}}

Unrecognized schema: {"not":{}}

Unrecognized schema: {"not":{}}

Unrecognized schema: {"not":{}}

Unrecognized schema: {"not":{}}

Unrecognized schema: {"not":{}}

Unrecognized schema: {"not":{}}

Unrecognized schema: {"not":{}}

Unrecognized schema: {"not":{}}

2025-07-29 14:30:13 error: [api/server/controllers/agents/client.js #sendCompletion] Unhandled error type 500 JSON schema conversion failed:

Unrecognized schema: {"not":{}}

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

This seems like a shortcoming with the “llama-swap” custom endpoint you are using, maybe it can’t process specific function definitions that are needed for the MCP servers you are using. Have you tried another endpoint?

which specific server are you attempting to use at the time of the error?

[NotYourAverageAl]

This seems like a shortcoming with the “llama-swap” custom endpoint you are using, maybe it can’t process specific function definitions that are needed for the MCP servers you are using. Gave you tried another endpoint?

which specific server are you attempting to use at the time of the error?

@danny-avila Forgot to mention openAI works fine. Ollama seems to stumble sometimes, but mostly works.
the issue stems from either of the memory, filesystem,duckduckgo servers; the time server works fine

[NotYourAverageAl]

@danny-avila Llama-swap works fine with the same mcp server with Jan and open webui. I suspect something is happening with Librechat and llama-swap; notice from the logs librechat is failing to fetch models from Llamaswap API error error: Failed to fetch models from Llamaswap API The server responded with status 404: Request failed with status code 404.
Jan and openwebui can fetch models and work with the same mcps

[NotYourAverageAl]

@danny-avila, i ran llama cpp in docker and I still get the same error, so its not llama-swap related.
Do you get any errors with the same setup?

docker run --gpus=all --pull always --name Qwen3-30B-A3B-Thinking-2507 \ --init --rm -p 8087:8080 -v /home/al/dockerdata/models:/models ghcr.io/ggml-org/llama.cpp:server-cuda \ --model /models/Qwen_Qwen3-30B-A3B-Thinking-2507-Q4_K_M.gguf \ --ctx-size 32768 --no-context-shift \ --gpu-layers 999 \ --jinja \ --parallel 1 \ --flash-attn \ --cache-type-k q8_0 --cache-type-v q8_0 \ --temp 0.6 --min-p 0.00 \ --top-k 20 --top-p 0.95 \ --presence-penalty 1.0 \ --no-mmap

[christianwengert]

I have this problem too with llama.cpp
Unrecognized schema: {"not":{}}

[christianwengert]

I dug a bit into it, but it seems to be in the mcp/zod.ts package where one could set zod to not include any other stuff:

- let objectSchema = z.object(shape);
+ // default to passthrough to avoid ZodNever -> {"not":{}}
+ let objectSchema = z.object(shape).passthrough();

Not sure if there are other places where we need to do that in that file. But I think this could be the solution, but I do not know zod very well

[Te-eMster]

Changing only that line (433) somehow did not work for me. Are there any other places to look?
(I am trying to use Notion MCP via streamable http)