MCP issue with Llama cpp docker models

[alhadebe]

What happened?

I have Librechat setup with custom endpoint with llama-swap.
I have setup a couple of mcp servers but I get an error message when I try to call an mcp error: [api/server/controllers/agents/client.js #sendCompletion] Unhandled error type 500 JSON schema conversion failed: Unrecognized schema: {"not":{}}

Any thoughts?

Version Information

ce3b00be875a

Steps to Reproduce

1. Setup llama-swap custom endpoint
2. Setup mcp servers with librechat
3. Activate mcps on a llama-swap model and try calling a tool

What browsers are you seeing the problem on?

Microsoft Edge

Relevant log output

2025-07-29 14:27:05 info: Cleaning up FlowStateManager intervals...

> LibreChat@v0.7.9 backend

> cross-env NODE_ENV=production node api/server/index.js

2025-07-29 14:27:08 info: Connected to MongoDB

2025-07-29 14:27:08 info: [indexSync] Starting index synchronization check...

2025-07-29 14:27:08 info: Custom config file loaded:

2025-07-29 14:27:08 info: {

  "version": "1.2.8",

  "cache": true,

  "interface": {

    "privacyPolicy": {

      "externalUrl": "https://librechat.ai/privacy-policy",

      "openNewTab": true

    },

    "termsOfService": {

      "externalUrl": "https://librechat.ai/tos",

      "openNewTab": true

    }

  },

  "registration": {

    "socialLogins": [

      "discord",

      "facebook",

      "github",

      "google",

      "openid"

    ]

  },

  "endpoints": {

    "custom": [

      {

        "name": "Llamaswap",

        "apiKey": "swap",

        "baseURL": "http://host.docker.internal:65000/v1/",

        "models": {

          "default": [

            "Mistral-Small-3.2-24B-Instruct-2506",

            "Qwen3-30B-A3B",

            "ERNIE-4.5-21B-A3B",

            "EXAONE-4.0-32B",

            "Jan-nano-128k",

            "Qwen3-32B",

            "gemma-3-27b-it-qat",

            "llama3.2-1b",

            "nomic-embed-text-v1.5"

          ],

          "fetch": true

        },

        "titleConvo": true,

        "titleModel": "current_model",

        "summarize": false,

        "summaryModel": "current_model",

        "forcePrompt": false,

        "modelDisplayLabel": "Llamaswap"

      },

      {

        "name": "Ollama",

        "apiKey": "ollama",

        "baseURL": "http://host.docker.internal:11434/v1/",

        "models": {

          "default": [

            ""

          ],

          "fetch": true

        },

        "titleConvo": true,

        "titleModel": "current_model",

        "summarize": false,

        "summaryModel": "current_model",

        "forcePrompt": false,

        "modelDisplayLabel": "Ollama"

      }

    ]

  },

  "mcpServers": {

    "filesystem": {

      "type": "stdio",

      "command": "npx",

      "args": [

        "-y",

        "@modelcontextprotocol/server-filesystem",

        "/app/librechat_mcp_filesystem"

      ],

      "timeout": 30000,

      "initTimeout": 10000

    },

    "memory": {

      "type": "stdio",

      "command": "npx",

      "args": [

        "-y",

        "@modelcontextprotocol/server-memory"

      ],

      "timeout": 30000,

      "initTimeout": 10000,

      "env": {

        "MEMORY_FILE_PATH": "/app/librechat_mcp_memory/memory.json"

      }

    },

    "time": {

      "type": "stdio",

      "command": "uvx",

      "args": [

        "mcp-server-time",

        "--local-timezone=Australia/Melbourne"

      ],

      "timeout": 30000,

      "initTimeout": 10000

    },

    "web_search": {

      "type": "stdio",

      "command": "uvx",

      "args": [

        "duckduckgo-mcp-server"

      ],

      "timeout": 30000,

      "initTimeout": 10000

    }

  }

}

2025-07-29 14:27:08 info: RAG API is running and reachable at http://host.docker.internal:8008.

2025-07-29 14:27:08 info: No changes needed for 'USER' role permissions

2025-07-29 14:27:08 info: No changes needed for 'ADMIN' role permissions

2025-07-29 14:27:08 info: Turnstile is DISABLED (no siteKey provided).

2025-07-29 14:27:08 info: Configuring social logins...

2025-07-29 14:27:08 info: Server listening on all interfaces at port 3080. Use http://localhost:3080 to access it

2025-07-29 14:27:08 info: Initializing MCP servers...

2025-07-29 14:27:09 info: [indexSync] Messages are fully synced: 96/96

2025-07-29 14:27:09 info: [indexSync] Conversations are fully synced: 15/15

2025-07-29 14:27:09 info: [MCP][filesystem] Access token missing

2025-07-29 14:27:09 info: [MCP][memory] Access token missing

2025-07-29 14:27:09 info: [MCP][time] Access token missing

2025-07-29 14:27:09 info: [MCP][filesystem] Access token missing and no refresh token available

2025-07-29 14:27:09 info: [MCP][filesystem] Setting up OAuth event listener

2025-07-29 14:27:09 info: [MCP][memory] Access token missing and no refresh token available

2025-07-29 14:27:09 info: [MCP][memory] Setting up OAuth event listener

2025-07-29 14:27:09 info: [MCP][web_search] Access token missing

2025-07-29 14:27:09 info: [MCP][time] Access token missing and no refresh token available

2025-07-29 14:27:09 info: [MCP][time] Setting up OAuth event listener

2025-07-29 14:27:09 info: [MCP][web_search] Access token missing and no refresh token available

2025-07-29 14:27:09 info: [MCP][web_search] Setting up OAuth event listener

realpath: --: No such file or directory

realpath: --: No such file or directory

2025-07-29 14:27:09 info: [MCP][time] Instructions not included (serverInstructions not configured)

2025-07-29 14:27:09 info: [MCP][time] Capabilities: {"experimental":{},"tools":{"listChanged":false}}

2025-07-29 14:27:09 info: [MCP][time] Available tools: get_current_time, convert_time

2025-07-29 14:27:09 info: [MCP][time] ✓ Initialized

[07/29/25 14:27:10] INFO     Processing request of type            server.py:625

                             PingRequest                                        

                    INFO     Processing request of type            server.py:625

                             PingRequest                                        

                    INFO     Processing request of type            server.py:625

                             PingRequest                                        

2025-07-29 14:27:10 info: [MCP][web_search] Instructions not included (serverInstructions not configured)

2025-07-29 14:27:10 info: [MCP][web_search] Capabilities: {"experimental":{},"prompts":{"listChanged":false},"resources":{"subscribe":false,"listChanged":false},"tools":{"listChanged":false}}

                    INFO     Processing request of type            server.py:625

                             ListToolsRequest                                   

2025-07-29 14:27:10 info: [MCP][web_search] Available tools: search, fetch_content

2025-07-29 14:27:10 info: [MCP][web_search] ✓ Initialized

Knowledge Graph MCP Server running on stdio

2025-07-29 14:27:12 info: [MCP][memory] Instructions not included (serverInstructions not configured)

2025-07-29 14:27:12 info: [MCP][memory] Capabilities: {"tools":{}}

2025-07-29 14:27:12 info: [MCP][memory] Available tools: create_entities, create_relations, add_observations, delete_entities, delete_observations, delete_relations, read_graph, search_nodes, open_nodes

2025-07-29 14:27:12 info: [MCP][memory] ✓ Initialized

Secure MCP Filesystem Server running on stdio

Allowed directories: [ '/app/librechat_mcp_filesystem' ]

2025-07-29 14:27:12 info: [MCP][filesystem] Instructions not included (serverInstructions not configured)

2025-07-29 14:27:12 info: [MCP][filesystem] Capabilities: {"tools":{}}

2025-07-29 14:27:12 info: [MCP][filesystem] Available tools: read_file, read_multiple_files, write_file, edit_file, create_directory, list_directory, list_directory_with_sizes, directory_tree, move_file, search_files, get_file_info, list_allowed_directories

2025-07-29 14:27:12 info: [MCP][filesystem] ✓ Initialized

2025-07-29 14:27:12 info: [MCP] Initialized 4/4 app-level server(s)

2025-07-29 14:27:12 info: [MCP][filesystem] ✓ Initialized

2025-07-29 14:27:12 info: [MCP][memory] ✓ Initialized

2025-07-29 14:27:12 info: [MCP][time] ✓ Initialized

2025-07-29 14:27:12 info: [MCP][web_search] ✓ Initialized

2025-07-29 14:27:12 info: [MCP] All app-level servers initialized successfully

[07/29/25 14:27:12] INFO     Processing request of type            server.py:625

                             PingRequest                                        

                    INFO     Processing request of type            server.py:625

                             ListToolsRequest                                   

2025-07-29 14:27:12 info: MCP servers initialized successfully

2025-07-29 14:27:48 info: [Login] [Login successful] [Username: xxxx@yahoo.com] [Request-IP: 172.18.0.1]

2025-07-29 14:27:49 error: Failed to load service key from file: /app/api/data/auth.json ENOENT: no such file or directory, open '/app/api/data/auth.json'

2025-07-29 14:27:50 error: Failed to fetch models from Llamaswap API The server responded with status 404: Request failed with status code 404

[07/29/25 14:27:51] INFO     Processing request of type            server.py:625

                             PingRequest                                        

                    INFO     Processing request of type            server.py:625

                             ListToolsRequest                                   

2025-07-29 14:30:13 error: [api/server/controllers/agents/client.js #sendCompletion] Operation aborted 500 JSON schema conversion failed:

Unrecognized schema: {"not":{}}

Unrecognized schema: {"not":{}}

Unrecognized schema: {"not":{}}

Unrecognized schema: {"not":{}}

Unrecognized schema: {"not":{}}

Unrecognized schema: {"not":{}}

Unrecognized schema: {"not":{}}

Unrecognized schema: {"not":{}}

Unrecognized schema: {"not":{}}

Unrecognized schema: {"not":{}}

Unrecognized schema: {"not":{}}

Unrecognized schema: {"not":{}}

2025-07-29 14:30:13 error: [api/server/controllers/agents/client.js #sendCompletion] Unhandled error type 500 JSON schema conversion failed:

Unrecognized schema: {"not":{}}

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

This seems like a shortcoming with the “llama-swap” custom endpoint you are using, maybe it can’t process specific function definitions that are needed for the MCP servers you are using. Have you tried another endpoint?

which specific server are you attempting to use at the time of the error?

[alhadebe]

This seems like a shortcoming with the “llama-swap” custom endpoint you are using, maybe it can’t process specific function definitions that are needed for the MCP servers you are using. Gave you tried another endpoint?

which specific server are you attempting to use at the time of the error?

@danny-avila Forgot to mention openAI works fine. Ollama seems to stumble sometimes, but mostly works.
the issue stems from either of the memory, filesystem,duckduckgo servers; the time server works fine

[alhadebe]

@danny-avila Llama-swap works fine with the same mcp server with Jan and open webui. I suspect something is happening with Librechat and llama-swap; notice from the logs librechat is failing to fetch models from Llamaswap API error error: Failed to fetch models from Llamaswap API The server responded with status 404: Request failed with status code 404.
Jan and openwebui can fetch models and work with the same mcps

[alhadebe]

@danny-avila, i ran llama cpp in docker and I still get the same error, so its not llama-swap related.
Do you get any errors with the same setup?

docker run --gpus=all --pull always --name Qwen3-30B-A3B-Thinking-2507 \ --init --rm -p 8087:8080 -v /home/al/dockerdata/models:/models ghcr.io/ggml-org/llama.cpp:server-cuda \ --model /models/Qwen_Qwen3-30B-A3B-Thinking-2507-Q4_K_M.gguf \ --ctx-size 32768 --no-context-shift \ --gpu-layers 999 \ --jinja \ --parallel 1 \ --flash-attn \ --cache-type-k q8_0 --cache-type-v q8_0 \ --temp 0.6 --min-p 0.00 \ --top-k 20 --top-p 0.95 \ --presence-penalty 1.0 \ --no-mmap

[christianwengert]

I have this problem too with llama.cpp
Unrecognized schema: {"not":{}}

[christianwengert]

I dug a bit into it, but it seems to be in the mcp/zod.ts package where one could set zod to not include any other stuff:

- let objectSchema = z.object(shape);
+ // default to passthrough to avoid ZodNever -> {"not":{}}
+ let objectSchema = z.object(shape).passthrough();

Not sure if there are other places where we need to do that in that file. But I think this could be the solution, but I do not know zod very well

[Te-eMster]

Changing only that line (433) somehow did not work for me. Are there any other places to look?
(I am trying to use Notion MCP via streamable http)

[marcmcd]

@danny-avila
@alhadebe
@christianwengert
@Te-eMster
@DKingAlpha
@WillCodeForCash
@chadvoegele

AFAIK, there have been three LibreChat or "@modelcontextprotocol/server-filesystem" issues that have prevented the combination of llama.cpp + LibreChat + "@modelcontextprotocol/server-filesystem" from working, I've listed them below.

Of these three, only issue 1 titled "Unrecognized schema: {"not":{}}" remains to be fixed (which is also mentioned by DKingAlpha in issue 2, but it's really issue 1).
I submitted this issue to perplexity dot ai and it produced a zod.ts patch that I have included further below.
This patch fixes the "Unrecognized schema: {"not":{}}" issue on my system.
Note that I'm using the filesystem MCP server patched fork "https://github.com/danny-avila/server-filesystem".

My system:

• Windows 10
• llama.cpp Git SHA-1: 746f9ee88941c2f259268c484fe8278375387081
• LibreChat v0.8.1-rc2 with packages/api/src/mcp/zod.ts patch (see further below)
  (LibreChat is not running inside a Docker container, while MongoDB and MeiliSearch are running in a Docker container, this should not change this fix)
• "@modelcontextprotocol/server-filesystem" with danny-avila filesystem MCP server patched fork: https://github.com/danny-avila/server-filesystem

---

Three llama.cpp + LibreChat + "@modelcontextprotocol/server-filesystem" issues:

llama.cpp - LibreChat - filesystem MCP Issues:

  -------------------------------------------------------------------
  Issue 1 Title "Unrecognized schema: {"not":{}}"
  
  Issue #8732
  https://github.com/danny-avila/LibreChat/issues/8732
  alhadebe
  opened on Jul 29
  "[Bug]: MCP issue with Llama-swap #8732"
  Closed

    danny-avila on Jul 29, 2025
    converted this issue into a discussion #8736
    "MCP issue with Llama cpp docker models #8736"
    https://github.com/danny-avila/LibreChat/discussions/8736
    
      Solution?: apply zod.ts patch included further below


  -------------------------------------------------------------------
  Issue 2 Title "Support for the integer/float JSON Schema type"
  
  Issue #9446
  https://github.com/danny-avila/LibreChat/issues/9446
  Te-eMster
  opened on Sep 4, 2025
  "[Bug]: zod schema definition breaks with llama.cpp when having optional parameters #9446"
  Closed
  but also see comment by DKingAlpha on Sep 30, 2025
  https://github.com/danny-avila/LibreChat/issues/9446#issuecomment-3350326364  
  "Not exactly a bug, but there's a similiar issue related to json schema 
  generated by librechat which llama.cpp cannot handle.
  To be fair, its overcomplicated for a json schema user to parse string 
  to typed value using schema like:

  "width": {"anyOf": [{ "anyOf": [{"not": {}},{"type": "number"}]},{"type": "null"}]},


    Solution: 
    danny-avila on Sep 5, 2025
    linked a pull request that will close this issue: 
    "feat: Add Support for Integer and Float JSON Schema Types #9469"
    https://github.com/danny-avila/LibreChat/pull/9469
    

  -------------------------------------------------------------------
  Issue 3 Title "Required server-filesystem patch"

  Issue #10578
  https://github.com/danny-avila/LibreChat/issues/10578
  WillCodeForCash
  opened ~Nov 28, 2025
  "[Bug]: [MCP][filesystem] Does not work #10578"
  Closed

    Solution:
    danny-avila on ~Nov 28, 2025?
    converted this issue into a discussion #8736
    "[MCP][filesystem] Does not work #10579"
    https://github.com/danny-avila/LibreChat/discussions/10579
    "I've patched the issue in a fork by explicitly pinning zod: "^3.23.3" as a dependency, 
    forcing all packages to use compatible Zod v3 versions."
    Patched fork: https://github.com/danny-avila/server-filesystem
    

---

zod.ts patch:

 packages/api/src/mcp/zod.ts | 43 ++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 40 insertions(+), 3 deletions(-)

diff --git a/packages/api/src/mcp/zod.ts b/packages/api/src/mcp/zod.ts
index ea9d17c0b..4b6f05764 100644
--- a/packages/api/src/mcp/zod.ts
+++ b/packages/api/src/mcp/zod.ts
@@ -1,6 +1,40 @@
 import { z } from 'zod';
 import type { JsonSchemaType, ConvertJsonSchemaToZodOptions } from '@librechat/data-schemas';
 
+function replaceNever(schema: z.ZodType): z.ZodType {
+  if (schema._def.typeName === 'ZodNever') {
+    return z.unknown();
+  }
+  
+  // Add intersection handling (common source of ZodNever)
+  if (schema._def.typeName === 'ZodIntersection' && 'left' in schema._def && 'right' in schema._def) {
+    return z.intersection(
+      replaceNever((schema as any)._def.left),
+      replaceNever((schema as any)._def.right)
+    );
+  }
+  
+  // Recursively handle objects, unions, intersections, arrays
+  if (schema._def.typeName === 'ZodObject' && 'shape' in schema._def) {
+    const shape = (schema as any)._def.shape();
+    const newShape = Object.fromEntries(
+      Object.entries(shape).map(([key, sub]) => [key, replaceNever(sub)]),
+    );
+    return z.object(newShape);
+  }
+  
+  if (schema._def.typeName === 'ZodUnion' && 'options' in schema._def) {
+    const options = (schema as any)._def.options.map(replaceNever);
+    return z.union(options as any);
+  }
+  
+  if (schema._def.typeName === 'ZodArray' && 'type' in schema._def) {
+    return z.array(replaceNever((schema as any)._def.type));
+  }
+  
+  return schema;
+}
+
 function isEmptyObjectSchema(jsonSchema?: JsonSchemaType): boolean {
   return (
     jsonSchema != null &&
@@ -436,13 +470,13 @@ export function convertJsonSchemaToZod(
       const partial = Object.fromEntries(
         Object.entries(shape).map(([key, value]) => [
           key,
-          schema.required?.includes(key) === true ? value : value.optional().nullable(),
+          schema.required?.includes(key) === true ? value : value.optional(),
         ]),
       );
       objectSchema = z.object(partial);
     } else {
       const partialNullable = Object.fromEntries(
-        Object.entries(shape).map(([key, value]) => [key, value.optional().nullable()]),
+        Object.entries(shape).map(([key, value]) => [key, value.optional()]),
       );
       objectSchema = z.object(partialNullable);
     }
@@ -459,7 +493,7 @@ export function convertJsonSchemaToZod(
       );
       zodSchema = objectSchema.catchall((additionalSchema ?? z.unknown()) as z.ZodType);
     } else {
-      zodSchema = objectSchema;
+      zodSchema = objectSchema.passthrough(); // fallback to passthrough here
     }
   } else {
     zodSchema = z.unknown();
@@ -469,6 +503,9 @@ export function convertJsonSchemaToZod(
   if (schema.description != null && schema.description !== '') {
     zodSchema = zodSchema.describe(schema.description);
   }
+  
+  // ADD THIS LINE HERE - ONCE, before return
+  zodSchema = replaceNever(zodSchema);
 
   return zodSchema;
 }

[chadvoegele]

@marcmcd Thanks for your investigation and patch!

I have been trying to understand the issue better myself and was trying to nail down where the "not": {}" fragment comes from.

Both the patched and upstream filesystem MCP servers use for the read_text_file schema:

const ReadTextFileArgsSchema = z.object({
  path: z.string(),
  tail: z.number().optional(),
  head: z.number().optional()
});

LibreChat and the patched filesystem MCP use zod version 3 for the schema and use zod-to-json-schema to convert to a JSON schema.

The "not": {} gets produced in this conversion.

import { z as z3 } from "zod/v3";
import { zodToJsonSchema } from "./dist/cjs/zodToJsonSchema.js";

let s3 = zodToJsonSchema(
  z3.object({
    head: z3.nullable(z3.optional(z3.number())),
    path: z3.string()
  }),
)
console.log(s3)

// {
//   type: 'object',
//   properties: { head: { anyOf: [Array] }, path: { type: 'string' } },
//   required: [ 'path' ],
//   additionalProperties: false,
//   '$schema': 'http://json-schema.org/draft-07/schema#'
// }

console.log(s3.properties.head)
// { anyOf: [ { anyOf: [Array] }, { type: 'null' } ] }

console.log(s3.properties.head.anyOf[0])
// { anyOf: [ { not: {} }, { type: 'number' } ] }

Zod v4 has native JSON schema conversion, and the "not": {} doesn't present there.

import { z as z4 } from "zod/v4";

let s4 = z4.toJSONSchema(z4.object({
    head: z4.nullable(z4.optional(z4.number())),
    path: z4.string()
  }))
console.log(s4)
// {
//   '$schema': 'https://json-schema.org/draft/2020-12/schema',
//   type: 'object',
//   properties: { head: { anyOf: [Array] }, path: { type: 'string' } },
//   required: [ 'path' ],
//   additionalProperties: false
// }

console.log(s4.properties.head)
// { anyOf: [ { type: 'number' }, { type: 'null' } ] }

Long term the fix seems to be to upgrade LibreChat and filesystem MCP to zod v4. We'd have to check with @danny-avila to see if there are plans for that, and how we could help.

Short term we can try to patch zod-to-json-schema with something like

diff --git a/src/parsers/optional.ts b/src/parsers/optional.ts
index 9d62924..0c3a732 100644
--- a/src/parsers/optional.ts
+++ b/src/parsers/optional.ts
@@ -20,9 +20,7 @@ export const parseOptionalDef = (
   return innerSchema
     ? {
         anyOf: [
-          {
-            not: parseAnyDef(refs),
-          },
+          { type: "null" },
           innerSchema,
         ],
       }

Putting this patch into LibreChat makes the filesystem MCP work also.

It looks like the maintainer of zod-to-json-schema is stepping down so we'd have to see what options are available for releasing the patch.

[danny-avila]

Try the latest version of LibreChat, it also works now without bumping any of the mentioned packages:

mcpServers:
  filesystem:
    type: stdio
    command: npx
    args:
      - -y
      - "@modelcontextprotocol/server-filesystem"
      - /home/danny/LibreChat/

[chadvoegele]

Hi @danny-avila, thanks for your reply!

I agree that the upstream server-filesystem works in LibreChat with OpenAI endpoints, and likely with other cloud providers as well.

The issue that I'm seeing comes up when using llama-server from llama.cpp as a custom endpoint. The "not": {} clause in the grammar which is coming from the now-deprecated zod-to-json-schema is not supported in llama-server, and will likely not be (llama.cpp grammar README).

The fix I'm proposing is to upgrade to zod v4 which removes the "not": {} from the grammar.

[vaaale]

I'm seeing the same issues when using basically any MCP server with llama-server.
The latest librechat version does not fix the issue, nor does any of the latest branches (main / dev) work as of the time of writing.
It would be very nice if this issue could be resolved.

On a slightly different note. While implementing support for specifying the flag "returnDirect" on tools and actions, I upgraded langchain to the latest version. I also saw dozens of type errors and failing tests. Having said that, I think it's important to get langchain updated so that librechat is able to keep up with the development in the agent space. Just my opinion though.

[JMLX42]

The fix I'm proposing is to upgrade to zod v4 which removes the "not": {} from the grammar.

I strongly support this suggestion. Not a single MCP server is working with llama-server because of this zod bug. Meaning LibreChat users are left with the more lenient and slower Ollama. IMHO that's a big issue.

@chadvoegele have you opened a PR yet? Let me know how I can help.

[chadvoegele]

Nope, I didn't open a PR yet. I was hoping to get some kind of go ahead from @danny-avila before starting since it's a relatively larger change.

For now, I'm manually patching the container by changing /app/node_modules/zod-to-json-schema/dist/esm/parsers/optional.js to anyOf: [ { type: "null" }, ... ].

Would be great to get an upstream-ed fix!

[JMLX42]

FYI I don't have the not: {} problem anymore on librechat-dev:252a5cc7ca37cb6cee78a4e75ad9569d3c9877c1