Web search not working with Qwen3 served with mlx-lm

[Odrec]

What happened?

I am using the latest 0.7.9-rc1 version. I use the websearch with OpenAI models and it works fine. Qwen3 seems to know that there is a tool (as shown in thinking tab) and actually does the search correctly because I saw the links when it was generating the answer but then the answer got deleted and showed this error message. Also in the logs it shows the same error with the StepID.

Is it possible that the error arises because mlx-lm is not formatting the answer in a proper OpenAI format? or is this an issue with LibreChat?

Version Information

ghcr.io/danny-avila/librechat-dev latest 3aac4ce063a1 31 hours ago 1.11GB
ghcr.io/danny-avila/librechat-rag-api-dev latest 64a31a1f9483 7 days ago 7.87GB
ghcr.io/librechat-ai/codesandbox-client/bundler latest 1b2b35059f1d 2 weeks ago 152MB
ghcr.io/danny-avila/librechat-dev 637400d782e3 5 weeks ago 1.06GB
ghcr.io/danny-avila/librechat-rag-api-dev 766bcc667331 5 weeks ago 7.86GB
ghcr.io/librechat-ai/codesandbox-client/bundler 83da1a6ff0dc 6 weeks ago 148MB
ghcr.io/virtuos/librechat_exporter 2.1.0 db9e0104d6af 7 weeks ago 283MB

Steps to Reproduce

1. Serve Qwen 3 235B with mlx-lm
2. Make custom endpoint in LibreChat
3. Activate websearch and ask question that uses it

What browsers are you seeing the problem on?

No response

Relevant log output

{"level":"error","message":"[api/server/controllers/agents/client.js #sendCompletion] Operation aborted No stepId found for tool_call_id undefined","stack":"Error: No stepId found for tool_call_id undefined\n    at StandardGraph.handleToolCallCompleted (/app/node_modules/@librechat/agents/dist/cjs/graphs/Graph.cjs:466:19)\n    at ToolEndHandler.handle (/app/node_modules/@librechat/agents/dist/cjs/events.cjs:63:15)\n    at Run.processStream (/app/node_modules/@librechat/agents/dist/cjs/run.cjs:154:25)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async runAgent (/app/api/server/controllers/agents/client.js:846:9)\n    at async AgentClient.chatCompletion (/app/api/server/controllers/agents/client.js:859:7)\n    at async AgentClient.sendCompletion (/app/api/server/controllers/agents/client.js:548:5)\n    at async AgentClient.sendMessage (/app/api/app/clients/BaseClient.js:670:24)\n    at async AgentController (/app/api/server/controllers/agents/request.js:184:20)\n    at async controller (/app/api/server/routes/agents/chat.js:33:3)"}


2025-07-17 08:42:53 error: Failed to load service key from file: /app/api/data/auth.json ENOENT: no such file or directory, open '/app/api/data/auth.json'
2025-07-17 08:42:53 error: [getUserPluginAuthValue] No plugin auth FIRECRAWL_API_URL found for user 68122240d5bdaf39458ca507
Firecrawl scraper initialized with API URL: https://api.firecrawl.dev/v1/scrape
2025-07-17 08:42:59 info: [onSearchResults] user: 68122240d5bdaf39458ca507 | thread_id: ef4ba103-bede-4984-87fa-e9b00219a993 | run_id: b8ff73fc-6caa-474f-aa10-1b0324ecab5c
2025-07-17 08:42:59 error: Error scraping https://www.nytimes.com/2023/09/29/books/review/benjamin-labatut-the-maniac.html: Firecrawl API request failed: Request failed with status code 403
2025-07-17 08:42:59 error: Error scraping https://www.nytimes.com/2023/09/29/books/review/benjamin-labatut-the-maniac.html: 
2025-07-17 08:43:07 error: Error scraping https://www.clereviewofbooks.com/writing/benjamin-labatut-the-maniac: Firecrawl API request failed: timeout of 7500ms exceeded
2025-07-17 08:43:07 error: Error scraping https://www.clereviewofbooks.com/writing/benjamin-labatut-the-maniac: 
2025-07-17 08:43:07 error: Error scraping https://www.blinkist.com/en/books/the-maniac-en: Firecrawl API request failed: timeout of 7500ms exceeded
2025-07-17 08:43:07 error: Error scraping https://www.blinkist.com/en/books/the-maniac-en: 
2025-07-17 08:43:22 warn: [api/server/controllers/agents/client.js #titleConvo] Error getting endpoint config

Screenshots

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

Thanks for the report @Odrec

Is it possible that the error arises because mlx-lm is not formatting the answer in a proper OpenAI format? or is this an issue with LibreChat?

I'm double-checking this to confirm if we can do anything about this

[Odrec]

Thanks for the report @Odrec

Is it possible that the error arises because mlx-lm is not formatting the answer in a proper OpenAI format? or is this an issue with LibreChat?

I'm double-checking this to confirm if we can do anything about this

Thank you so much! I found this reddit post that seems to confirm mlx-lm has issues formatting the answer with tools for OpenAI

[danny-avila]

Indeed qwen3 has this issue even on different providers. However, I'm still looking to see if we can overcome the edge case it creates.

[Odrec]

An update on this. We have the exact same issues with GLM 4.5, both the full and Air version when served with mlx-lm so it's not exclusive to Qwen models.

[christianwengert]

I have the same issue (same error) with llama.cpp and gpt-oss-120b