How to run Devstral and mistral-vibe with MLX?

[codrut-popescu]

So, I have downloaded Devstral-2-123B-Instruct-2512-4bit (I am running on a MBP M4 Max/128 GB)

cd ~
python -m venv mlx-env
source ~/mlx-env/bin/activate
pip install mlx-lm

Download Devstral-2-123B-Instruct-2512-4bit
python -c "from huggingface_hub import snapshot_download snapshot_download('mlx-community/Devstral-2-123B-Instruct-2512-4bit',local_dir='/Users/codrut/devstral-2-4bit-mlx')"
Run
mlx_lm.chat --model ~/devstral-2-4bit-mlx
Start as a server

mkdir -p ~/.cache/huggingface/hub
mlx_lm.server --model ~/devstral-2-4bit-mlx --port 8080 

In another terminal window I have installed mistral-vibe

curl -LsSf https://mistral.ai/vibe/install.sh | bash
vibe

in my ~/.vibe/config.toml

I have added

active_model = "mlx-local"

[[providers]]
name = "mlx"
api_base = "http://127.0.0.1:8080/v1"
api_key_env_var = ""
api_style = "openai"
backend = "generic"

[[models]]
name = "devstral-2-4bit-mlx"
provider = "mlx"
alias = "mlx-local"
temperature = 0.2
input_price = 0.0
output_price = 0.0

The problem seems to be that the answer from LLM is very very slow

in the server terminal I can see

2026-02-07 22:00:13,345 - WARNING - Received tools but model does not support tool calling. If you think this is an error, file an issue here: https://github.com/ml-explore/mlx-lm/issues
127.0.0.1 - - [07/Feb/2026 22:00:13] "POST /v1/chat/completions HTTP/1.1" 200 -
2026-02-07 22:01:02,124 - INFO - Prompt processing progress: 2048/6540
2026-02-07 22:01:52,304 - INFO - Prompt processing progress: 4096/6540
2026-02-07 22:02:44,796 - INFO - Prompt processing progress: 6144/6540
2026-02-07 22:02:56,113 - INFO - Prompt processing progress: 6539/6540

is there any solution to this?

[codrut-popescu]

Anyone here?
What does this "Received tools but model does not support tool calling" mean? That mlx supports only dense models?
Please!

[awni]

This means we haven't yet added support for tool calling for that specific model. So if you are using it as a coding agent with mlx_lm.server it won't work yet since the server won't properly parse the tools.

[codrut-popescu]

Thanks for replying. Interesting. Does that mean that there are models which are supported?

[awni]

Yes, GLM, Kimi, M2, Qwen Coder are a few that I know are supported. But we can also add support for Devstral tool calling. I will take a look at it.

[codrut-popescu]

working great in v0.30.7
Thank you sooo much!