Which models from ollama can you use for proper tool server use?

[Fowler]

I started around a bit but could really find any, that would reliably use the tools (for local time eg, or the file system access)

[a-nldisr]

From the documentation I gather that at the moment only ChatGPT 4o is capable.

Tried the go through the model registry and tried by playing around with the following models with native enabled but they didnt work:

• mistral:instruct
• llama3-groq-tool-use:8b
• toolbench-llama2:7b

I dont even see an attempt on the mcpo server, when i directly call the mcpo server i see these calls logged. So even with the model enabled in my open-webui interface and explictly telling the model to use the tools it still is not making any attempt.

See screenshot:

I suspect no opensource models are ReACT-style enabled yet (HORRIBLE NAME BTW! Trying to Google on these terms is impossible without getting react.dev)

[rminchev1]

Yeah, the same experience as well.., perhaps the models don't support the tools calling.. need to dig into the code though 🥷

[a-nldisr]

Going through documentations of each individual model, found this. But no luck:

When I go over the topics here i only see hosted models supporting this (4o, deepseek-chat), no local models so far.

[westtexasfish]

I managed to get open-webui's native tool calling working on self-hosted models running with VLLM. It's nowhere near the ease of use of Ollama, but in my experience, can provide increased inference speed performance over Ollama. It wasn't perfect, and I haven't messed with it much more. I am still new to VLLM, but I'm sure that extended tinkering, and reading the VLLM Docs, would improve the overall tool calling experience.

If interested:
The model I was playing with is ToolAce-2-Llama-3.1-8B. I haven't messed with any of the more generic models inside VLLM, but this one was very fast at tool choice and response. Expect to have to play with the system prompt, and the following VLLM command line options, specifically with tool calling: --enable-auto-tool-choice, --tool-call-parser, --chat-template