Have you tested it with open weights local quantized llm's ?

[Greatz08]

First of all, Great project and the way you present your work on main page is what i like the most and i wonder why people put soooo much efforts into their projects but couldnt put some videos, comparisons and easier instructions to follow so i truly appreciate your work and the way you presented it.

I did enjoyed recent stream of yours with OAlabs regarding mcp+reverse engineering :-)) .

Now coming back to topic, while i was viewing that stream claud sonnet was used for testing which is pretty great model in dev/coding space for a long time and it did worked as well but have you thought about testing it with 14B,22B or 32B range quantized local models ?

I have seen some agentic framework rankings of local llms which performed great but i dont know if their is any mcp based benchmark on which we can see how local small llms perform. If you have tested or find something interesting then do let me know :-)).
( I know reverse engineering might not be that easy thing for llm's because they are not trained for it and especially for small local llms god knows how well they will perform but i want to see that future where we have specialized 14B,21B models who are specialized in coding + reverse engineering and can at least assist us in reverse engineering so that we can save our precious time+brain cells )

Thank you for this project

[mrexodia]

Currently the MCP support for LM Studio and Ollama is lacking, so it would require some custom code to try this. I found https://github.com/patruff/ollama-mcp-bridge, which appears to be roughly what you would need but I did not try it myself. In theory this can be done without custom code, just requires the JSON configuration file, similarly to how it is installed in Cline/Roo Code at the moment.

This might also be useful: https://github.com/SecretiveShell/MCP-Bridge (found it in this blog post)

[williballenthin]

i will try setting up Pydantic AI with Ollama and MCP and report back here.

[williballenthin]

I started by trying to use Gemma 3 1B but it doesn't support tool calling. Will try Llama 3.1 1B next. (Obviously models of this size probably won't be able to do anything intelligent, but it should demonstrate the tool calling works. I've only got a 16GB mac so I can't run very large models right now.)

[mrexodia]

The models with the hammer next to it (allegedly) support tool calling, might be helpful for testing. Qwen 2.5 was quite light on RAM iirc and I was using it with continue.dev for copilot at work in the past.

[williballenthin]

I was able to get Ollama with llama 3.2 1B to call IDA Pro tools when working with oterm:

As mentioned by @mrexodia, Ollama doesn't have built-in support for MCP, but these interfaces seem to work ok to glue Ollama to MCP.

Does this model work well? Surely not - its way too small. But I wanted to explain how I was able to get something up and running. Perhaps someone else can chime in on how well local models can do analysis.