Ollama is not optimal to run your agents

[jdsalasca]

I saw in your ReadMe that you're installing Ollama and running the agent from that place, but that is not efficient for the majority of users. Also, Ollama is highly inefficient, so a lot of people can hit CPU limits running those models. For that reason, I suggest you run a server natively to "mock" the OpenAI connector as an agent. The magic could be in choosing the correct LLM based on User specs, so if it's running on an Nvidia GPU> 3070 with GPU > 8 gb, then pick a GGUf 8b model, if running an Arc > 16 gb/GPU, then run an OpenVino model with 20b parameters...

That part is interesting and could help you improve your project.

God bless you and thanks!

[jeanpaul]

If you are on Linux and have an Nvidia GPU, you can use the gpu profile when starting the ollama docker. Otherwise, if you are running a mac, there are instructions in the readme on how to connect to the Ollama that's running on your mac directly. Does that match your expectations?