Tool-calls not working with local Ollama models

[heesuju]

Hello,
I am trying to make an AI agent that plans out tasks according to the user's query similar to Manus AI.

As a test, I made a workflow with only one task using GPT-4o.
This task makes a to-do list and saves it as a text file.
This works perfectly fine until I changed to a local ollama model.

I tried using 'qwen2.5:32b-instruct' and 'qwen3:32b', but the tool-call required to output the text file is not called.
The process terminates right after generating the to-do list, skipping the output step entirely.

Upon closer inspection, I found that no tools were available when praisonai outputs the instruction.
If possible, I would like to know the reason or possible solutions to this problem.

[heesuju]

I tried modifying 'praisonaiagents/agent/agent.py' with the little understanding I have.
To be more specific, I modified line 809 :

tool_param = self.tools if tools is None else tools

to

if tools is None or len(tools) <= 0:
    tool_param = self.tools
else:
    tool_param = tools

'self.tools' had the list of tools while 'tools' was an empty list [](which was not None)
After making this change, the ollama model started executing tools successfully.

However, this causes a problem with agents that call tools optionally.
Even if there is no tool-call, it tries to get the 'name' attribute causing an error message.
This is probably due to me checking length of tools instead.

It seems odd to me that this only happens with local models that use LiteLLM.
I already tried changing some environment variables as suggested by others, but I only got an error saying that I had the wrong API key.
However, I am quite certain I am using the local model through ollama judging by the GPU usage.
The following is the '.env' file I am loading with python-dotenv in linux.

OPENAI_API_KEY=none
OPENAI_API_BASE=http://localhost:11434/v1
OPENAI_BASE_URL=http://localhost:11434/v1
OPENAI_MODEL_NAME=ollama/qwen3:32b
MODEL_NAME=ollama/qwen3:32b

I would appreciate any help.

[heesuju]

Okay so I got the tool-calls to work for ollama models.

All you have to do is provide tools to tasks as well (not just agents).
Most of the examples I have seen so far had tools passed to agents so I naturally assumed it would not matter.
However, it seems like ollama models cannot call those tools from agents for some reason.
After providing the tools to tasks, it works fine.

plan_task = Task(
    name="plan_task",
    description="prompt",
    expected_output="text file",
    agent=planner,
    task_type='task',
    is_start=True,
    tools=[read_file, save_file]
)

[MervinPraison]

@heesuju Thanks for providing this information. I will work on simplifying this.

[ihsankursadunal]

Greetings,

I'm still having some issues using tools with local models.

from praisonaiagents import Agent, Task, PraisonAIAgents

def hello_world_tool(name: str) -> str:
    """
    Says hello using the given argument.

    Args:
        name (str): The name to greet.

    Returns:
        text (str): A greeting message.
    """
    return f"Hello {name}!"

helloworldAgent = Agent(
    name="helloworldAgent",
    llm="qwen2.5:latest",
    backstory="Expert in running tools",
    tools=[hello_world_tool]
)

helloworldTask = Task(
    description="Say hello with hello_world_tool to 'My Little Friend'",
    expected_output="Greeting message",
    agent=helloworldAgent,
    tools=[hello_world_tool]
)

agents = PraisonAIAgents(
    agents=[helloworldAgent],
    tasks=[helloworldTask]
)

agents.start()

However, it just returns:

╭─ Agent Info ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│                                                                                                                                                                                                          │
│  👤 Agent: helloworldAgent                                                                                                                                                                               │
│  Role: Assistant                                                                                                                                                                                         │
│  Tools: hello_world_tool                                                                                                                                                                                 │
│                                                                                                                                                                                                          │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

╭─────────────────────────────── Error ───────────────────────────────╮
│ Error in chat completion: Expecting value: line 1 column 1 (char 0) │
╰─────────────────────────────────────────────────────────────────────╯

This happens several times.
Do you have any ideas about what might be wrong?

I am runing ollama on my docker container btw

[malv-c]

comment llm....
load the framework with
export OPENAI_API_KEY="NA" OPENAI_BASE_URL=http://localhost:11434/v1 OPENAI_MODEL_NAME=qwen3:8b

╭─ Agent Info ──────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ 👤 Agent: helloworldAgent │
│ Role: Assistant │
│ Tools: hello_world_tool │
│ │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────╯

╭─────────────────────────────── Error ───────────────────────────────╮
│ Error in chat completion: Expecting value: line 1 column 1 (char 0) │
╰─────────────────────────────────────────────────────────────────────╯
╭─ Agent Info ──────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ 👤 Agent: helloworldAgent │
│ Role: Assistant │
│ Tools: hello_world_tool │
│ │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────╯

Response generated in 11.5s
╭────────────────────────────────────────────────── Task ───────────────────────────────────────────────────╮
│ You need to do the following task: Say hello with hello_world_tool to 'My Little Friend'. Expected │
│ Output: Greeting message. Please provide only the final result of your work. Do not add any conversation │
│ or extra explanation. │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭──────────────────────────────────────────────── Response ─────────────────────────────────────────────────╮
│ │
│ Hello, My Little Friend! │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────╯
((pai) ) 3.12.10.pyenv:/2/A

the error is qwen too slow

[ihsankursadunal]

Thanks for helping me debug this — I really appreciate your time and input! 🙏

Update:

The response is not looking valid — I actually expect it to be:

Hello My Little Friend!

After trying qwen3:8b and commenting out the llm option, it still gives the same error message five times within 12 seconds.

Also, here are the Ollama logs:

[GIN] 2025/05/20 - 18:29:13 | 200 |  1.669073308s |      172.17.0.1 | POST     "/v1/chat/completions"
[GIN] 2025/05/20 - 18:29:15 | 200 |  1.130180715s |      172.17.0.1 | POST     "/v1/chat/completions"
[GIN] 2025/05/20 - 18:29:17 | 200 |  1.160607878s |      172.17.0.1 | POST     "/v1/chat/completions"
[GIN] 2025/05/20 - 18:29:20 | 200 |  1.162554017s |      172.17.0.1 | POST     "/v1/chat/completions"
[GIN] 2025/05/20 - 18:29:23 | 200 |  1.405119506s |      172.17.0.1 | POST     "/v1/chat/completions"

[MervinPraison]

Two ways of using Ollama

one is here: https://docs.praison.ai/mcp/ollama

1. pip install -U "praisonaiagents[llm]"
2. Agent(llm="ollama/llama3.2", instructions="Write an essay about AI")

other way: https://docs.praison.ai/models/ollama

1. export OPENAI_BASE_URL=http://localhost:11434/v1

More information below:
https://docs.praison.ai/models

Please test each and let me know if its working fine.
I also fixed a recent issue. Thanks

[MervinPraison]

@ihsankursadunal @heesuju please test now

[ihsankursadunal]

Thank you for your response. 🙏

The options you asked us to recreate and test actually do not involve any tool usage.

I tried both of those scenarios, and they work as expected as long as no tool is used.
However, I need to use a tool, and when I include any tool in the agent and/or task, it continues to fail with the same error:

Error in chat completion: Expecting value: line 1 column 1 (char 0)

[MervinPraison]

@ihsankursadunal It could be because of the model you are using. Its working for me as shown below.

also make sure you have upgrade that pip install -U "praisonaiagents[llm]"

❯ python app.py
[09:02:12] INFO     [09:02:12] llm.py:230    llm.py:230
                    INFO Getting response              
                    from openai/gpt-4o-mini            
╭─ Agent Info ────────────────────────────────────────╮
│                                                     │
│  👤 Agent: helloworldAgent                          │
│  Role: Assistant                                    │
│  Tools: hello_world_tool                            │
│                                                     │
╰─────────────────────────────────────────────────────╯

╭───────────────────── Tool Call ─────────────────────╮
│ Agent helloworldAgent called function               │
│ 'hello_world_tool' with arguments: {'name': 'My     │
│ Little Friend'}                                     │
│ Function returned: Hello My Little Friend!          │
╰─────────────────────────────────────────────────────╯
╭──────────────── Generating... 2.7s ─────────────────╮
│ Hello My Little Friend!                             │
╰─────────────────────────────────────────────────────╯
Response generated in 2.7s
╭─────────────────────── Task ────────────────────────╮
│ You need to do the following task: Say hello with   │
│ hello_world_tool to 'My Little Friend'. Expected    │
│ Output: Greeting message. Please provide only the   │
│ final result of your work. Do not add any           │
│ conversation or extra explanation.                  │
╰─────────────────────────────────────────────────────╯
╭───────────────────── Response ──────────────────────╮
│ Hello My Little Friend!                             │
╰─────────────────────────────────────────────────────╯
❯ python app.py
[09:02:21] INFO     [09:02:21] llm.py:230    llm.py:230
                    INFO Getting response              
                    from ollama/llama3.2               
╭─ Agent Info ────────────────────────────────────────╮
│                                                     │
│  👤 Agent: helloworldAgent                          │
│  Role: Assistant                                    │
│  Tools: hello_world_tool                            │
│                                                     │
╰─────────────────────────────────────────────────────╯
╭──────────────── Generating... 4.5s ─────────────────╮
│ {"name": "hello_world_tool", "arguments": {"name":  │
│ "My Little Friend"}}                                │
╰─────────────────────────────────────────────────────╯
╭───────────────────── Tool Call ─────────────────────╮
│ Agent helloworldAgent called function               │
│ 'hello_world_tool' with arguments: {'name': 'My     │
│ Little Friend'}                                     │
│ Function returned: Hello My Little Friend!          │
╰─────────────────────────────────────────────────────╯
╭──────────────── Generating... 4.9s ─────────────────╮
│ Hello My Little Friend!                             │
╰─────────────────────────────────────────────────────╯
Response generated in 4.9s
╭─────────────────────── Task ────────────────────────╮
│ You need to do the following task: Say hello with   │
│ hello_world_tool to 'My Little Friend'. Expected    │
│ Output: Greeting message. Please provide only the   │
│ final result of your work. Do not add any           │
│ conversation or extra explanation.                  │
╰─────────────────────────────────────────────────────╯
╭───────────────────── Response ──────────────────────╮
│ Hello My Little Friend!                             │
╰─────────────────────────────────────────────────────╯
❯ python app.py
[09:02:32] INFO     [09:02:32] llm.py:230    llm.py:230
                    INFO Getting response              
                    from ollama/qwen3                  
╭─ Agent Info ────────────────────────────────────────╮
│                                                     │
│  👤 Agent: helloworldAgent                          │
│  Role: Assistant                                    │
│  Tools: hello_world_tool                            │
│                                                     │
╰─────────────────────────────────────────────────────╯
╭──────────────── Generating... 4.4s ─────────────────╮
│ { "name": "hello_world_tool", "arguments": {        │
│ "name": "My Little Friend" } }                      │
╰─────────────────────────────────────────────────────╯
╭───────────────────── Tool Call ─────────────────────╮
│ Agent helloworldAgent called function               │
│ 'hello_world_tool' with arguments: {'name': 'My     │
│ Little Friend'}                                     │
│ Function returned: Hello My Little Friend!          │
╰─────────────────────────────────────────────────────╯
╭──────────────── Generating... 8.1s ─────────────────╮
│                                                     │
│ Hello My Little Friend!                             │
╰─────────────────────────────────────────────────────╯
Response generated in 8.1s
╭─────────────────────── Task ────────────────────────╮
│ You need to do the following task: Say hello with   │
│ hello_world_tool to 'My Little Friend'. Expected    │
│ Output: Greeting message. Please provide only the   │
│ final result of your work. Do not add any           │
│ conversation or extra explanation.                  │
╰─────────────────────────────────────────────────────╯
╭───────────────────── Response ──────────────────────╮
│                                                     │
│ Hello My Little Friend!                             │
╰─────────────────────────────────────────────────────╯
❯ 

[malv-c]

sorry mervin but it work only one time :(

[MervinPraison]

@malv-c Can you please try using different models

https://docs.praison.ai/models

it also depend on the quality of the model you use.

Please try using some advanced model

[malv-c]

i used
export OPENAI_API_KEY="NA" OPENAI_BASE_URL=http://localhost:11434/v1 OPENAI_MODEL_NAME=hf.co/bartowski/google_gemma-3-27b-it-qat-GGUF:IQ4_XS
slow but hey no gpu !
to remove timeout set it to 0 ?
how to remove trunkating for llm output ?