llama3.2-3B Instruct doesnt call tools

[DickyAdi]

Currently im trying to learn how to develop an agentic AI based on langgraph academy video, while in the langgraph academy video is using openAI GPT, i decide to use llama3.2 3B as it is free.

Below is my code

model defintion

tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.2-3B-Instruct", token=hf_token,)
model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.2-3B-Instruct", token=hf_token)
pipe = pipeline('text-generation', model=model, tokenizer=tokenizer, max_length=2048)
hf = HuggingFacePipeline(pipeline=pipe)
llm = ChatHuggingFace(llm=hf).bind(skip_prompt=True)

tools definition

def multiply(a: int, b: int) -> int:
    """Multiply a and b.

    Args:
        a: first int
        b: second int
    """
    return a * b

def add(a: int, b: int) -> int:
    """Adds a and b.

    Args:
        a: first int
        b: second int
    """
    return a + b

def divide(a: int, b: int) -> float:
    """Divide a and b.

    Args:
        a: first int
        b: second int
    """
    return a / b
tools = [add, multiply, divide]

chatbot definition

llm_with_tools = llm.bind_tools(tools)
def assistant(state:MessagesState):
    sys_message = [SystemMessage(content="You are a helpful assistant tasked with performing arithmetic on a set of inputs. You are encouraged to call a function for each of the given question!")]
    return {'messages' : [llm_with_tools.invoke(sys_message + state['messages'])]}

graph building and compiling

builder = StateGraph(MessagesState)
builder.add_node('assistant', assistant)
builder.add_node('tools', ToolNode(tools))
builder.add_edge(START, 'assistant')
builder.add_conditional_edges('assistant', tools_condition)
builder.add_edge('tools', 'assistant')
state_graph = builder.compile()

messages = [HumanMessage(content='What is 99 multiplied by 99?')]
resp = state_graph.invoke({'messages' : messages})
for m in resp['messages']:
    m.pretty_print()

With the given definition and graph, below is the results of the llm when i turn on the debugging mode.

{
  "generations": [
    [
      {
        "text": "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\nCutting Knowledge Date: December 2023\nToday Date: 31 Jan 2025\n\nYou are a helpful assistant tasked with performing arithmetic on a set of inputs. You are encouraged to call a function for each of the given question!<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nWhat is 99 multiplied by 99?<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\nTo calculate 99 multiplied by 99, I'll use the built-in multiplication function.\n\n```python\nresult = 99 * 99\nprint(result)\n```\n\nThe result of 99 multiplied by 99 is: 9801",
        "generation_info": null,
        "type": "ChatGeneration",
        "message": {
          "lc": 1,
          "type": "constructor",
          "id": [
            "langchain",
            "schema",
            "messages",
            "AIMessage"
          ],
          "kwargs": {
            "content": "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\nCutting Knowledge Date: December 2023\nToday Date: 31 Jan 2025\n\nYou are a helpful assistant tasked with performing arithmetic on a set of inputs. You are encouraged to call a function for each of the given question!<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nWhat is 99 multiplied by 99?<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\nTo calculate 99 multiplied by 99, I'll use the built-in multiplication function.\n\n```python\nresult = 99 * 99\nprint(result)\n```\n\nThe result of 99 multiplied by 99 is: 9801",
            "type": "ai",
            "id": "run-19f4ccaa-a9ba-46f1-905a-aab7d3455f02-0",
            "tool_calls": [],
            "invalid_tool_calls": []
          }
        }
      }
    ]
  ],
  "llm_output": null,
  "run": null,
  "type": "LLMResult"
}

Any idea what im doing wrong? or is it because the capability of the llm? AFAIK llama3.2 3B supports function/tool calling. Any explanation or help is very much appreciated.

[vbarda]

you need to use ChatOllama (i think you might be using regular Ollama class)

[DickyAdi]

well, i use chathuggingface with chathuggingfacepipe. But, will it work if i use chatollama?

PS: will edit the post to add the model definition

[vbarda]

i am not sure about huggingface, but yes, i would recommend langchain_ollama.ChatOllama

[DickyAdi]

thanks for the resp, will try it soon

[DickyAdi]

i guess you're right, tried chatollama and the tool calling works. Thanks a lot.