Which local LLM support tool calling

[cometta]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

import {
  AIMessage,
  HumanMessage,
  SystemMessage,
  ToolMessage,
} from "@langchain/core/messages";
import { MessagesAnnotation } from "@langchain/langgraph";
import { ToolNode } from "@langchain/langgraph/prebuilt";

// Step 1: Generate an AIMessage that may include a tool-call to be sent.
async function queryOrRespond(state: typeof MessagesAnnotation.State) {
  const llmWithTools = llm.bindTools([retrieve]);
  const response = await llmWithTools.invoke(state.messages);
  // MessagesState appends messages to state instead of overwriting
  return { messages: [response] };
}

// Step 2: Execute the retrieval.
const tools = new ToolNode([retrieve]);

// Step 3: Generate a response using the retrieved content.
async function generate(state: typeof MessagesAnnotation.State) {
  // Get generated ToolMessages
  let recentToolMessages = [];
  for (let i = state["messages"].length - 1; i >= 0; i--) {
    let message = state["messages"][i];
    if (message instanceof ToolMessage) {
      recentToolMessages.push(message);
    } else {
      break;
    }
  }
  let toolMessages = recentToolMessages.reverse();

  // Format into prompt
  const docsContent = toolMessages.map((doc) => doc.content).join("\n");
  const systemMessageContent =
    "You are an assistant for question-answering tasks. " +
    "Use the following pieces of retrieved context to answer " +
    "the question. If you don't know the answer, say that you " +
    "don't know. Use three sentences maximum and keep the " +
    "answer concise." +
    "\n\n" +
    `${docsContent}`;

  const conversationMessages = state.messages.filter(
    (message) =>
      message instanceof HumanMessage ||
      message instanceof SystemMessage ||
      (message instanceof AIMessage && message.tool_calls.length == 0)
  );
  const prompt = [
    new SystemMessage(systemMessageContent),
    ...conversationMessages,
  ];

  // Run
  const response = await llm.invoke(prompt);
  return { messages: [response] };
}

Description

Can add a readme on which local model support tool calling. I tried the list of local models and get Uncaught (in promise) TypeError: E.bindTools is not a function

System Info

run on chrome browser

[dosubot[bot]]

Hey @cometta, I'm here to help you with any bugs, questions, or contributions you have. Let's tackle this issue together!

The supported local models in LangChainJS that can utilize tool calling include those running via Ollama ^[1].

The error Uncaught (in promise) TypeError: E.bindTools is not a function suggests that the model you are using does not implement the bindTools method. To resolve this, ensure that the local model you are using supports tool calling and has the bindTools method implemented. If the model does not support this method, you may need to switch to a compatible model that does.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[cometta]

aside from ollama, is any model below support tool calling?

Llama-3.2-1B-Instruct-q4f32_1-MLC

Llama-3.2-1B-Instruct-q4f16_1-MLC

Llama-3.2-1B-Instruct-q0f32-MLC

Llama-3.2-1B-Instruct-q0f16-MLC

Llama-3.2-3B-Instruct-q4f32_1-MLC

Llama-3.2-3B-Instruct-q4f16_1-MLC

Llama-3.1-8B-Instruct-q4f32_1-MLC-1k

Llama-3.1-8B-Instruct-q4f16_1-MLC-1k

Llama-3.1-8B-Instruct-q4f32_1-MLC

Llama-3.1-8B-Instruct-q4f16_1-MLC

DeepSeek-R1-Distill-Qwen-7B-q4f16_1-MLC

DeepSeek-R1-Distill-Qwen-7B-q4f32_1-MLC

DeepSeek-R1-Distill-Llama-8B-q4f32_1-MLC

DeepSeek-R1-Distill-Llama-8B-q4f16_1-MLC

Hermes-2-Theta-Llama-3-8B-q4f16_1-MLC

Hermes-2-Theta-Llama-3-8B-q4f32_1-MLC

Hermes-2-Pro-Llama-3-8B-q4f16_1-MLC

Hermes-2-Pro-Llama-3-8B-q4f32_1-MLC

Hermes-3-Llama-3.2-3B-q4f32_1-MLC

Hermes-3-Llama-3.2-3B-q4f16_1-MLC

Hermes-3-Llama-3.1-8B-q4f32_1-MLC

Hermes-3-Llama-3.1-8B-q4f16_1-MLC

Hermes-2-Pro-Mistral-7B-q4f16_1-MLC

Phi-3.5-mini-instruct-q4f16_1-MLC

Phi-3.5-mini-instruct-q4f32_1-MLC

Phi-3.5-mini-instruct-q4f16_1-MLC-1k

Phi-3.5-mini-instruct-q4f32_1-MLC-1k

Mistral-7B-Instruct-v0.3-q4f16_1-MLC

Mistral-7B-Instruct-v0.3-q4f32_1-MLC

Mistral-7B-Instruct-v0.2-q4f16_1-MLC

OpenHermes-2.5-Mistral-7B-q4f16_1-MLC

NeuralHermes-2.5-Mistral-7B-q4f16_1-MLC

WizardMath-7B-V1.1-q4f16_1-MLC

SmolLM2-1.7B-Instruct-q4f16_1-MLC

SmolLM2-1.7B-Instruct-q4f32_1-MLC

SmolLM2-360M-Instruct-q0f16-MLC

SmolLM2-360M-Instruct-q0f32-MLC

SmolLM2-360M-Instruct-q4f16_1-MLC

SmolLM2-360M-Instruct-q4f32_1-MLC

SmolLM2-135M-Instruct-q0f16-MLC

SmolLM2-135M-Instruct-q0f32-MLC

gemma-2-2b-it-q4f16_1-MLC

gemma-2-2b-it-q4f32_1-MLC

gemma-2-2b-it-q4f16_1-MLC-1k

gemma-2-2b-it-q4f32_1-MLC-1k

gemma-2-9b-it-q4f16_1-MLC

gemma-2-9b-it-q4f32_1-MLC

gemma-2-2b-jpn-it-q4f16_1-MLC

gemma-2-2b-jpn-it-q4f32_1-MLC

Qwen2.5-0.5B-Instruct-q4f16_1-MLC

Qwen2.5-0.5B-Instruct-q4f32_1-MLC

Qwen2.5-0.5B-Instruct-q0f16-MLC

Qwen2.5-0.5B-Instruct-q0f32-MLC

Qwen2.5-1.5B-Instruct-q4f16_1-MLC

Qwen2.5-1.5B-Instruct-q4f32_1-MLC

Qwen2.5-3B-Instruct-q4f16_1-MLC

Qwen2.5-3B-Instruct-q4f32_1-MLC

Qwen2.5-7B-Instruct-q4f16_1-MLC

Qwen2.5-7B-Instruct-q4f32_1-MLC

Qwen2.5-Coder-0.5B-Instruct-q4f16_1-MLC

Qwen2.5-Coder-0.5B-Instruct-q4f32_1-MLC

Qwen2.5-Coder-0.5B-Instruct-q0f16-MLC

Qwen2.5-Coder-0.5B-Instruct-q0f32-MLC

Qwen2.5-Coder-1.5B-Instruct-q4f16_1-MLC

Qwen2.5-Coder-1.5B-Instruct-q4f32_1-MLC

Qwen2.5-Coder-3B-Instruct-q4f16_1-MLC

Qwen2.5-Coder-3B-Instruct-q4f32_1-MLC

Qwen2.5-Coder-7B-Instruct-q4f16_1-MLC

Qwen2.5-Coder-7B-Instruct-q4f32_1-MLC

Qwen2.5-Math-1.5B-Instruct-q4f16_1-MLC

Qwen2.5-Math-1.5B-Instruct-q4f32_1-MLC

stablelm-2-zephyr-1_6b-q4f16_1-MLC

stablelm-2-zephyr-1_6b-q4f32_1-MLC

stablelm-2-zephyr-1_6b-q4f16_1-MLC-1k

stablelm-2-zephyr-1_6b-q4f32_1-MLC-1k

RedPajama-INCITE-Chat-3B-v1-q4f16_1-MLC

RedPajama-INCITE-Chat-3B-v1-q4f32_1-MLC

RedPajama-INCITE-Chat-3B-v1-q4f16_1-MLC-1k

RedPajama-INCITE-Chat-3B-v1-q4f32_1-MLC-1k

TinyLlama-1.1B-Chat-v1.0-q4f16_1-MLC

TinyLlama-1.1B-Chat-v1.0-q4f32_1-MLC

TinyLlama-1.1B-Chat-v1.0-q4f16_1-MLC-1k

TinyLlama-1.1B-Chat-v1.0-q4f32_1-MLC-1k

Llama-3.1-70B-Instruct-q3f16_1-MLC

Qwen2-0.5B-Instruct-q4f16_1-MLC

Qwen2-0.5B-Instruct-q0f16-MLC

Qwen2-0.5B-Instruct-q0f32-MLC

Qwen2-1.5B-Instruct-q4f16_1-MLC

Qwen2-1.5B-Instruct-q4f32_1-MLC

Qwen2-7B-Instruct-q4f16_1-MLC

Qwen2-7B-Instruct-q4f32_1-MLC

Qwen2-Math-1.5B-Instruct-q4f16_1-MLC

Qwen2-Math-1.5B-Instruct-q4f32_1-MLC

Qwen2-Math-7B-Instruct-q4f16_1-MLC

Qwen2-Math-7B-Instruct-q4f32_1-MLC

Llama-3-8B-Instruct-q4f32_1-MLC-1k

Llama-3-8B-Instruct-q4f16_1-MLC-1k

Llama-3-8B-Instruct-q4f32_1-MLC

Llama-3-8B-Instruct-q4f16_1-MLC
Llama-3-70B-Instruct-q3f16_1-MLC

Phi-3-mini-4k-instruct-q4f16_1-MLC

Phi-3-mini-4k-instruct-q4f32_1-MLC

Phi-3-mini-4k-instruct-q4f16_1-MLC-1k

Phi-3-mini-4k-instruct-q4f32_1-MLC-1k

Llama-2-7b-chat-hf-q4f32_1-MLC-1k

Llama-2-7b-chat-hf-q4f16_1-MLC-1k

Llama-2-7b-chat-hf-q4f32_1-MLC

Llama-2-7b-chat-hf-q4f16_1-MLC

Llama-2-13b-chat-hf-q4f16_1-MLC

gemma-2b-it-q4f16_1-MLC

gemma-2b-it-q4f32_1-MLC

gemma-2b-it-q4f16_1-MLC-1k

gemma-2b-it-q4f32_1-MLC-1k

phi-2-q4f16_1-MLC

phi-2-q4f32_1-MLC

phi-2-q4f16_1-MLC-1k

phi-2-q4f32_1-MLC-1k

phi-1_5-q4f16_1-MLC

phi-1_5-q4f32_1-MLC

phi-1_5-q4f16_1-MLC-1k

phi-1_5-q4f32_1-MLC-1k

TinyLlama-1.1B-Chat-v0.4-q4f16_1-MLC

TinyLlama-1.1B-Chat-v0.4-q4f32_1-MLC

TinyLlama-1.1B-Chat-v0.4-q4f16_1-MLC-1k

TinyLlama-1.1B-Chat-v0.4-q4f32_1-MLC-1k

[dosubot[bot]]

Yes, aside from Ollama, the models that support tool calling include DeepSeek and Mistral ^[1][2][3].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[cometta]

tested Mistral-7B-Instruct-v0.3-q4f32_1-MLC and DeepSeek-R1-Distill-Llama-8B-q4f32_1-MLC , Uncaught (in promise) Error: llm [object Object] must define bindTools method.