Bind Tools do not work with ChatHuggingFace

[frankoz07]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from langchain_huggingface import HuggingFacePipeline
from langchain_huggingface import ChatHuggingFace

llm = HuggingFacePipeline.from_model_id(
    model_id="HuggingFaceH4/zephyr-7b-beta",
    task="text-generation",
    pipeline_kwargs=dict(
        max_new_tokens=512,
        do_sample=False,
        repetition_penalty=1.03,
    ),
)
chat_model = ChatHuggingFace(llm=llm)

from langchain_core.pydantic_v1 import BaseModel, Field


class Calculator(BaseModel):
    """Multiply two integers together."""

    a: int = Field(..., description="First integer")
    b: int = Field(..., description="Second integer")

from langchain_core.output_parsers.openai_tools import PydanticToolsParser

llm_with_multiply = chat_model.bind_tools([Calculator], tool_choice="auto")
parser = PydanticToolsParser(tools=[Calculator])
tool_chain = llm_with_multiply | parser
tool_chain.invoke("How much is 3 multiplied by 12?")

Description

I follow the tutorial Explore the tool calling with ChatHuggingFace.
https://python.langchain.com/v0.2/docs/integrations/chat/huggingface/
There was no error generated, the result as it should be [Calculator(a=3, b=12)],but it just print nothing like []...

System Info

langchain 0.2.3
langchain_huggingface: 0.0.3

[dosubot[bot]]

Hey there, @zwkfrank! I'm here to help you out with any bugs, questions, or contributions you have in mind. Let's dive into this together!

To resolve the issue with the bind_tools method in ChatHuggingFace from the LangChain library, ensure that the tools are correctly formatted and that the tool_choice parameter is properly handled. Here is the relevant code snippet for the bind_tools method:

def bind_tools(
    self,
    tools: Sequence[Union[Dict[str, Any], Type[BaseModel], Callable, BaseTool]],
    *,
    tool_choice: Optional[Union[dict, str, Literal["auto", "none"], bool]] = None,
    **kwargs: Any,
) -> Runnable[LanguageModelInput, BaseMessage]:
    """Bind tool-like objects to this chat model.

    Assumes model is compatible with OpenAI tool-calling API.

    Args:
        tools: A list of tool definitions to bind to this chat model.
            Can be  a dictionary, pydantic model, callable, or BaseTool. Pydantic
            models, callables, and BaseTools will be automatically converted to
            their schema dictionary representation.
        tool_choice: Which tool to require the model to call.
            Must be the name of the single provided function or
            "auto" to automatically determine which function to call
            (if any), or a dict of the form:
            {"type": "function", "function": {"name": <<tool_name>>}}.
        **kwargs: Any additional parameters to pass to the
            :class:`~langchain.runnable.Runnable` constructor.
    """

    formatted_tools = [convert_to_openai_tool(tool) for tool in tools]
    if tool_choice is not None and tool_choice:
        if len(formatted_tools) != 1:
            raise ValueError(
                "When specifying `tool_choice`, you must provide exactly one "
                f"tool. Received {len(formatted_tools)} tools."
            )
        if isinstance(tool_choice, str):
            if tool_choice not in ("auto", "none"):
                tool_choice = {
                    "type": "function",
                    "function": {"name": tool_choice},
                }
        elif isinstance(tool_choice, bool):
            tool_choice = formatted_tools[0]
        elif isinstance(tool_choice, dict):
            if (
                formatted_tools[0]["function"]["name"]
                != tool_choice["function"]["name"]
            ):
                raise ValueError(
                    f"Tool choice {tool_choice} was specified, but the only "
                    f"provided tool was {formatted_tools[0]['function']['name']}."
                )
        else:
            raise ValueError(
                f"Unrecognized tool_choice type. Expected str, bool or dict. "
                f"Received: {tool_choice}"
            )
        kwargs["tool_choice"] = tool_choice
    return super().bind(tools=formatted_tools, **kwargs)

To print the result correctly, ensure that the tools and tool_choice parameters are correctly passed and formatted. Here is an example of how to use the bind_tools method:

from langchain_core.pydantic_v1 import BaseModel, Field

class GetWeather(BaseModel):
    '''Get the current weather in a given location'''
    location: str = Field(..., description="The city and state, e.g. San Francisco, CA")

class GetPopulation(BaseModel):
    '''Get the current population in a given location'''
    location: str = Field(..., description="The city and state, e.g. San Francisco, CA")

chat_with_tools = chat.bind_tools([GetWeather, GetPopulation])
ai_msg = chat_with_tools.invoke("Which city is hotter today and which is bigger: LA or NY?")
print(ai_msg.tool_calls)

Additionally, you can refer to the unit tests provided in the LangChain library to understand how the bind_tools method is tested and validated. For example, the test_bind_tools_errors and test_bind_tools functions in the test_chat_models.py file demonstrate how to handle different scenarios and ensure the method works correctly:

def test_bind_tools_errors(
    chat_hugging_face: Any,
    tools: Dict[str, str],
    tool_choice: Any,
    expected_exception: Any,
    expected_message: str,
) -> None:
    with patch(
        "langchain_huggingface.chat_models.huggingface.convert_to_openai_tool",
        side_effect=lambda x: x,
    ):
        with pytest.raises(expected_exception) as excinfo:
            chat_hugging_face.bind_tools(tools, tool_choice=tool_choice)
        assert expected_message in str(excinfo.value)

def test_bind_tools(chat_hugging_face: Any) -> None:
    tools = [MagicMock(spec=BaseTool)]
    with patch(
        "langchain_huggingface.chat_models.huggingface.convert_to_openai_tool",
        side_effect=lambda x: x,
    ), patch("langchain_core.runnables.base.Runnable.bind") as mock_super_bind:
        chat_hugging_face.bind_tools(tools, tool_choice="auto")
        mock_super_bind.assert_called_once()
        _, kwargs = mock_super_bind.call_args
        assert kwargs["tools"] == tools
        assert kwargs["tool_choice"] == "auto"

This should ensure that the result is printed correctly ^[1][2].

To continue talking to Dosu, mention @dosu.

[hadifar]

Any success in addressing this issue? I also have the same problem.

[rchan26]

Also having the same issue right now if anyone has a fix?

[jedick]

See #29033 for a possible workaround.