Error code: 400 - {'error': {'message': "An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'. The following tool_call_ids did not have response messages: call_iY0nNb7BlKHtLUT3BRmK28r8", 'type': 'invalid_request_error', 'param': 'messages', 'code': None}}

[mrctito]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

class State(TypedDict):
    messages: Annotated[list, add_messages]
...
    graph_builder = StateGraph(State)
    graph_builder.add_node("agent", self.call_model)
    graph_builder.add_node("tools", self.tool_node)
    graph_builder.add_edge("agent", "tools")
...

async def call_model(self, state: State):
    messages = state["messages"]
    response = await self.llm_with_tools.ainvoke(state["messages"])  #error here
    return {"messages": [response]}

Description

Anybody knows what error this is?

Error code: 400 - {'error': {'message': "An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'. The following tool_call_ids did not have response messages: call_iY0nNb7BlKHtLUT3BRmK28r8", 'type': 'invalid_request_error', 'param': 'messages', 'code': None}}

System Info

langchain==0.2.6
langchain-community==0.2.6
langchain-core==0.2.10
langchain-experimental==0.0.62
langchain-openai==0.1.13
langchain-qdrant==0.1.0
langchain-text-splitters==0.2.1

[dosubot[bot]]

To resolve the error indicating that an assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id', you need to ensure that each tool call in the assistant message is followed by a corresponding tool message that responds to the 'tool_call_id'.

Here's an example of how you can structure your messages to include the necessary tool messages:

from langchain_core.messages import AIMessage, ToolMessage

# Example assistant message with tool calls
assistant_message = AIMessage(
    content='',
    tool_calls=[
        {
            'id': 'call_iY0nNb7BlKHtLUT3BRmK28r8',
            'function': {
                'name': 'YourToolFunctionName',
                'arguments': '{"arg1":"value1","arg2":"value2"}'
            },
            'type': 'function'
        }
    ]
)

# Corresponding tool message responding to the tool call
tool_message = ToolMessage(
    role='tool',
    content='{"result":"Your tool result"}',
    name='YourToolFunctionName',
    tool_call_id='call_iY0nNb7BlKHtLUT3BRmK28r8'
)

# Ensure the tool message follows the assistant message
messages = [assistant_message, tool_message]

Additionally, you can use the bind_tools() method to handle converting and binding tools to the model, ensuring that tool schemas are included in its calls to the LLM. For example:

from langchain_core.tools import tool

@tool
def add(a: int, b: int) -> int:
    """Adds a and b."""
    return a + b

@tool
def multiply(a: int, b: int) -> int:
    """Multiplies a and b."""
    return a * b

tools = [add, multiply]

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0)
llm_with_tools = llm.bind_tools(tools)

When invoking the model, ensure that tool calls are properly handled and tool results are passed back to the model using ToolMessages:

from langchain_core.messages import HumanMessage, ToolMessage

query = "What is 3 * 12? Also, what is 11 + 49?"
messages = [HumanMessage(query)]
ai_msg = llm_with_tools.invoke(messages)
messages.append(ai_msg)
for tool_call in ai_msg.tool_calls:
    selected_tool = {"add": add, "multiply": multiply}[tool_call["name"].lower()]
    tool_output = selected_tool.invoke(tool_call["args"])
    messages.append(ToolMessage(tool_output, tool_call_id=tool_call["id"]))
messages

This ensures that the tool messages are correctly appended and the model can process them accordingly ^{[1][2][3][4][5]}.

To continue talking to Dosu, mention @dosu.

[TomGoodwinCode]

I had this problem because my call_model() function was incorrectly dealing with tool call messages. My prompt template was turning messages with type tool into type human

prompt = ChatPromptTemplate.from_messages(
    [
        ("system", configuration.system_prompt), 
        ("placeholder", "{messages}"),
        ("user", "{input}"),    # <-----  if {input} is type **tool** then it's turned into a 'user' message
    ]
)


model = load_chat_model(configuration.model).bind_tools(TOOLS)

message_value = await prompt.ainvoke(
    {
        "messages": state.messages,
        "input": state.messages[-1].content,  <---- I should never have split off the input state like this
        "title": "London",
        "wiki_title": "London",
        "wiki_extract": "London is the capital city of England.",
        "wiki_description": "London is the capital city of England.",
    },
    config,
)

Obviously this is not correct. I should have been leveraging the ("placeholder", "{messages}") to send all of my messsages. As placeholder maintains the type of the message.

I should have just used placeholder as intended:

prompt = ChatPromptTemplate.from_messages(
    [("system", configuration.system_prompt), ("placeholder", "{messages}")]
)

model = load_chat_model(configuration.model).bind_tools(TOOLS)

message_value = await prompt.ainvoke(
    {
        "messages": state.messages,
        "title": "London",
        "wiki_title": "London",
        "wiki_extract": "London is the capital city of England.",
        "wiki_description": "London is the capital city of England.",
    },
    config,
)

Just thought I would post this incase it gives anyone guidence on solving the same error

[algotechnology]

The worst aspect is that it needs a whole reboot of everything as this error condition (no matter how frequent or infrequent) breaks the whole system. Can this be handled better to recover from this condition (please!). Thanks

[345ishaan]

I am also seeing same error. An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'. The following tool_call_ids did not have response messages: call_femmloc3wNKeN3zrEeg5v8zS", 'type': 'invalid_request_error', 'param': 'messages.[2].role', 'code': None

[345ishaan]

I am also seeing same error. An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'. The following tool_call_ids did not have response messages: call_femmloc3wNKeN3zrEeg5v8zS", 'type': 'invalid_request_error', 'param': 'messages.[2].role', 'code': None

It seems it happens because the llm outputs wrong format for tool function arguments.

[345ishaan]

I started seeing this when i started making a tools which takes non primitive data types as arguments like Dict[str,str]. Is langraph not well supported in those cases @rlancemartin ?

[akashAD98]

im facing this issue pls help me

[s1923]

I change the thread and it works

[emilia-assistente]

Same here, but pretty annoying. I think if you are using threads while testing and some wrong data gets into the thread, then its fucked forever, and there is no way to reset a thread at the moment, that would be nice

[KodyKendall]

Thank you. 👍

[crmado]

I observed an interesting phenomenon: I am switching the blending model from 4o-mini to 4o, and the issue disappeared. This is quite fascinating.

[sagorbrur]

Last year, while working with Langgraph, I encountered an issue related to how tool calls and their corresponding outputs were handled. Here's a breakdown of the problem and its solution:

Problem Description

When a tool call is made, the OpenAI-compatible API expects the next role to be "tool" with the tool's output in the content field. For example:

{
    "role": "tool",
    "content": "tool output"
}

However, in cases where multiple tool calls occur, the API expects multiple "tool" roles, one for each tool call, with corresponding outputs:

{
    "role": "tool",
    "content": "tool output"
}
{
    "role": "tool",
    "content": "tool output"
}

The differentiation is managed through tool_call_id, which uniquely identifies each tool call.

---

Issue Encountered

In my case:

• Multiple tool calls were being invoked (i.e., appearing in the tools=[] array).
• I was returning only one tool output, rather than outputs for all the tool calls.

This mismatch between the number of tool calls and tool outputs raised an error.

---

Solution

To resolve this, I updated my code to ensure that multiple tool calls are handled sequentially rather than in parallel. The fix was implemented as follows:

llm.bind_tools(tools, parallel_tool_calls=False)

By disabling parallel tool calls, I ensured that:

1. Each tool call is associated with a unique output (tool_call_id).
2. The correct tool outputs are sent back to the LLM for subsequent steps.

---

Key Takeaway

For any scenario involving tool calls:

1. Each tool call must be followed by a corresponding tool output before proceeding to the next step.
2. If multiple tools are invoked, ensure that their outputs are sequentially and individually returned to the LLM to avoid mismatches or errors.

By implementing the above fix, the issue was resolved, and I haven't encountered it again since.

[dentroai]

Thank you Sagor, you just saved us 🙏🏼

Got this bug in prod, but with your post we could fix the production environment in under 2 hours.

[xrl0123]

Thanks for the information! Just wanted to add - I did this and it solved my problem:

llm = AzureChatOpenAI(
    azure_deployment=os.getenv('DEPLOYMENT'),  
    api_version=os.getenv('API_VERSION'), 
    temperature=0,
    max_tokens=None,
    timeout=None,
    model = os.getenv('MODEL'),
    model_kwargs={
        'parallel_tool_calls': False}
)

[octaviopavon]

I was spending months with that error, doing workarounds. I used Runnables with Messages history, like Redis, to store my messages. But it turned out that the problem was just simple like:

We were using that RunnableMemory wrapper in the Agents nodes, but not in the ToolNodes, and in consequence, the tool result wasn't uploaded to redis, so , whenever the runnablememory do an invoke, it retrieves the messages from the store, and so, there is the AI MEssage requesting the tool call, but the Tool result wasn't there, so we do the following:

from typing import Any
from langgraph.prebuilt import ToolNode
from app.utils.db_connections import db_connection
from langchain_core.messages import message_to_dict

class ToolNodeWithHistory(ToolNode):
    def __init__(self, *args: Any, **kwargs: Any):
        super().__init__(*args, **kwargs)

    async def ainvoke(self, input: dict, config: dict | None = None) -> dict:
        result = await super().ainvoke(input, config)
        tool_message = result["messages"]
        await db_connection.append_to_message_history(phone_number=config["configurable"]["session_id"], new_history=[message_to_dict(msg) for msg in tool_message])
        return result

We just create a Wrapper for tool node to once the tool call is solved, append to the message history. An that solved our problem from the root :).

[maaroofunbxd]

one of the tools may not be used although it is listed (thereby not producing outputs), thats why probably?

[KodyKendall]

Yes. If the ToolMessage doesn't get saved in the checkpointer, then OpenAI will start to reject all following messages that use that same thread_id, and the thread is tainted unless you manually add in the ToolMessage after the AIMessage to show OpenAI that the tool call was actually invoked.

[KodyKendall]

From when I investigated, it's because there's an AIMessage in the thread_id that has an empty content field, and a parameter ToolCall, but then it lacks the corresponding ToolMessage that should come immediately after it in the message history.

This is what causes OpenAI's API to throw the error.

So if your app throws an error in between the AIMessage that attempts to invoke the tool call is saved in the checkpointer, and when the ToolCall actually occurs (or in the middle of the tool call) and gets saved as a ToolMessage in the checkpointer, then it puts that specific thread_id in an unstable state where subsequent messages to OpenAI will be rejected by their API.