Why tool call arguments are streamed in separate AIMessageChunk?

[matescharnitzky]

Description

Hi Team,

Thank you for building langgraph, it's a great framework!
I have a question related to the how to stream data from within a tool tutorial. In streaming mode I noticed that not only the the ToolMessageand the final AIMessage is streamed back by tokens in AIMessageChunk (expected), but also the input arguments of the tool call are also streamed. See below.

Message 1: AIMessageChunk with empty content and tool_calls[0].get("name") == "get_items"

AIMessageChunk(
    content='',
    additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_puj4BV8uR3kpv1VVKVtKCrzu', 'function': {'arguments': '', 'name': 'get_items'}, 'type': 'function'}]},
    response_metadata={} id='run-81c87987-f9a7-4b9f-8387-ac1cd373d8c4',
    tool_calls=[{'name': 'get_items', 'args': {}, 'id': 'call_puj4BV8uR3kpv1VVKVtKCrzu', 'type': 'tool_call'}],
    tool_call_chunks=[{'name': 'get_items', 'args': '', 'id': 'call_puj4BV8uR3kpv1VVKVtKCrzu', 'index': 0, 'type': 'tool_call_chunk'}]
)

Message 2: AIMessageChunk with empty content and empty tool_calls

AIMessageChunk(
    content='',
    additional_kwargs={'tool_calls': [{'index': 0, 'id': None, 'function': {'arguments': '{"', 'name': None}, 'type': None}]},
    response_metadata={} id='run-81c87987-f9a7-4b9f-8387-ac1cd373d8c4',
    tool_calls=[{'name': '', 'args': {}, 'id': None, 'type': 'tool_call'}],
    tool_call_chunks=[{'name': None, 'args': '{"', 'id': None, 'index': 0, 'type': 'tool_call_chunk'}]
)

Message 3: AIMessageChunk with empty content and tool_call_chunks[0].get("args")=="place"

AIMessageChunk(
    content='',
    additional_kwargs={'tool_calls': [{'index': 0, 'id': None, 'function': {'arguments': 'place', 'name': None}, 'type': None}]},
    response_metadata={} id='run-81c87987-f9a7-4b9f-8387-ac1cd373d8c4',
    invalid_tool_calls=[{'name': None, 'args': 'place', 'id': None, 'error': None, 'type': 'invalid_tool_call'}],
    tool_call_chunks=[{'name': None, 'args': 'place', 'id': None, 'index': 0, 'type': 'tool_call_chunk'}]
)

Message 4: AIMessageChunk with empty content and tool_call_chunks[0].get("args")==":"

AIMessageChunk(
    content='',
    additional_kwargs={'tool_calls': [{'index': 0, 'id': None, 'function': {'arguments': '":"', 'name': None}, 'type': None}]},
    response_metadata={} id='run-81c87987-f9a7-4b9f-8387-ac1cd373d8c4',
    invalid_tool_calls=[{'name': None, 'args': '":"', 'id': None, 'error': None, 'type': 'invalid_tool_call'}],
    tool_call_chunks=[{'name': None, 'args': '":"', 'id': None, 'index': 0, 'type': 'tool_call_chunk'}]
)

I understand that there are ToolCallChunk that can be merged.

Questions

• What't the reason behind that calls are also streamed in chunks, especially with invalid_tool_calls? I understand why it's an invalid tool call, but I don't understand why this is the default implementation/interface for it.
• ToolMessages are streamed by chunk, but also the full tool message is returned. However, the full tool call is not returned. Is there a standard way to do this other than concatenating the chunks by the index field?

Thank you!

How to reproduce

from dotenv import load_dotenv
load_dotenv()

from langchain_core.callbacks import Callbacks
from langchain_core.messages import HumanMessage
from langchain_core.tools import tool

from langgraph.prebuilt import create_react_agent
from langchain_openai import ChatOpenAI


@tool
async def get_items(
    place: str,
    callbacks: Callbacks,  # <--- Manually accept callbacks (needed for Python <= 3.10)
) -> str:
    """Use this tool to look up which items are in the given place."""
    # Attention when using async, you should be invoking the LLM using ainvoke!
    # If you fail to do so, streaming will not WORK.
    return await llm.ainvoke(
        [
            {
                "role": "user",
                "content": f"Can you tell me what kind of items i might find in the following place: '{place}'. "
                "List at least 3 such items separating them by a comma. And include a brief description of each item..",
            }
        ],
        {"callbacks": callbacks},
    )


llm = ChatOpenAI(model_name="gpt-4o")
tools = [get_items]
agent = create_react_agent(llm, tools=tools)

i = 1
async for msg, metadata in agent.astream(
    {"messages": [("human", "what items are on the shelf?")]}, stream_mode="messages"
):
    print(f"----- Iteration: {i} -----")
    print(f"Message type: {type(msg)}")
    print(f"Message: {msg}")
    print(f"Metadata: {metadata}")
    i += 1

Environment

platform: mac (m3)
python: 3.10.14
langchain: 0.3.1
langgraph: 0.2.39

[ccurme]

Hello,

AIMessageChunks can in all cases be added across a stream to produce a valid message. See example below:

from langgraph.prebuilt import create_react_agent


def get_weather(location: str) -> str:
    """Get weather at a location."""
    return "It's sunny."


agent = create_react_agent("openai:gpt-4.1-mini", [get_weather])

input_message = {
    "role": "user",
    "content": "What's the weather in Boston?",
}

full = None
for chunk, metadata in agent.stream(
    {"messages": [input_message]}, stream_mode="messages"
):
    
    if (metadata["langgraph_step"] == 1):  # initial tool call
        full = chunk if full is None else full + chunk  # <-- add message chunks
        print(full.tool_calls)

[{'name': 'get_weather', 'args': {}, 'id': 'call_XcNgZ0fIYWdPe4ATFZX3ksiS', 'type': 'tool_call'}]
[{'name': 'get_weather', 'args': {}, 'id': 'call_XcNgZ0fIYWdPe4ATFZX3ksiS', 'type': 'tool_call'}]
[{'name': 'get_weather', 'args': {}, 'id': 'call_XcNgZ0fIYWdPe4ATFZX3ksiS', 'type': 'tool_call'}]
[{'name': 'get_weather', 'args': {'location': ''}, 'id': 'call_XcNgZ0fIYWdPe4ATFZX3ksiS', 'type': 'tool_call'}]
[{'name': 'get_weather', 'args': {'location': 'Boston'}, 'id': 'call_XcNgZ0fIYWdPe4ATFZX3ksiS', 'type': 'tool_call'}]
[{'name': 'get_weather', 'args': {'location': 'Boston'}, 'id': 'call_XcNgZ0fIYWdPe4ATFZX3ksiS', 'type': 'tool_call'}]
[{'name': 'get_weather', 'args': {'location': 'Boston'}, 'id': 'call_XcNgZ0fIYWdPe4ATFZX3ksiS', 'type': 'tool_call'}]

Although tool_calls on a single token are typically meaningless, as you accumulate the message chunks you should recover valid tool calls. The concatenation by arg, index, etc. is done for you during the addition.

Hope this helps, let me know if I misunderstood the question.

[luciotudisco]

@ccurme Thanks for the pointer above.

Is that the only way to get the full list of args?

In a context of a multi agent graph where multiple tools can be called in the same run (e.g. supervisor agent transferring to sub agent and sub agent calling a tool) adding the message chunks - like in your example - means that the args of different tools get merged together.

why args needs to be chunked in the first instance? instead of being immediately available in the very first message?

[ccurme]

Thanks for that point. We shouldn't add chunks unless they are from the same message.

You can access both completed messages and individual tokens as the application is running by streaming multiple stream modes. Example:

from langchain_core.messages import AIMessage
from langgraph.prebuilt import create_react_agent


def get_weather(location: str) -> str:
    """Get weather at a location."""
    return "It's sunny."


agent = create_react_agent("openai:gpt-4.1-mini", [get_weather])

input_message = {
    "role": "user",
    "content": "What's the weather in Boston, and also SF?",
}

full = None
for stream_mode, step in agent.stream(
    {"messages": [input_message]},
    stream_mode=["messages", "values"],
):
    if stream_mode == "values":
        # Access completed message
        last_message = step["messages"][-1]
        last_message.pretty_print()
        if isinstance(last_message, AIMessage) and last_message.tool_calls:
            # Access completed tool_calls as desired
            pass
    else:
        # Access tokens
        chunk, metadata = step
        if metadata["langgraph_node"] == "agent":
            # Here, we just print any text tokens that are generated
            text_content = chunk.text()
            if text_content:
                print(text_content)

Tool-call args do not need to be chunked, that is just supported because LLM providers support it.