How to use 'artifact' attribute in ToolMessage to output both text and image?

[Cybertyann]

Issue with current documentation:

In https://python.langchain.com/api_reference/core/messages/langchain_core.messages.tool.ToolMessage.html, it introduces the concept of 'arifact' and give a simple example:

from langchain_core.messages import ToolMessage
tool_output = {
"stdout": "From the graph we can see that the correlation between x and y is ...",
"stderr": None,
"artifacts": {"type": "image", "base64_data": "/9j/4gIcSU..."},
}

ToolMessage(
content=tool_output["stdout"],
artifact=tool_output,
tool_call_id='call_Jja7J89XsjrOLA5r!MEOW!SL',
)

,
if the tool function called by llm return an output likes tool_output = { "stdout": "From the graph we can see that the correlation between x and y is ...", "stderr": None, "artifacts": {"type": "image", "base64_data": "/9j/4gIcSU..."}, }
, can the agent automatically return a toolmessage contains 'artifact'? Or how to use 'artifact' attribute in ToolMessage to output both text and image? There are few examples about how to get multimodal output. Thank you very much.

Idea or request for content:

No response

[vbarda]

@Cybertyann i recommend checking out this conceptual guide https://python.langchain.com/docs/concepts/tools/#tool-artifacts. the API page is a bit misleading -- you would typically just return tool message content and artifact directly from the tool, as is shown in the concept doc