The tool called by chat-messages cannot be displayed immediately

[ronylee]

Self Checks

• This is only for bug report, if you would like to ask a question, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)
• Please do not modify this template :) and fill in all the required fields.

Dify version

1.3.0

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

v1/chat-messages

{
"inputs": {},
"query": "content...",
"response_mode": "streaming",
"conversation_id": "",
"user": "superadmin",
"files": [
]
}

The message sending interface calls the tool. The interface only returns after the tool call is completed. There is no data stream information returned when the tool call starts, so the front end cannot display it immediately. This is not a good effect because it takes a long time to get the result. However, we usually do not need to show the result of the tool call.

✔️ Expected Behavior

No response

❌ Actual Behavior

No response

[dosubot[bot]]

The issue you're experiencing with the v1/chat-messages endpoint, where the tool call results are not displayed immediately, is related to how the streaming mode handles tool calls. In streaming mode, the endpoint is designed to return data in chunks as they are generated, but it seems that the tool call results are not being streamed immediately, causing delays in the frontend display.

To address this, you can consider the following:

1. Ensure Streaming Mode is Properly Configured: Verify that the response_mode is set to streaming in your request, which should enable real-time data transmission. This mode is intended to provide immediate feedback by streaming data as it becomes available ^[1].

2. Tool Call Handling: The platform supports tool call handling through events like agent_thought, which includes fields such as tool, tool_input, and observation. These fields can be used to manage tool interactions dynamically during the conversation ^[2].

3. Debugging and Observability: Utilize the DifyAgentCallbackHandler for managing tool call lifecycles, which provides debug information such as tool start, end, and error events. This can help in understanding the tool call process and identifying any delays ^[3].

4. Configuration Options: You can modify the behavior of tool calls and data streaming using configuration options available in Dify. The invoke method in the LargeLanguageModel class includes parameters like tools and stream to configure tool calling and streaming behavior ^[4].

By ensuring these configurations and utilizing the available debugging tools, you should be able to improve the immediate display of tool call results in the frontend.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}