HOWTO: Send intermediate messages back to the user while later chains are processing?

[dkindlund]

When constructing an elaborate set of LLM Chain nodes, is there any way to provide the user with some sort of "intermediate progress" messages back through the chat history window -- while the user waits for the full results to be returned?

I'm thinking about a query that end-to-end might take 2+ minutes to complete. Perhaps something like sending a message as an internal callback webhook via a Custom Tool to the front-end?

[Jaredude]

I don't think this is something that can be accomplished within a chain itself as an intermediate message. You could accomplish this via chat bot client code by streaming inserted text after the user has sent their message and then continue waiting and stream the text back from the server.

[dkindlund]

You could accomplish this via chat bot client code by streaming inserted text after the user has sent their message and then continue waiting and stream the text back from the server.

Okay, this makes sense. It would be nice if the default Flowise client code supported this feature, nonetheless.

[Jaredude]

After looking at the code for FlowiseChatEmbed, I see it's using signals, which is great! I have an idea... I'll submit a PR to that project tomorrow.

[Jaredude]

Made a PR to the chatbot repo to support the ability to observe the signals inside the bot: FlowiseAI/FlowiseChatEmbed#78

Screen.Recording.2024-01-10.at.2.05.28.PM.mov

[HenryHengZJ]

awesome @Jaredude !!

[dkindlund]

@Jaredude , this is really cool! Thank you so much for this PR