On conversational memory

[khteh]

I use llama3.3 model with a ReAct agent. The agent binds to tools such as [Chroma, Neo4J graph database]. Basicall, it is an extension of this tutorial: https://python.langchain.com/docs/tutorials/qa_chat_history/#agents. I use Ollama for model serving and set the context size to 8192. I have built a Quart template UI for the chat agent and the experience has not been great as I notice that the agent is not aware of and does not keep the context of the conversation at all.

What are the missing pieces of the puzzle? How can I enhance the application to be aware of and keep conversation context?

[hinthornw]

I'd recommend using a checkpointer and confirming the conversation history is being sent as expected to the LLM.

You can use langsmith to confirm what data is being sent.

In general if something is misbehaving, I'd suggest looking at the langsmith trace to ensure the data being sent to the LLM contains the history and that the LLM has the prerequisite information to respond appropriately

[khteh]

I have always had checkpointer using PostgreSQL and been using LangSmith. The missing piece I found is that I create thread_id and user_id in the controller blueprint endpoint receiving the request. When every request has a different ID, it won't be possible to correlate them. So, I create the IDs in a new session and persist them in the session. Now, it is better: