How to use ConversationSummaryMemory in a RunnableSequence?

[mauriciocirelli]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

const model = new ChatOpenAI({
	temperature: 0,
	verbose: false,
});

const chatHistory = new RedisChatMessageHistory({
	sessionId: `chat_history/${chatId}`,
	sessionTTL: parseInt(process.env.REDIS_TTL || "86400"),
	client: redis,
});

const memory = new ConversationSummaryMemory({
	chatHistory: chatHistory,
	llm: model,
	memoryKey: "history",
	inputKey: "question",
	outputKey: "answer",
	returnMessages: false,
});

const chain = RunnableSequence.from([
    ChatPromptTemplate.fromMessages([
      SystemMessagePromptTemplate.fromTemplate(`
        ${PROMPT}
        ------------
        CHAT HISTORY: {history}
      `),
      HumanMessagePromptTemplate.fromTemplate(`
        QUESTION: {question}
      `),
    ]),
    model,
    new StringOutputParser(),
]);

const main = RunnableSequence.from([
    // Gets the history of the conversation
    RunnablePassthrough.assign({ 
      mem: async () => memory.loadMemoryVariables({}),
    }),
    RunnablePassthrough.assign({ 
      history: (input: any) => input.mem[memory.memoryKeys[0]],
    }),
    RunnablePassthrough.assign({ 
      answer: chain,
    }),    
]);


const outputs = await main.invoke({ question: "Example question here...." });
await memory.saveContext(inputs, outputs);

Description

I am having a hard time implementing a ConversationSummaryMemory in a RunnableSequence.
It is a big sequence, so I have ommitted some intermediate chains to keep the question clean.

I would like the history variable to contain the summary of the conversation so far, but everytime I run this code, I can see that although the chatHistory contains the previous messages, the summary is always empty.
I have enabled LangSmith to trace it and I can see that there is a call, outside my RunnableSequence, to a LLMChain that makes the summary.

What is the correct way to use it?

Thank you!

System Info

[email protected] | MIT | deps: 17 | versions: 261

[dosubot[bot]]

Hey @mauriciocirelli! After diving into the details of your implementation and considering the behavior you're experiencing, it seems like the issue might be related to how the ConversationSummaryMemory is being utilized within your RunnableSequence.

From your description, it appears that the chatHistory does indeed contain the previous messages, but the summary remains empty. This could indicate that the process of loading and summarizing the conversation history isn't being executed as expected.

Here are a few steps and considerations to ensure ConversationSummaryMemory works correctly within your sequence:

1. Ensure Proper Initialization: Double-check that the ConversationSummaryMemory is properly initialized with all required parameters. It looks correct in your snippet, but ensuring that the chatHistory, llm, memoryKey, inputKey, and outputKey are correctly set is crucial.

2. Verify Chat History Retrieval: Before attempting to summarize, verify that the chat history is being correctly retrieved from Redis. You can do this by temporarily logging the output of chatHistory.load() to ensure it's not empty.

3. Memory Loading: In the sequence where you're attempting to load the memory variables, ensure that the memory.loadMemoryVariables({}) call is correctly awaited and that it's being passed the necessary context if required. It might be helpful to log the output of this call to ensure it's not empty or missing expected data.

4. Check Summary Generation: The ConversationSummaryMemory relies on the underlying language model (llm) to generate summaries. Ensure that the model is correctly configured to generate summaries and not just responses to prompts. This might require adjusting the model's parameters or ensuring it's invoked with the correct context for summarization.

5. Integration with RunnableSequence: Ensure that the sequence correctly integrates the memory loading and summarization steps. This might involve adjusting the sequence to correctly pass the loaded history to the summarization step and then integrating the summary into the sequence where it's needed.

6. Debugging Tips: Consider isolating the summarization part of your sequence to test it independently. This can help identify if the issue lies within the summarization logic or elsewhere in the sequence.

Given the complexity of integrating ConversationSummaryMemory within a RunnableSequence, it's also a good idea to review any relevant documentation or examples provided by LangChain to ensure there's no missed step or misconfiguration.

If after these steps the issue persists, it might be helpful to reach out to the LangChain community or check if there are any known issues related to ConversationSummaryMemory usage within sequences.

I hope these suggestions help you move forward with your implementation. If you have any more questions or need further assistance, feel free to reach out!

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[mauriciocirelli]

Hi!

I have checked steps (1) and (2).
The chatHistory contains the previous messages and memory.loadMemoryVariables({}) only returns a non-empty object when I set returnMessages: true.

However, I do not want a list of previous messages. I need the summary of the conversation.

Looking at LangSmith, it seems that the summary is being built when I call await memory.saveContext(inputs, outputs);, which happens after the chain.

I have confirmed this behavior:

logger.info(JSON.stringify(await memory.loadMemoryVariables({})));			// Empty!
const inputs = { question: question };
const outputs = await chain.invoke(inputs);
await memory.saveContext(inputs, outputs);
logger.info(JSON.stringify(await memory.loadMemoryVariables({})));			// Summary is available here!

Is there a way to make it build the summary before calling the chain?

[mauriciocirelli]

Hi!

I managed to get this to work by adding the following RunnableSequence to my chain:

const memoryChain = RunnableSequence.from([
    RunnablePassthrough.assign({ 
      msgs: async () => chatHistory.getMessages(),
    }),
    async (input: any) => memory.predictNewSummary(input.msgs, ""),
]);

// Gets the history of the conversation
RunnablePassthrough.assign({ 
  history: memoryChain,
}),