Skip to content

Conversation

@ThomasVitale
Copy link
Contributor

@ThomasVitale ThomasVitale commented Apr 18, 2025

  • ChatMemory will become a generic interface to implement different memory management strategies. It’s been moved from the “”spring-ai-client-chat” package to “spring-ai-model” package while retaining the same package, so it’s transparent to users.
  • A MessageWindowChatMemory has been introduced to provide support for a chat memory that keeps at most N messages in the memory.
  • A MessageWindowProcessingPolicy API has been introduced to customise the processing policy for the message window. A default implementation is provided out-of-the-box.
  • A ChatMemoryRepository interface has been introduced to support different storage strategies for the chat memory. It’s meant to be used as part of a ChatMemory implementation. This is different than before, where the storage-specific implementation was directly tied to the ChatMemory. This design is familiar to Spring users since it’s used already in the ecosystem. The goal was to use a programming model similar to Spring Session and Spring Data.
  • The JdbcChatMemory has been supersed by JdbcChatMemoryRepository.
  • The ChatClient now supports memory as a first-class citizen, superseding the need for an Advisor to manage the chat memory. It also simplifies providing a conversationId. This feature lays the foundation for including the intermediate messages in tool calling in the memory as well.
  • All the changes introduced in this PR are backword-compatible.

Usage:

ChatClient chatClient = ChatClient.builder(this.chatModel)
	.defaultMemory(MessageWindowChatMemory.builder().build())
	.build();

String conversationId = "007";

ChatResponse response1 = chatClient.prompt("My name is Bond. James Bond.")
	.conversationId(conversationId)
	.call()
	.chatResponse(); // Hello, James Bond!

ChatResponse response2 = chatClient.prompt("What is my name?")
	.conversationId(conversationId)
	.call()
	.chatResponse(); // Your name is James Bond!

- ChatMemory will become a generic interface to implement different memory management strategies. It’s been moved from the “”spring-ai-client-chat” package to “spring-ai-model” package while retaining the same package, so it’s transparent to users.
- A MessageWindowChatMemory has been introduced to provide support for a chat memory that keeps at most N messages in the memory.

- A MessageWindowProcessingPolicy API has been introduced to customise the processing policy for the message window. A default implementation is provided out-of-the-box.
- A ChatMemoryRepository interface has been introduced to support different storage strategies for the chat memory. It’s meant to be used as part of a ChatMemory implementation. This is different than before, where the storage-specific implementation was directly tied to the ChatMemory. This design is familiar to Spring users since it’s used already in the ecosystem. The goal was to use a programming model similar to Spring Session and Spring Data.
- The JdbcChatMemory has been supersed by JdbcChatMemoryRepository.
- The ChatClient now supports memory as a first-class citizen, superseding the need for an Advisor to manage the chat memory. It also simplifies providing a conversationId. This feature lays the foundation for including the intermediate messages in tool calling in the memory as well.
- All the changes introduced in this PR are backword-compatible.

Signed-off-by: Thomas Vitale <[email protected]>
@markpollack
Copy link
Member

#1722

@ThomasVitale ThomasVitale marked this pull request as draft April 23, 2025 17:23
@deejay1
Copy link
Contributor

deejay1 commented Apr 25, 2025

Should we get this in for 1.0?

@ThomasVitale
Copy link
Contributor Author

For 1.0.0, we are enhancing the core memory APIs: #2890.

Right after 1.0.0 is released, we will introduce first-class memory support in the ChatClient API, whose design requires a bit more time to ensure we comply with all the main requirements.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants