MessageChatMemoryAdvisor adds empty user message

[fmunch]

Bug description
When calling

ChatClient.builder(chatModel)
        .defaultAdvisors(new MessageChatMemoryAdvisor(chatMemory))
        .build()
        .prompt()
        .call()
        .chatResponse()

MessageChatMemoryAdvisor adds an empty UserMessage in the chat memory (see line 94).

In our use case we send the user message in a POST, store it in the conversation, then send a GET to benefit from SseEmitter. Hence the empty call to prompt().

Environment
1.0.0-M3

Expected behavior
When using an empty prompt MessageChatMemoryAdvisor should probably not add an empty UserMessage in the chat memory.

[DHKIM-0511]

Fix PR #1708

[ThomasVitale]

@fmunch thanks for reporting this. Could you elaborate a bit more on this use case? I'm asking because I see the empty prompt and, in particular, the absence of a user message in ChatClient as an incorrect state (unless it's part of a function calling special flow). Indeed, the current snapshot version of Spring AI throws an exception if there's no user message (whereas in M3 an empty message was created, as you noticed) after adding null-safety to the ChatClient and Advisor APIs. Supporting a null/empty user message would make the behaviour of ChatClient unpredictable and hide pesky errors from developers.

If I understood correctly, your application is adding user messages to the ChatMemory object explicitly, outside the ChatClient workflow. The MessageChatMemoryAdvisor is designed to handle the entire lifecycle of a chat memory for each conversation using an internal ChatMemory object. It might not be the best candidate for handling only parts of the lifecycle.

Since you have already a ChatMemory object where you're storing a user message explicitly, you could pass directly the messages from the memory via the .prompt() clause. And then save the response back into the memory.

Something like this:

var chatResponse = ChatClient.builder(chatModel).build()
                .prompt(new Prompt(chatMemory.get(conversationId, lastN)))
                .call()
                .chatResponse();
chatMemory.add(chatResponse.getResult().getOutput());

You can also use the .messages() clause, if you prefer.

var chatResponse = ChatClient.builder(chatModel).build()
                .prompt()
                .messages(chatMemory.get(conversationId, lastN))
                .call()
                .chatResponse();
chatMemory.add(chatResponse.getResult().getOutput());

Full, standalone example to showcase the behavior.

@Bean
CommandLineRunner chat(ChatClient.Builder chatClientBuilder) {
    return _ -> {
        var chatClient = chatClientBuilder.build();
        var chatMemory = new InMemoryChatMemory();
        var lastN = 10;

        var conversationId = "007";

        var userMessage1 = new UserMessage("My name is Bond. James Bond.");
        chatMemory.add(conversationId, userMessage1);

        var assistantMessage1 = chatClient
                .prompt(new Prompt(chatMemory.get(conversationId, lastN)))
                .call()
                .chatResponse().getResult().getOutput();
        chatMemory.add(conversationId, assistantMessage1);

        System.out.println(assistantMessage1.toString());

        var userMessage2 = new UserMessage("What's my name?");
        chatMemory.add(conversationId, userMessage2);

        var assistantMessage2 = chatClient
                .prompt(new Prompt(chatMemory.get(conversationId, lastN)))
                .call()
                .chatResponse().getResult().getOutput();
        chatMemory.add(conversationId, assistantMessage2);

        System.out.println(assistantMessage2.toString());
    };
}

[fmunch]

@ThomasVitale Thank you for taking the time to reply to this.

To summarize, our Spring AI application has a REST API to add messages in a conversation (via POST calls) and to submit the conversation and stream the text/event-stream response (which requires a GET call when using EventSource AFAIK). The POST requests update the chat memory and the text/event-stream request does not send anything, that's why the prompt is empty, everything is already stored in the chat memory.

Your suggestion of using .messages(chatMemory.get(conversationId, lastN)) would work great for us. I initially decided to use the advisor because it was convenient and to benefit from the rest of it (handling the chatMemoryRetrieveSize for example and what might come in the future).

[markpollack]

closing ashe empty prompt behavior was fixed, and the special use case reported is supported as shown in the example.