Tool call after an initial response is being streamed causes duplicate responses

[Cameronjmayfield]

Bug description
Once a response has started streaming, if the LLM decides to make another tool call, the initial response is repeated.

Environment
Spring AI 1.0.1
Spring Boot 3.5.3

Steps to reproduce
Ask LLM to list available tools and stream the response. If one of the tools might describe a tool, the LLM will execute it, then run the prompt again with the tool response. This causes duplicate answers to stream back because it creates a second LLM call.

Expected behavior
If a tool is executed after the LLM has started generating a response, the tool response should continue from where the previous assistant message completed. Alternatively, allow a tool call to be executed without sending the response to the user OR the LLM.

Minimal Complete Reproducible example
I will work on this.

[seschulz]

which model/provider are you using? i see the same issue on bedrock

[Cameronjmayfield]

Gemini 2.5 through the vertex API does it most often. Easy to reproduce by setting the system prompt to stream data first and ask for a footnote from a tool.

[seschulz]

Ah ok. Then I might open a new issue for bedrock. I analyzed the code, and the issue is in the specific bedrock implementation for me (not in the general spring ai framework). The previous streamed output is ignored before making a new tool execution in the bedrock...so it is missing in the history.

Here you see the hardcoded empty string in org.springframework.ai.bedrock.converse.api.ConverseApiUtils. Maybe this helps already for analyzing the issue here for Vertex.

AssistantMessage assistantMessage = new AssistantMessage("", Map.of(), toolCalls);
Generation toolCallGeneration = new Generation(assistantMessage,
ChatGenerationMetadata.builder().finishReason("tool_use").build());

var chatResponseMetaData = ChatResponseMetadata.builder()
.usage(new DefaultUsage(promptTokens, generationTokens, totalTokens))
.build();