Skip to content

fix(bedrock): Handle streaming tool calling with no input arguments #3618

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from
Closed

fix(bedrock): Handle streaming tool calling with no input arguments #3618

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6e4d95c
fix(bedrock): Handle streaming tool calling with no input arguments
tzolov Jun 19, 2025
dad7721
remove redundant test
tzolov Jun 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions ...-converse/src/main/java/org/springframework/ai/bedrock/converse/api/ConverseApiUtils.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -421,8 +421,7 @@ public List<ToolUseEntry> toolUseEntries() {
}

public boolean isEmpty() {
return (this.index == null || this.id == null || this.name == null
|| !StringUtils.hasText(this.partialJson));
return (this.index == null || this.id == null || this.name == null || this.partialJson == null);
}

ToolUseAggregationEvent withIndex(Integer index) {
Expand Down Expand Up @@ -451,7 +450,9 @@ ToolUseAggregationEvent appendPartialJson(String partialJson) {
}

void squashIntoContentBlock() {
this.toolUseEntries.add(new ToolUseEntry(this.index, this.id, this.name, this.partialJson, this.usage));
// Workaround to handle streaming tool calling with no input arguments.
String json = StringUtils.hasText(this.partialJson) ? this.partialJson : "{}";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we should report this as a bug?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure. this appears to be an Anthropic Model's inconsistency b/w the call and stream json payload for parameterless tools.
The behavior is is consistent for all Anthropic models.

this.toolUseEntries.add(new ToolUseEntry(this.index, this.id, this.name, json, this.usage));
this.index = null;
this.id = null;
this.name = null;
Expand Down
10 changes: 8 additions & 2 deletions ...src/test/java/org/springframework/ai/bedrock/converse/client/BedrockNovaChatClientIT.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@
import java.util.stream.Collectors;

import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.ValueSource;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import reactor.core.publisher.Flux;
Expand Down Expand Up @@ -186,12 +188,15 @@ void toolAnnotationWeatherForecast() {
assertThat(response).contains("20 degrees");
}

@Test
void toolAnnotationWeatherForecastStreaming() {
// https://github.com/spring-projects/spring-ai/issues/1878
@ParameterizedTest
@ValueSource(strings = { "amazon.nova-pro-v1:0", "us.anthropic.claude-3-7-sonnet-20250219-v1:0" })
void toolAnnotationWeatherForecastStreaming(String modelName) {

ChatClient chatClient = ChatClient.builder(this.chatModel).build();

Flux<ChatResponse> responses = chatClient.prompt()
.options(ToolCallingChatOptions.builder().model(modelName).build())
.tools(new DummyWeatherForecastTools())
.user("Get current weather in Amsterdam")
.stream()
Expand Down Expand Up @@ -257,6 +262,7 @@ public static class Config {
public BedrockProxyChatModel bedrockConverseChatModel() {

String modelId = "amazon.nova-pro-v1:0";
// String modelId = "us.anthropic.claude-3-7-sonnet-20250219-v1:0";

return BedrockProxyChatModel.builder()
.credentialsProvider(EnvironmentVariableCredentialsProvider.create())
Expand Down
4 changes: 2 additions & 2 deletions pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -264,8 +264,8 @@
<kotlin.version>1.9.25</kotlin.version>

<!-- NOTE: keep bedrockruntime and awssdk versions aligned -->
<bedrockruntime.version>2.31.26</bedrockruntime.version>
<awssdk.version>2.29.29</awssdk.version>
<bedrockruntime.version>2.31.65</bedrockruntime.version>
<awssdk.version>2.31.65</awssdk.version>

<djl.version>0.32.0</djl.version>
<onnxruntime.version>1.19.2</onnxruntime.version>
Expand Down