Skip to content

Commit 7e7c49f

Browse files
robinmayerhoferchedim
authored andcommitted
Fix reordering issue in AzureOpenAiChatModel (spring-projects#4156)
Reordering may occur due to `Flux::flatMap` not providing ordering guarantees; `flatMapSequential` addresses this issue. Fixes spring-projects#4155 Auto-cherry-pick to 1.0.x Signed-off-by: robinmayerhofer <[email protected]>
1 parent af47629 commit 7e7c49f

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

models/spring-ai-azure-openai/src/main/java/org/springframework/ai/azure/openai/AzureOpenAiChatModel.java

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -351,7 +351,7 @@ public Flux<ChatResponse> internalStream(Prompt prompt, ChatResponse previousCha
351351
MergeUtils::mergeChatCompletions);
352352
return List.of(reduce);
353353
})
354-
.flatMap(mono -> mono);
354+
.flatMapSequential(mono -> mono);
355355

356356
final Flux<ChatResponse> chatResponseFlux = accessibleChatCompletionsFlux.map(chatCompletion -> {
357357
if (previousChatResponse == null) {
@@ -377,7 +377,7 @@ public Flux<ChatResponse> internalStream(Prompt prompt, ChatResponse previousCha
377377
return chatResponse1;
378378
});
379379

380-
return chatResponseFlux.flatMap(chatResponse -> {
380+
return chatResponseFlux.flatMapSequential(chatResponse -> {
381381
if (this.toolExecutionEligibilityPredicate.isToolExecutionRequired(prompt.getOptions(), chatResponse)) {
382382
// FIXME: bounded elastic needs to be used since tool calling
383383
// is currently only synchronous

0 commit comments

Comments
 (0)