Skip to content

Commit 618c451

Browse files
robinmayerhoferspring-builds
authored andcommitted
Fix reordering issue in AzureOpenAiChatModel (#4156)
Reordering may occur due to `Flux::flatMap` not providing ordering guarantees; `flatMapSequential` addresses this issue. Fixes #4155 Signed-off-by: robinmayerhofer <[email protected]> (cherry picked from commit f888f73)
1 parent 0649db5 commit 618c451

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

models/spring-ai-azure-openai/src/main/java/org/springframework/ai/azure/openai/AzureOpenAiChatModel.java

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -350,7 +350,7 @@ public Flux<ChatResponse> internalStream(Prompt prompt, ChatResponse previousCha
350350
MergeUtils::mergeChatCompletions);
351351
return List.of(reduce);
352352
})
353-
.flatMap(mono -> mono);
353+
.flatMapSequential(mono -> mono);
354354

355355
final Flux<ChatResponse> chatResponseFlux = accessibleChatCompletionsFlux.map(chatCompletion -> {
356356
if (previousChatResponse == null) {
@@ -376,7 +376,7 @@ public Flux<ChatResponse> internalStream(Prompt prompt, ChatResponse previousCha
376376
return chatResponse1;
377377
});
378378

379-
return chatResponseFlux.flatMap(chatResponse -> {
379+
return chatResponseFlux.flatMapSequential(chatResponse -> {
380380
if (this.toolExecutionEligibilityPredicate.isToolExecutionRequired(prompt.getOptions(), chatResponse)) {
381381
// FIXME: bounded elastic needs to be used since tool calling
382382
// is currently only synchronous

0 commit comments

Comments
 (0)