Skip to content

Conversation

@ogirardot
Copy link
Contributor

This PR fixes a null pointer exception that can happen when models check for profanity, selfHarm etc... the last chat completions does not contain any delta and therefor fails at this step.

To reproduce the issue one can just try a simple :

@GetMapping("/ai/generateStream")
    fun generateStream(
        @RequestParam(
            value = "message",
            defaultValue = "Tell me a joke"
        ) message: String?
    ): Flux<ChatResponse> {
        val prompt = Prompt(UserMessage(message))
        return chatModel.stream(prompt)
    }

that will fail with a :

There was an unexpected error (type=Internal Server Error, status=500).
Cannot invoke "com.azure.ai.openai.models.ChatResponseMessage.getToolCalls()" because the return value of "com.azure.ai.openai.models.ChatChoice.getDelta()" is null
java.lang.NullPointerException: Cannot invoke "com.azure.ai.openai.models.ChatResponseMessage.getToolCalls()" because the return value of "com.azure.ai.openai.models.ChatChoice.getDelta()" is null
	at org.springframework.ai.azure.openai.AzureOpenAiChatModel.lambda$internalStream$4(AzureOpenAiChatModel.java:297)
	at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:113)
	at reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onNext(FluxFilterFuseable.java:118)
	at reactor.core.publisher.FluxReplay$UnboundedReplayBuffer.replayNormal(FluxReplay.java:618)
	at reactor.core.publisher.FluxReplay$UnboundedReplayBuffer.replay(FluxReplay.java:709)
	at reactor.core.publisher.FluxReplay$ReplaySubscriber.onNext(FluxReplay.java:1345)
	at reactor.core.publisher.FluxConcatMapNoPrefetch$FluxConcatMapNoPrefetchSubscriber.innerNext(FluxConcatMapNoPrefetch.java:259)
	at reactor.core.publisher.FluxConcatMap$ConcatMapInner.onNext(FluxConcatMap.java:865)
	at reactor.core.publisher.FluxIterable$IterableSubscription.slowPath(FluxIterable.java:335)
	at reactor.core.publisher.FluxIterable$IterableSubscription.request(FluxIterable.java:294)
	at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.set(Operators.java:2367)
	at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onSubscribe(Operators.java:2241)
	at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:201)
	at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:83)
	at reactor.core.publisher.Flux.subscribe(Flux.java:8891)
	at reactor.core.publisher.FluxConcatMapNoPrefetch$FluxConcatMapNoPrefetchSubscriber.onNext(FluxConcatMapNoPrefetch.java:207)
	at reactor.core.publisher.FluxPublishOn$PublishOnSubscriber.runAsync(FluxPublishOn.java:446)
	at reactor.core.publisher.FluxPublishOn$PublishOnSubscriber.run(FluxPublishOn.java:533)
	at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84)
	at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)

@tzolov
Copy link
Contributor

tzolov commented Feb 14, 2025

Thanks @ogirardot for flagging this.
While I agree that the suggested code change provides a general code improvement, the provided example (chatModel.stream(Prompt(UserMessage("Tell me a joke")))) isn't enough to reproduce the issue.
Could you share more context so we can potentially add relevant tests?

@tzolov tzolov self-assigned this Feb 14, 2025
@tzolov tzolov added the bug Something isn't working label Feb 14, 2025
@tzolov tzolov added this to the 1.0.0-M6 milestone Feb 14, 2025
@ilayaperumalg
Copy link
Member

The changes look good and merging it as it captures the null check. For the test, I think we need to produce a prompt in such a way for the response to filter out. I am going ahead with the merge for now.

@ilayaperumalg
Copy link
Member

Merged as dd9e574

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants