Flux flow response doesn't seem to work as expected

[Mr-LiuDC]

Bug description
Flux flow response doesn't seem to work as expected.

Environment
Spring AI 1.0.0-SNAPSHOT
Java version 21
Windows 10

Steps to reproduce
I Tried OpenAI completions API, it replied SSE content

But the following examples replied nothing.

@PostMapping(value = "/ai/openai/completions", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
Flux<OpenAiApi.ChatCompletionChunk> aiChat(@RequestBody OpenAiApi.ChatCompletionRequest chatRequest) {
    final OpenAiApi openAiApi = new OpenAiApi(openaiBaseUrl, openaiApiKey);
    return openAiApi.chatCompletionStream(chatRequest);
}

That's the same

@GetMapping(value = "/ai/generateStream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
    Prompt prompt = new Prompt(new UserMessage(message));
    return chatModel.stream(prompt);
}

[markpollack]

@tzolov I believe SSE to Flux is handled based on some internal conversions related to content type negotation, try without the produces element in the annotation, e.g.

    @GetMapping("/ai/generateStream")
	public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
        var prompt = new Prompt(new UserMessage(message));
        return this.chatModel.stream(prompt);
    }

and other examples in the docs.

[Mr-LiuDC]

I have tried it and it's the same problem.

[markpollack]

what do you observe? no response? We can make a small app to reproduce, but i've done this type of stuff many times in demos so feel it maybe some other issue in your setup.

[Mr-LiuDC]

what do you observe? no response? We can make a small app to reproduce, but i've done this type of stuff many times in demos so feel it maybe some other issue in your setup.

As I pointed out in the previous picture, it does not continuously output the content of event data, but only responded with a status code of 200.

[sobychacko]

@Mr-LiuDC I am unable to reproduce the issue you are seeing. Here is a simple app that uses the same components you have.

@SpringBootApplication
public class Gh1236Application {
	public static void main(String[] args) {
		SpringApplication.run(Gh1236Application.class, args);
	}
}

@RestController
@RequestMapping("/ai")
class ChatController {
	private final OpenAiChatModel chatModel;

	@Autowired
	public ChatController(OpenAiChatModel chatModel) {
		this.chatModel = chatModel;
	}

	@GetMapping(value = "/generateStream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
	public Flux<ChatResponse> generateStream(
			@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
		return chatModel.stream(new Prompt(new UserMessage(message)));
	}
}

When I invoke the endpoint as below, I get the streaming output without any issues.

curl localhost:8080/ai/generateStream | sed 's/data://' | jq .

Can you start with this sample code and provide us a way to reproduce the issue you are facing? Thanks!