Skip to content

Commit 61198e1

Browse files
Added documentation
1 parent 6abcd59 commit 61198e1

File tree

2 files changed

+29
-2
lines changed

2 files changed

+29
-2
lines changed

docs/guides/ORCHESTRATION_CHAT_COMPLETION.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -186,6 +186,33 @@ var result =
186186

187187
In this example, the input will be masked before the call to the LLM and will remain masked in the output.
188188

189+
### Stream chat completion
190+
191+
It's possible to pass a stream of chat completion delta elements, e.g. from the application backend to the frontend in real-time.
192+
193+
#### Asynchronous Streaming
194+
195+
This is a blocking example for streaming and printing directly to the console:
196+
197+
```java
198+
String msg = "Can you give me the first 100 numbers of the Fibonacci sequence?";
199+
200+
// try-with-resources on stream ensures the connection will be closed
201+
try (Stream<String> stream = client.streamChatCompletion(prompt, config)) {
202+
stream.forEach(
203+
deltaString -> {
204+
System.out.print(deltaString);
205+
System.out.flush();
206+
});
207+
}
208+
```
209+
210+
#### Spring Boot example
211+
212+
Please find [an example in our Spring Boot application](../../sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/controllers/OrchestrationController.java). It shows the usage of Spring
213+
Boot's `ResponseBodyEmitter` to stream the chat completion delta messages to the frontend in real-time.
214+
215+
189216
### Set model parameters
190217

191218
Change your LLM configuration to add model parameters:

sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/controllers/OrchestrationController.java

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -74,8 +74,8 @@ public ResponseEntity<ResponseBodyEmitter> streamChatCompletion() {
7474
try (stream) {
7575
stream.forEach(
7676
deltaMessage -> {
77-
log.info("Controller: {}", deltaMessage.getDeltaContent());
78-
send(emitter, deltaMessage.getDeltaContent());
77+
log.info("Controller: {}", deltaMessage);
78+
send(emitter, deltaMessage);
7979
});
8080
} finally {
8181
emitter.complete();

0 commit comments

Comments
 (0)