feat: SpringAI integration in OpenAI (#538)

* first draft

* Align with docs

* Codestyle

* feat: [OpenAI] Spring AI integration

* test

* test

* Fixing errors in OpenAiChatOptions according to "Upgrade to Spring AI 1.0.0 (GA Version) (#503)"

* Fixing errors in OpenAiChatOptions according to "Upgrade to Spring AI 1.0.0 (GA Version) (#503)"

* Fixing SpringAiAgenticWorkflowService according to "Upgrade to Spring AI 1.0.0 (GA Version) (#503)".

* Implementation of completion and streamChatCompletion in SpringAiOpenAiService + their corresponding passed tests in SpringAiOpenAiTest class.

Regarding the OpenAiChatModel class it was just formatting, nothing changed.

* Removing a comment

* Removing a comment

* Chat Memory test working.

* Formatting for SpringAiOpenAiService.

* Removing unneccessary imports in SpringAiOpenAiService.

* Updating the toOpenAiRequest in OpenAiChatModel.java.

* Implementing the new approach
--> 4 methods in OpenAiChatCompletionRequest.java return only this now and other constructors other than main are removed for now.

* Editing the approach.

* Fix compilation and format and annotations and javadoc

* Remove unrelated code

* implementation hint

* Formatting

* Updating OpenAiChatOptions.java with our Config Object.

* Formatting

* Passing our Config Object as an input parameter for OpenAiChatOptions()

* Fixing NullPointerException in toOpenAiRequest method for ToolCallng Test to pass.

* Adding topK for the Config Class ??

* Formatting

* Update foundation-models/openai/src/main/java/com/sap/ai/sdk/foundationmodels/openai/spring/OpenAiChatModel.java

Co-authored-by: Alexander Dümont <[email protected]>

* Failing Test of testToolCallingWithoutExecution() in SpringAiOpenAiTest.java

* Resolving Reviewed Issues.

* Resolving Reviewed Issues.

* --> still having testToolCallingWithoutExecution() in SpringAiOpenAiTest.java failing.

--> still fix of null of message.getText() in toAssistantMessage() method in OpenAiChatModel.java pending.

* format

* format

* Removing wild cards imports

* Sucessful build of OpenAi

* Sucessful build of Spring Boot app.

* Formatting

* Removing this test for now.

* Fix nullcheck

* Fix unit test

* chore: Reduce constructor visibility in OpenAI / SpringAI PR (#531)

* Reduce constructors

* Update thresholds

* Update javadoc and factory name

* Replacing config.toolsExecutable with getter-usage + adding tolerate to withStop() method.

* 2

* Assistant message wiht tools calls

* unit test

* Creating OpenAiChatModelTest.java and updating the controller and index page.

* Formatting

* formatting

* Finishing the tests

* Added more options and metadata

* Formatting

* Fixing Format/Style (Minor).

* Fixing Format/Style (Minor).

* Updating the Release notes according to integrating SpringAI with our OpenAi Client.

* Update docs/release_notes.md

Co-authored-by: Charles Dubois <[email protected]>

* Handling JsonProcessingException

* Handling JsonProcessingException

* Handling JsonProcessingException

* Formatting

* protected extractOptions

---------

Co-authored-by: Jonas Israel <[email protected]>
Co-authored-by: Jonas-Isr <[email protected]>
Co-authored-by: I538344 <[email protected]>
Co-authored-by: Alexander Dümont <[email protected]>
Co-authored-by: SAP Cloud SDK Bot <[email protected]>
Co-authored-by: Alexander Dümont <[email protected]>
Co-authored-by: Roshin Rajan Panackal <[email protected]>
Co-authored-by: Charles Dubois <[email protected]>

Author: 6 people

docs/release_notes.md

@@ -12,7 +12,8 @@
 
 ### ✨ New Functionality
 
-- Extend `OpenAiClientException` and `OrchestrationClientException` to  retrieve error diagnostics information received from remote service.
+- Extend `OpenAiClientException` and `OrchestrationClientException` to retrieve error diagnostics information received
+  from remote service.
   New available accessors for troubleshooting: `getErrorResponse()`, `getHttpResponse()` and, `getHttpRequest()`.
   Please note: depending on the error response, these methods may return `null` if the information is not available.
 - [OpenAI] Added new models for `OpenAiModel`: `GPT_5`, `GPT_5_MINI` and `GPT_5_NANO`.
@@ -22,6 +23,8 @@
   `OrchestrationAiModel.GEMINI_1_5_FLASH`
   - Replacement are `GEMINI_2_5_PRO` and `GEMINI_2_5_FLASH`.
 - [Orchestration] Deprecated `OrchestrationAiModel.IBM_GRANITE_13B_CHAT` with no replacement.
+- [OpenAI] [Introduced SpringAI integration with our OpenAI client.](https://sap.github.io/ai-sdk/docs/java/spring-ai/openai)
+  - Added `OpenAiChatModel`
 
 ### 📈 Improvements
 

foundation-models/openai/pom.xml

@@ -38,12 +38,12 @@
   </scm>
   <properties>
     <project.rootdir>${project.basedir}/../../</project.rootdir>
-    <coverage.complexity>72%</coverage.complexity>
-    <coverage.line>80%</coverage.line>
-    <coverage.instruction>76%</coverage.instruction>
-    <coverage.branch>70%</coverage.branch>
-    <coverage.method>83%</coverage.method>
-    <coverage.class>84%</coverage.class>
+    <coverage.complexity>81%</coverage.complexity>
+    <coverage.line>91%</coverage.line>
+    <coverage.instruction>88%</coverage.instruction>
+    <coverage.branch>79%</coverage.branch>
+    <coverage.method>90%</coverage.method>
+    <coverage.class>92%</coverage.class>
   </properties>
   <dependencies>
     <dependency>
@@ -112,6 +112,11 @@
       <artifactId>spring-ai-model</artifactId>
       <optional>true</optional>
     </dependency>
+    <dependency>
+      <groupId>io.projectreactor</groupId>
+      <artifactId>reactor-core</artifactId>
+      <optional>true</optional>
+    </dependency>
     <!-- scope "provided" -->
     <dependency>
       <groupId>org.projectlombok</groupId>
@@ -149,6 +154,12 @@
       <artifactId>javaparser-core</artifactId>
       <scope>test</scope>
     </dependency>
+    <dependency>
+      <groupId>org.springframework.ai</groupId>
+      <artifactId>spring-ai-client-chat</artifactId>
+      <scope>test</scope>
+      <optional>true</optional>
+    </dependency>
   </dependencies>
 
   <profiles>

foundation-models/openai/src/main/java/com/sap/ai/sdk/foundationmodels/openai/OpenAiMessage.java

@@ -1,6 +1,7 @@
 package com.sap.ai.sdk.foundationmodels.openai;
 
 import com.google.common.annotations.Beta;
+import java.util.ArrayList;
 import java.util.List;
 import javax.annotation.Nonnull;
 
@@ -46,6 +47,18 @@ static OpenAiAssistantMessage assistant(@Nonnull final String message) {
     return new OpenAiAssistantMessage(message);
   }
 
+  /**
+   * A convenience method to create an assistant message.
+   *
+   * @param toolCalls tool calls to associate with the message.
+   * @return the assistant message.
+   */
+  @Nonnull
+  static OpenAiAssistantMessage assistant(@Nonnull final List<OpenAiToolCall> toolCalls) {
+    return new OpenAiAssistantMessage(
+        new OpenAiMessageContent(List.of()), new ArrayList<>(toolCalls));
+  }
+
   /**
    * A convenience method to create a system message.
    *

foundation-models/openai/src/main/java/com/sap/ai/sdk/foundationmodels/openai/OpenAiToolCall.java

@@ -1,11 +1,27 @@
 package com.sap.ai.sdk.foundationmodels.openai;
 
 import com.google.common.annotations.Beta;
+import javax.annotation.Nonnull;
 
 /**
  * Represents a tool called by an OpenAI model.
  *
  * @since 1.6.0
  */
 @Beta
-public sealed interface OpenAiToolCall permits OpenAiFunctionCall {}
+public sealed interface OpenAiToolCall permits OpenAiFunctionCall {
+  /**
+   * Creates a new instance of {@link OpenAiToolCall}.
+   *
+   * @param id The unique identifier for the tool call.
+   * @param name The name of the tool to be called.
+   * @param arguments The arguments for the tool call, encoded as a JSON string.
+   * @return A new instance of {@link OpenAiToolCall}.
+   * @since 1.10.0
+   */
+  @Nonnull
+  static OpenAiToolCall function(
+      @Nonnull final String id, @Nonnull final String name, @Nonnull final String arguments) {
+    return new OpenAiFunctionCall(id, name, arguments);
+  }
+}

foundation-models/openai/src/main/java/com/sap/ai/sdk/foundationmodels/openai/spring/OpenAiChatModel.java

@@ -0,0 +1,223 @@
+package com.sap.ai.sdk.foundationmodels.openai.spring;
+
+import static org.springframework.ai.model.tool.ToolCallingChatOptions.isInternalToolExecutionEnabled;
+
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.core.type.TypeReference;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.sap.ai.sdk.foundationmodels.openai.OpenAiChatCompletionDelta;
+import com.sap.ai.sdk.foundationmodels.openai.OpenAiChatCompletionRequest;
+import com.sap.ai.sdk.foundationmodels.openai.OpenAiChatCompletionResponse;
+import com.sap.ai.sdk.foundationmodels.openai.OpenAiClient;
+import com.sap.ai.sdk.foundationmodels.openai.OpenAiMessage;
+import com.sap.ai.sdk.foundationmodels.openai.OpenAiToolCall;
+import com.sap.ai.sdk.foundationmodels.openai.generated.model.ChatCompletionMessageToolCall;
+import com.sap.ai.sdk.foundationmodels.openai.generated.model.ChatCompletionTool;
+import com.sap.ai.sdk.foundationmodels.openai.generated.model.CreateChatCompletionResponseChoicesInner;
+import com.sap.ai.sdk.foundationmodels.openai.generated.model.FunctionObject;
+import io.vavr.control.Option;
+import java.math.BigDecimal;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.function.Function;
+import javax.annotation.Nonnull;
+import lombok.RequiredArgsConstructor;
+import lombok.extern.slf4j.Slf4j;
+import lombok.val;
+import org.springframework.ai.chat.messages.AssistantMessage;
+import org.springframework.ai.chat.messages.AssistantMessage.ToolCall;
+import org.springframework.ai.chat.messages.Message;
+import org.springframework.ai.chat.messages.ToolResponseMessage;
+import org.springframework.ai.chat.metadata.ChatGenerationMetadata;
+import org.springframework.ai.chat.model.ChatModel;
+import org.springframework.ai.chat.model.ChatResponse;
+import org.springframework.ai.chat.model.Generation;
+import org.springframework.ai.chat.prompt.ChatOptions;
+import org.springframework.ai.chat.prompt.Prompt;
+import org.springframework.ai.model.tool.DefaultToolCallingManager;
+import org.springframework.ai.model.tool.ToolCallingChatOptions;
+import reactor.core.publisher.Flux;
+
+/**
+ * OpenAI Chat Model implementation that interacts with the OpenAI API to generate chat completions.
+ */
+@Slf4j
+@RequiredArgsConstructor
+public class OpenAiChatModel implements ChatModel {
+
+  private final OpenAiClient client;
+
+  @Nonnull
+  private final DefaultToolCallingManager toolCallingManager =
+      DefaultToolCallingManager.builder().build();
+
+  @Override
+  @Nonnull
+  public ChatResponse call(@Nonnull final Prompt prompt) {
+    val options = prompt.getOptions();
+    var request = new OpenAiChatCompletionRequest(extractMessages(prompt));
+
+    if (options != null) {
+      request = extractOptions(request, options);
+    }
+    if ((options instanceof ToolCallingChatOptions toolOptions)) {
+      request = request.withTools(extractTools(toolOptions));
+    }
+
+    val result = client.chatCompletion(request);
+    val response = new ChatResponse(toGenerations(result));
+
+    if (options != null && isInternalToolExecutionEnabled(options) && response.hasToolCalls()) {
+      val toolExecutionResult = toolCallingManager.executeToolCalls(prompt, response);
+      // Send the tool execution result back to the model.
+      return call(new Prompt(toolExecutionResult.conversationHistory(), options));
+    }
+    return response;
+  }
+
+  @Override
+  @Nonnull
+  public Flux<ChatResponse> stream(@Nonnull final Prompt prompt) {
+    val options = prompt.getOptions();
+    var request = new OpenAiChatCompletionRequest(extractMessages(prompt));
+
+    if (options != null) {
+      request = extractOptions(request, options);
+    }
+    if ((options instanceof ToolCallingChatOptions toolOptions)) {
+      request = request.withTools(extractTools(toolOptions));
+    }
+
+    val stream = client.streamChatCompletionDeltas(request);
+    final Flux<OpenAiChatCompletionDelta> flux =
+        Flux.generate(
+            stream::iterator,
+            (iterator, sink) -> {
+              if (iterator.hasNext()) {
+                sink.next(iterator.next());
+              } else {
+                sink.complete();
+              }
+              return iterator;
+            });
+    return flux.map(
+        delta -> {
+          val assistantMessage = new AssistantMessage(delta.getDeltaContent(), Map.of());
+          val metadata =
+              ChatGenerationMetadata.builder().finishReason(delta.getFinishReason()).build();
+          return new ChatResponse(List.of(new Generation(assistantMessage, metadata)));
+        });
+  }
+
+  private static List<OpenAiMessage> extractMessages(final Prompt prompt) {
+    final List<OpenAiMessage> result = new ArrayList<>();
+    for (final Message message : prompt.getInstructions()) {
+      switch (message.getMessageType()) {
+        case USER -> Option.of(message.getText()).peek(t -> result.add(OpenAiMessage.user(t)));
+        case SYSTEM -> Option.of(message.getText()).peek(t -> result.add(OpenAiMessage.system(t)));
+        case ASSISTANT -> addAssistantMessage(result, (AssistantMessage) message);
+        case TOOL -> addToolMessages(result, (ToolResponseMessage) message);
+      }
+    }
+    return result;
+  }
+
+  private static void addAssistantMessage(
+      final List<OpenAiMessage> result, final AssistantMessage message) {
+    if (message.getText() != null) {
+      result.add(OpenAiMessage.assistant(message.getText()));
+      return;
+    }
+    final Function<ToolCall, OpenAiToolCall> callTranslate =
+        toolCall -> OpenAiToolCall.function(toolCall.id(), toolCall.name(), toolCall.arguments());
+    val calls = message.getToolCalls().stream().map(callTranslate).toList();
+    result.add(OpenAiMessage.assistant(calls));
+  }
+
+  private static void addToolMessages(
+      final List<OpenAiMessage> result, final ToolResponseMessage message) {
+    for (final ToolResponseMessage.ToolResponse response : message.getResponses()) {
+      result.add(OpenAiMessage.tool(response.responseData(), response.id()));
+    }
+  }
+
+  @Nonnull
+  private static List<Generation> toGenerations(
+      @Nonnull final OpenAiChatCompletionResponse result) {
+    return result.getOriginalResponse().getChoices().stream()
+        .map(OpenAiChatModel::toGeneration)
+        .toList();
+  }
+
+  @Nonnull
+  private static Generation toGeneration(
+      @Nonnull final CreateChatCompletionResponseChoicesInner choice) {
+    val metadata =
+        ChatGenerationMetadata.builder().finishReason(choice.getFinishReason().getValue());
+    metadata.metadata("index", choice.getIndex());
+    if (choice.getLogprobs() != null && !choice.getLogprobs().getContent().isEmpty()) {
+      metadata.metadata("logprobs", choice.getLogprobs().getContent());
+    }
+    val message = choice.getMessage();
+    val calls = new ArrayList<ToolCall>();
+    if (message.getToolCalls() != null) {
+      for (final ChatCompletionMessageToolCall c : message.getToolCalls()) {
+        val fnc = c.getFunction();
+        calls.add(
+            new ToolCall(c.getId(), c.getType().getValue(), fnc.getName(), fnc.getArguments()));
+      }
+    }
+
+    val assistantMessage = new AssistantMessage(message.getContent(), Map.of(), calls);
+    return new Generation(assistantMessage, metadata.build());
+  }
+
+  /**
+   * Adds options to the request.
+   *
+   * @param request the request to modify
+   * @param options the options to extract
+   * @return the modified request with options applied
+   */
+  @Nonnull
+  protected static OpenAiChatCompletionRequest extractOptions(
+      @Nonnull OpenAiChatCompletionRequest request, @Nonnull final ChatOptions options) {
+    request = request.withStop(options.getStopSequences()).withMaxTokens(options.getMaxTokens());
+    if (options.getTemperature() != null) {
+      request = request.withTemperature(BigDecimal.valueOf(options.getTemperature()));
+    }
+    if (options.getTopP() != null) {
+      request = request.withTopP(BigDecimal.valueOf(options.getTopP()));
+    }
+    if (options.getPresencePenalty() != null) {
+      request = request.withPresencePenalty(BigDecimal.valueOf(options.getPresencePenalty()));
+    }
+    if (options.getFrequencyPenalty() != null) {
+      request = request.withFrequencyPenalty(BigDecimal.valueOf(options.getFrequencyPenalty()));
+    }
+    return request;
+  }
+
+  private static List<ChatCompletionTool> extractTools(final ToolCallingChatOptions options) {
+    val tools = new ArrayList<ChatCompletionTool>();
+    for (val toolCallback : options.getToolCallbacks()) {
+      val toolDefinition = toolCallback.getToolDefinition();
+      try {
+        final Map<String, Object> params =
+            new ObjectMapper().readValue(toolDefinition.inputSchema(), new TypeReference<>() {});
+        val toolType = ChatCompletionTool.TypeEnum.FUNCTION;
+        val toolFunction =
+            new FunctionObject()
+                .name(toolDefinition.name())
+                .description(toolDefinition.description())
+                .parameters(params);
+        val tool = new ChatCompletionTool().type(toolType).function(toolFunction);
+        tools.add(tool);
+      } catch (JsonProcessingException e) {
+        log.warn("Failed to add tool to the chat request: {}", e.getMessage());
+      }
+    }
+    return tools;
+  }
+}