feat: [OpenAI] PoC - Responses API support with OpenAI SDK Adapter by rpanackal · Pull Request #794 · SAP/ai-sdk-java

rpanackal · 2026-03-25T09:40:00Z

Context

This PoC provides an adapter for Official OpenAI SDK integrations with our SDK by implementing com.openai.core.http.HttpClient. You can find out more about the OpenAI recommended approach here in their docs.

Feature scope:

Introduces a new generic AiCoreOpenAiClient that can work as an adapter for any OpenAI endpoints
- Easy adoption of any openai endpoints (eg: /realtime) that are/will be supported by AiCore
Introduce AiCoreHttpClientImpl
- Implements a new request and response handling pathway (incl. streaming) that is compatible with OpenAI SDK.
- Returns a client with familiar OpenAI SDK API usage for the user.
Constrains allowed endpoint to /response (for now)

Usage

import com.openai.client.OpenAIClient
import com.sap.ai.sdk.foundationmodels.openai.v1.AiCoreOpenAiClient

OpenAiClient client = AiCoreOpenAiClient.forModel(GPT_5)
var request = ResponseCreateParams.builder().input(input).model(ChatModel.GPT_5).build()
Response response = client.responses().create(request)

Pros

Usage via official OpenAI SDK's API; minimal learning curve
Minimal maintenance overhead as we dont need to maintain models as well as client api.
Customisability and ease of integration: By relaxing allowed method, endpoint etc, we can easy support additional openai endpoints (eg /chat/completion, /realtime etc)

Cons

More complex implementation that introduces additional request/response handling pathway in addition to the existing ones
Two different classes for AI Model list which can be confusing
- com.sap.ai.sdk.foundationmodels.openai.OpenAiModel used to select available models in AICore and com.openai.models.ChatModel.ChatModel for request payload configuration in OpenAI SDK
- We can't pick one without loss of semantics

Definition of Done

Functionality scope stated & covered
Tests cover the scope above
Error handling created / updated & covered by the tests above
~~Aligned changes with the JavaScript SDK~~
Documentation updated
Release notes updated

- Streaming no fully enabled

CharlesDuboisSAP

We should:

enable chat completions
enable streaming
have a controller and an updated index.html
deprecate the old client
have a migration guide or link OpenAI's migration guide.

foundation-models/openai-v1/pom.xml

CharlesDuboisSAP · 2026-03-25T12:43:39Z

sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/services/AiCoreOpenAiService.java

+  @Nonnull
+  public Response createResponse(@Nonnull final String input) {
+    val params =
+        ResponseCreateParams.builder().input(input).model(ChatModel.GPT_5).store(false).build();


store false doesn't seem to make a difference

The store tells the server whether to persist the response produced, so it can be referenced in any future requests. So, yes the test will be green even without it. But to avoid creating persisted resources on each E2E we will run, I chose to set store=false.

Would like a comment because there is no Javadoc on OpenAI's client

sample-code/spring-app/src/test/java/com/sap/ai/sdk/app/controllers/OpenAiV1Test.java

foundation-models/openai-v1/pom.xml

sample-code/spring-app/src/test/java/com/sap/ai/sdk/app/controllers/OpenAiV1Test.java

CharlesDuboisSAP · 2026-03-25T12:57:44Z

sample-code/spring-app/src/test/java/com/sap/ai/sdk/app/controllers/OpenAiV1Test.java

+    final var response = service.createResponse("What is the capital of France?");
+    assertThat(response).isNotNull();
+    assertThat(response.output()).isNotNull();
+    assertThat(response.output()).isNotEmpty();


response.output().get(1).message().get().content().get(0).asOutputText().text() is not very convenient. Do we care?

I am sure its not the best, but this is the compromise we are making to reduce burden of maintenance.

...n-models/openai/src/main/java/com/sap/ai/sdk/foundationmodels/openai/AiCoreOpenAiClient.java

newtork · 2026-03-26T13:54:03Z

foundation-models/openai/pom.xml

    </dependency>
+    <dependency>
+      <groupId>com.openai</groupId>
+      <artifactId>openai-java-core</artifactId>


(Major)

I would argue either new module, or set this dependency as optional

Why? we are going to deprecate the 2024 generated API

rpanackal

I want to highlight current api limitation

import static com.sap.ai.sdk.foundationmodels.openai.OpenAiModel;
import com.openai.models.ChatModel;

// Get the client for a deployment by model name and version
OpenAiModel ourAiModel = OpenAiModel.GPT_5
OpenAiClient client = AiCoreOpenAiClient.forModel(ourAiModel) // 

// Supply model again for request payload. Throws without model.
ChatModel openAiModel = ChatModel.GPT_5
var request = ResponseCreateParams.builder().input(input).model(openAiModel).build()

Two sources of truth.

In the current api behaviour, the model in selected deployment takes precedence over the one in payload. But, this behaviour is not apparent to the user.

CharlesDuboisSAP · 2026-03-27T10:16:30Z

...n-models/openai/src/main/java/com/sap/ai/sdk/foundationmodels/openai/AiCoreOpenAiClient.java

+   * @throws DeploymentResolutionException If no running deployment is found for the model.
+   */
+  @Nonnull
+  public static OpenAIClient forModel(@Nonnull final AiModel model) {


You could modify the client to not be instantiated but instead be created at the request level and cached.

No strong preference but it is better API

yes. This is one of the ways along with few other, each with important caveats

Sniffing request: As Charles mentioned we can parse the body in HttpRequest back to JsonNode or request type to infer the model to fetch deployment for - at request time.

As you can imagine, means this deserializing already serialized response.

You will only find model in create() calls. But not in retrieve(), delete() or any other operation. Then how should we fetch a deployment ? just any deployment under foundation-model scenario?

At request time, we can't reliably infer version out of values like "gpt-5-nano", "gpt-5.2", "o3-2025-04-16". AiCore expects distinct fields for model name and model version to match with a deployment. We will have to rework our deployment resolution logic.

Wrapper API: We draft our own wrapper instead of directly returning an object of com.openai.client.OpenAIClient

// Our wrapper client AiCoreBoundOpenAiClient client = AiCoreOpenAiClient.forModel(OpenAiModel.GPT_41); ResponseCreateParams params = ResponseCreateParams.builder() .input("Hello") // .model(...) is optional. We inject or validate for match .build(); Response response = client.responses().create(params);

public interface AiCoreBoundOpenAiClient { AiCoreResponsesService responses(); AiCoreChatCompletionsService chatCompletions(); OpenAIClient raw(); // escape hatch }

Basically, inject model into params, or validate existing model for match with the one in deployment within in our wrapper api.

Maintenance burden is much higher, but we will be able to active choose UX.

newtork · 2026-03-27T11:40:51Z

...n-models/openai/src/main/java/com/sap/ai/sdk/foundationmodels/openai/AiCoreOpenAiClient.java

+    final ClientOptions clientOptions =
+        ClientOptions.builder().baseUrl(baseUrl).httpClient(httpClient).apiKey("unused").build();


I found a way to propagate the model information to the request body.
But it's super ugly :( and you would need to find a way to pass on model information.

(View code suggestion)

Suggested change

final ClientOptions clientOptions =

ClientOptions.builder().baseUrl(baseUrl).httpClient(httpClient).apiKey("unused").build();

final var m = new SimpleModule() {{

setSerializerModifier(new BeanSerializerModifier() {

@Override

@SuppressWarnings("unchecked")

public JsonSerializer<?> modifySerializer(SerializationConfig config, BeanDescription desc, JsonSerializer<?> serializer) {

if (!ResponseCreateParams.Body.class.isAssignableFrom(desc.getBeanClass()))

return serializer;

final var typed = (JsonSerializer<ResponseCreateParams.Body>) serializer;

return new StdSerializer<>(ResponseCreateParams.Body.class) {

@Override

public void serialize(ResponseCreateParams.Body value, JsonGenerator gen, SerializerProvider provider)

throws IOException {

final var buf = new TokenBuffer(gen.getCodec(), false);

typed.serialize(value, buf, provider);

final ObjectNode node = gen.getCodec().readTree(buf.asParser());

if (!node.has("model")) node.put("model", "gpt-5");

gen.writeTree(node);

}

};

}

});

}};

final ClientOptions clientOptions =

ClientOptions.builder().baseUrl(baseUrl).httpClient(httpClient).apiKey("unused")

.jsonMapper((JsonMapper) jsonMapper().registerModule(m))

.build();

I will try this out and get back to you.

I tried to make it work with mixin, without success.

rpanackal added 22 commits March 24, 2026 17:42

Minimal new module setup including spec

27b7e3f

Generation partial-success

a2c9456

Remove examples

aed34ae

Successfully filter by path

9e7c90d

Attach spec filter command

a9a299a

Initial setup

eaf74ba

Successful PoC with OpenAI Models

372769e

Version 1

3726089

Stable api

01c7f61

Change class name

c921a86

Add tests

2491d07

fix dependency analyse issues

c004481

Initial draft untested

fbff9b0

Second draft

68524b2

- Streaming no fully enabled

Successful E2E

37ea275

Streaming initial draft

3bda22a

Streaming E2E with chat completion

229db60

isStreaming check simplified

7ffe122

Cleanup PoC and rename module

5553e1b

Reduce Javadoc verbosity

2a55080

Restrict to /responses api

4ca8af1

Cleanup comments

7061cbc

rpanackal changed the title ~~feat: [OpenAI]/poc OpenAI responses apache~~ feat: [OpenAI] PoC: Responses API support with OpenAI SDK Adapter Mar 25, 2026

rpanackal changed the title ~~feat: [OpenAI] PoC: Responses API support with OpenAI SDK Adapter~~ feat: [OpenAI] PoC - Responses API support with OpenAI SDK Adapter Mar 25, 2026

CharlesDuboisSAP reviewed Mar 25, 2026

View reviewed changes

Charles review suggestions

6eb57fc

CharlesDuboisSAP reviewed Mar 26, 2026

View reviewed changes

rpanackal added 2 commits March 26, 2026 13:23

Charles review - round 2 suggestions

38ced86

Add dependency

d8e1902

newtork reviewed Mar 26, 2026

View reviewed changes

...n-models/openai/src/main/java/com/sap/ai/sdk/foundationmodels/openai/AiCoreOpenAiClient.java Show resolved Hide resolved

newtork reviewed Mar 26, 2026

View reviewed changes

Mark openai dependency optional and new client @Beta

dae1906

rpanackal commented Mar 27, 2026

View reviewed changes

CharlesDuboisSAP reviewed Mar 27, 2026

View reviewed changes

newtork reviewed Mar 27, 2026

View reviewed changes

		final ClientOptions clientOptions =
		ClientOptions.builder().baseUrl(baseUrl).httpClient(httpClient).apiKey("unused").build();

-    final ClientOptions clientOptions =
-        ClientOptions.builder().baseUrl(baseUrl).httpClient(httpClient).apiKey("unused").build();
+    final var m = new SimpleModule() {{
+        setSerializerModifier(new BeanSerializerModifier() {
+          @Override
+          @SuppressWarnings("unchecked")
+          public JsonSerializer<?> modifySerializer(SerializationConfig config, BeanDescription desc, JsonSerializer<?> serializer) {
+            if (!ResponseCreateParams.Body.class.isAssignableFrom(desc.getBeanClass()))
+              return serializer;
+            final var typed = (JsonSerializer<ResponseCreateParams.Body>) serializer;
+            return new StdSerializer<>(ResponseCreateParams.Body.class) {
+              @Override
+              public void serialize(ResponseCreateParams.Body value, JsonGenerator gen, SerializerProvider provider)
+                  throws IOException {
+                final var buf = new TokenBuffer(gen.getCodec(), false);
+                typed.serialize(value, buf, provider);
+                final ObjectNode node = gen.getCodec().readTree(buf.asParser());
+                if (!node.has("model")) node.put("model", "gpt-5");
+                gen.writeTree(node);
+              }
+            };
+          }
+        });
+      }};
+    final ClientOptions clientOptions =
+        ClientOptions.builder().baseUrl(baseUrl).httpClient(httpClient).apiKey("unused")
+            .jsonMapper((JsonMapper) jsonMapper().registerModule(m))
+            .build();

Conversation

rpanackal commented Mar 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Context

Feature scope:

Usage

Pros

Cons

Definition of Done

Uh oh!

CharlesDuboisSAP left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rpanackal Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rpanackal left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rpanackal Mar 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

rpanackal commented Mar 25, 2026 •

edited

Loading

CharlesDuboisSAP left a comment •

edited

Loading

rpanackal Mar 26, 2026 •

edited

Loading

rpanackal left a comment •

edited

Loading

rpanackal Mar 27, 2026 •

edited

Loading