Skip to content

Commit 260d0f7

Browse files
rpanackalbot-sdk-jsCharlesDuboisSAP
authored
test: [OpenAI] E2E Tool Call Execution (#386)
* Tool call full e2e unverified * CI * Reuse tool * Remove unnecessary line inserts * Create schema with jackson * First version - Mostly tested - API design complete - Function Call as message content item - Non nullability of `Message.content()` * Formatting * Testing and javadocs * improve test clarity * Remove strict tool invocation config * Fix maven dependency scope issue * Fix maven dependency scope issue + 1 * Refactor. toolCalls get dedicated field in assistant message - OpenAiAssistantMessage().content() may contain empty list * minor fixes - final keyword - remove redundant assertion - variable naming * minor fixes - final keyword - variable naming * Test message list in request externally unmodifiable * Test message list in request externally unmodifiable * minor javadoc update * Remove sample app changes * All e2e test code * @beta annotation * Update foundation-models/openai/src/main/java/com/sap/ai/sdk/foundationmodels/openai/OpenAiChatCompletionResponse.java Co-authored-by: Charles Dubois <[email protected]> * Apply suggestions from code review javadoc and naming Co-authored-by: Charles Dubois <[email protected]> * update getMessage method impl readability * Move test to generated client test class * Extend tests * Reduce test * Refactor test * Formatting * import statement * refactor: remove unused chatCompletionTools method and refactor sample code * pom dependency order and controller method params * pom dependency order and controller method params * update index file * Improve sample code * Move the service class out of test * PMD --------- Co-authored-by: Roshin Rajan Panackal <[email protected]> Co-authored-by: SAP Cloud SDK Bot <[email protected]> Co-authored-by: Charles Dubois <[email protected]>
1 parent f7ebcad commit 260d0f7

File tree

9 files changed

+186
-65
lines changed

9 files changed

+186
-65
lines changed

‎pom.xml‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,7 @@
7272
<mockito.version>5.16.1</mockito.version>
7373
<javaparser.version>3.26.3</javaparser.version>
7474
<jsonschema-generator.version>4.38.0</jsonschema-generator.version>
75+
<jackson.version>2.18.3</jackson.version>
7576
<!-- conflicts resolution -->
7677
<micrometer.version>1.14.2</micrometer.version>
7778
<json.version>20250107</json.version>
@@ -132,7 +133,12 @@
132133
<dependency>
133134
<groupId>com.fasterxml.jackson.module</groupId>
134135
<artifactId>jackson-module-parameter-names</artifactId>
135-
<version>2.18.3</version>
136+
<version>${jackson.version}</version>
137+
</dependency>
138+
<dependency>
139+
<groupId>com.fasterxml.jackson.module</groupId>
140+
<artifactId>jackson-module-jsonSchema</artifactId>
141+
<version>${jackson.version}</version>
136142
</dependency>
137143
<dependency>
138144
<groupId>com.github.victools</groupId>

‎sample-code/spring-app/pom.xml‎

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,10 @@
7070
<groupId>com.sap.cloud.sdk.cloudplatform</groupId>
7171
<artifactId>cloudplatform-core</artifactId>
7272
</dependency>
73+
<dependency>
74+
<groupId>com.sap.cloud.sdk.cloudplatform</groupId>
75+
<artifactId>cloudplatform-connectivity</artifactId>
76+
</dependency>
7377
<dependency>
7478
<groupId>com.sap.cloud.sdk.datamodel</groupId>
7579
<artifactId>openapi-core</artifactId>
@@ -131,6 +135,14 @@
131135
<groupId>com.fasterxml.jackson.core</groupId>
132136
<artifactId>jackson-annotations</artifactId>
133137
</dependency>
138+
<dependency>
139+
<groupId>com.fasterxml.jackson.module</groupId>
140+
<artifactId>jackson-module-jsonSchema</artifactId>
141+
</dependency>
142+
<dependency>
143+
<groupId>com.fasterxml.jackson.core</groupId>
144+
<artifactId>jackson-core</artifactId>
145+
</dependency>
134146
<!-- scope "runtime" -->
135147
<dependency>
136148
<groupId>ch.qos.logback</groupId>
@@ -177,10 +189,6 @@
177189
<artifactId>assertj-core</artifactId>
178190
<scope>test</scope>
179191
</dependency>
180-
<dependency>
181-
<groupId>com.sap.cloud.sdk.cloudplatform</groupId>
182-
<artifactId>cloudplatform-connectivity</artifactId>
183-
</dependency>
184192
</dependencies>
185193

186194
<build>

‎sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/controllers/OpenAiController.java‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -120,11 +120,13 @@ Object chatCompletionImage(
120120
return response.getChoices().get(0).getMessage();
121121
}
122122

123-
@GetMapping("/chatCompletionTool")
123+
@GetMapping("/chatCompletionToolExecution")
124124
@Nonnull
125-
Object chatCompletionTools(
126-
@Nullable @RequestParam(value = "format", required = false) final String format) {
127-
final var response = service.chatCompletionTools(12);
125+
Object chatCompletionToolExecution(
126+
@Nullable @RequestParam(value = "format", required = false) final String format,
127+
@Nonnull @RequestParam(value = "location", defaultValue = "Dubai") final String location,
128+
@Nonnull @RequestParam(value = "unit", defaultValue = "°C") final String unit) {
129+
final var response = service.chatCompletionToolExecution(location, unit);
128130
if ("json".equals(format)) {
129131
return response;
130132
}

‎sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/services/OpenAiService.java‎

Lines changed: 72 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,11 @@
55
import static com.sap.ai.sdk.foundationmodels.openai.OpenAiModel.TEXT_EMBEDDING_3_SMALL;
66
import static com.sap.ai.sdk.foundationmodels.openai.model.OpenAiChatCompletionTool.ToolType.FUNCTION;
77

8+
import com.fasterxml.jackson.core.JsonProcessingException;
9+
import com.fasterxml.jackson.core.type.TypeReference;
10+
import com.fasterxml.jackson.databind.JsonMappingException;
11+
import com.fasterxml.jackson.databind.ObjectMapper;
12+
import com.fasterxml.jackson.module.jsonSchema.JsonSchemaGenerator;
813
import com.sap.ai.sdk.core.AiCoreService;
914
import com.sap.ai.sdk.foundationmodels.openai.OpenAiClient;
1015
import com.sap.ai.sdk.foundationmodels.openai.model.OpenAiChatCompletionDelta;
@@ -13,8 +18,10 @@
1318
import com.sap.ai.sdk.foundationmodels.openai.model.OpenAiChatCompletionParameters;
1419
import com.sap.ai.sdk.foundationmodels.openai.model.OpenAiChatCompletionTool;
1520
import com.sap.ai.sdk.foundationmodels.openai.model.OpenAiChatMessage;
21+
import com.sap.ai.sdk.foundationmodels.openai.model.OpenAiChatToolCall;
1622
import com.sap.ai.sdk.foundationmodels.openai.model.OpenAiEmbeddingOutput;
1723
import com.sap.ai.sdk.foundationmodels.openai.model.OpenAiEmbeddingParameters;
24+
import java.util.ArrayList;
1825
import java.util.List;
1926
import java.util.Map;
2027
import java.util.stream.Stream;
@@ -26,6 +33,7 @@
2633
@Service
2734
@Slf4j
2835
public class OpenAiService {
36+
private static final ObjectMapper JACKSON = new ObjectMapper();
2937

3038
/**
3139
* Chat request to OpenAI
@@ -86,30 +94,79 @@ public OpenAiChatCompletionOutput chatCompletionImage(@Nonnull final String link
8694
}
8795

8896
/**
89-
* Chat request to OpenAI with a tool.
97+
* Executes a chat completion request to OpenAI with a tool that calculates the weather.
9098
*
91-
* @param months The number of months to be inferred in the tool
92-
* @return the assistant message response
99+
* @param location The location to get the weather for.
100+
* @param unit The unit of temperature to use.
101+
* @return The assistant message response.
93102
*/
94103
@Nonnull
95-
public OpenAiChatCompletionOutput chatCompletionTools(final int months) {
96-
final var question =
97-
"A pair of rabbits is placed in a field. Each month, every pair produces one new pair, starting from the second month. How many rabbits will there be after %s months?"
98-
.formatted(months);
99-
final var par = Map.of("type", "object", "properties", Map.of("N", Map.of("type", "integer")));
104+
public OpenAiChatCompletionOutput chatCompletionToolExecution(
105+
@Nonnull final String location, @Nonnull final String unit) {
106+
107+
// 1. Define the function
108+
final Map<String, Object> schemaMap = generateSchema(WeatherMethod.Request.class);
100109
final var function =
101110
new OpenAiChatCompletionFunction()
102-
.setName("fibonacci")
103-
.setDescription("Calculate the Fibonacci number for given sequence index.")
104-
.setParameters(par);
111+
.setName("weather")
112+
.setDescription("Get the weather for the given location")
113+
.setParameters(schemaMap);
105114
final var tool = new OpenAiChatCompletionTool().setType(FUNCTION).setFunction(function);
115+
116+
final var messages = new ArrayList<OpenAiChatMessage>();
117+
messages.add(
118+
new OpenAiChatMessage.OpenAiChatUserMessage()
119+
.addText("What's the weather in %s in %s?".formatted(location, unit)));
120+
121+
// Assistant will call the function
106122
final var request =
107123
new OpenAiChatCompletionParameters()
108-
.addMessages(new OpenAiChatMessage.OpenAiChatUserMessage().addText(question))
109-
.setTools(List.of(tool))
110-
.setToolChoiceFunction("fibonacci");
124+
.addMessages(messages.toArray(OpenAiChatMessage[]::new))
125+
.setTools(List.of(tool));
126+
127+
final OpenAiChatCompletionOutput response =
128+
OpenAiClient.forModel(GPT_4O_MINI).chatCompletion(request);
129+
130+
// 2. Optionally, execute the function.
131+
final OpenAiChatToolCall toolCall =
132+
response.getChoices().get(0).getMessage().getToolCalls().get(0);
133+
final WeatherMethod.Request arguments =
134+
parseJson(toolCall.getFunction().getArguments(), WeatherMethod.Request.class);
135+
final WeatherMethod.Response currentWeather = WeatherMethod.getCurrentWeather(arguments);
136+
137+
final OpenAiChatMessage.OpenAiChatAssistantMessage assistantMessage =
138+
response.getChoices().get(0).getMessage();
139+
messages.add(assistantMessage);
140+
141+
final var toolMessage =
142+
new OpenAiChatMessage.OpenAiChatToolMessage()
143+
.setToolCallId(toolCall.getId())
144+
.setContent(currentWeather.toString());
145+
messages.add(toolMessage);
146+
147+
final var finalRequest =
148+
new OpenAiChatCompletionParameters()
149+
.addMessages(messages.toArray(OpenAiChatMessage[]::new));
150+
151+
return OpenAiClient.forModel(GPT_4O_MINI).chatCompletion(finalRequest);
152+
}
153+
154+
private static <T> T parseJson(@Nonnull final String rawJson, @Nonnull final Class<T> clazz) {
155+
try {
156+
return JACKSON.readValue(rawJson, clazz);
157+
} catch (JsonProcessingException e) {
158+
throw new IllegalArgumentException("Failed to parse tool call arguments: " + rawJson, e);
159+
}
160+
}
111161

112-
return OpenAiClient.forModel(GPT_4O_MINI).chatCompletion(request);
162+
private static Map<String, Object> generateSchema(@Nonnull final Class<?> clazz) {
163+
final var jsonSchemaGenerator = new JsonSchemaGenerator(JACKSON);
164+
try {
165+
final var schema = jsonSchemaGenerator.generateSchema(clazz);
166+
return JACKSON.convertValue(schema, new TypeReference<>() {});
167+
} catch (JsonMappingException e) {
168+
throw new IllegalArgumentException("Could not generate schema for " + clazz.getName(), e);
169+
}
113170
}
114171

115172
/**

sample-code/spring-app/src/test/java/com/sap/ai/sdk/app/services/OpenAiServiceV2.java renamed to sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/services/OpenAiServiceV2.java

Lines changed: 67 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -5,18 +5,26 @@
55
import static com.sap.ai.sdk.foundationmodels.openai.OpenAiModel.TEXT_EMBEDDING_3_SMALL;
66
import static com.sap.ai.sdk.foundationmodels.openai.generated.model.ChatCompletionTool.TypeEnum.FUNCTION;
77

8+
import com.fasterxml.jackson.core.JsonProcessingException;
9+
import com.fasterxml.jackson.core.type.TypeReference;
10+
import com.fasterxml.jackson.databind.JsonMappingException;
11+
import com.fasterxml.jackson.databind.ObjectMapper;
12+
import com.fasterxml.jackson.module.jsonSchema.JsonSchemaGenerator;
813
import com.sap.ai.sdk.core.AiCoreService;
14+
import com.sap.ai.sdk.foundationmodels.openai.OpenAiAssistantMessage;
915
import com.sap.ai.sdk.foundationmodels.openai.OpenAiChatCompletionDelta;
1016
import com.sap.ai.sdk.foundationmodels.openai.OpenAiChatCompletionRequest;
1117
import com.sap.ai.sdk.foundationmodels.openai.OpenAiChatCompletionResponse;
1218
import com.sap.ai.sdk.foundationmodels.openai.OpenAiClient;
1319
import com.sap.ai.sdk.foundationmodels.openai.OpenAiEmbeddingRequest;
1420
import com.sap.ai.sdk.foundationmodels.openai.OpenAiEmbeddingResponse;
21+
import com.sap.ai.sdk.foundationmodels.openai.OpenAiFunctionCall;
1522
import com.sap.ai.sdk.foundationmodels.openai.OpenAiImageItem;
1623
import com.sap.ai.sdk.foundationmodels.openai.OpenAiMessage;
17-
import com.sap.ai.sdk.foundationmodels.openai.OpenAiToolChoice;
24+
import com.sap.ai.sdk.foundationmodels.openai.OpenAiToolCall;
1825
import com.sap.ai.sdk.foundationmodels.openai.generated.model.ChatCompletionTool;
1926
import com.sap.ai.sdk.foundationmodels.openai.generated.model.FunctionObject;
27+
import java.util.ArrayList;
2028
import java.util.List;
2129
import java.util.Map;
2230
import java.util.stream.Stream;
@@ -28,6 +36,7 @@
2836
@Service
2937
@Slf4j
3038
public class OpenAiServiceV2 {
39+
private static final ObjectMapper JACKSON = new ObjectMapper();
3140

3241
/**
3342
* Chat request to OpenAI
@@ -84,30 +93,71 @@ public OpenAiChatCompletionResponse chatCompletionImage(@Nonnull final String li
8493
}
8594

8695
/**
87-
* Chat request to OpenAI with a tool.
96+
* Executes a chat completion request to OpenAI with a tool that calculates the weather.
8897
*
89-
* @param months The number of months to be inferred in the tool
90-
* @return the assistant message response
98+
* @param location The location to get the weather for.
99+
* @param unit The unit of temperature to use.
100+
* @return The assistant message response.
91101
*/
92102
@Nonnull
93-
public OpenAiChatCompletionResponse chatCompletionTools(final int months) {
103+
public OpenAiChatCompletionResponse chatCompletionToolExecution(
104+
@Nonnull final String location, @Nonnull final String unit) {
105+
106+
// 1. Define the function
107+
final Map<String, Object> schemaMap = generateSchema(WeatherMethod.Request.class);
94108
final var function =
95109
new FunctionObject()
96-
.name("fibonacci")
97-
.description("Calculate the Fibonacci number for given sequence index.")
98-
.parameters(
99-
Map.of("type", "object", "properties", Map.of("N", Map.of("type", "integer"))));
100-
110+
.name("weather")
111+
.description("Get the weather for the given location")
112+
.parameters(schemaMap);
101113
final var tool = new ChatCompletionTool().type(FUNCTION).function(function);
102114

103-
final var request =
104-
new OpenAiChatCompletionRequest(
105-
"A pair of rabbits is placed in a field. Each month, every pair produces one new pair, starting from the second month. How many rabbits will there be after %s months?"
106-
.formatted(months))
107-
.withTools(List.of(tool))
108-
.withToolChoice(OpenAiToolChoice.function("fibonacci"));
115+
final var messages = new ArrayList<OpenAiMessage>();
116+
messages.add(OpenAiMessage.user("What's the weather in %s in %s?".formatted(location, unit)));
117+
118+
// Assistant will call the function
119+
final var request = new OpenAiChatCompletionRequest(messages).withTools(List.of(tool));
120+
final OpenAiChatCompletionResponse response =
121+
OpenAiClient.forModel(GPT_4O_MINI).chatCompletion(request);
122+
123+
// 2. Optionally, execute the function.
124+
final OpenAiAssistantMessage assistantMessage = response.getMessage();
125+
messages.add(assistantMessage);
126+
127+
final OpenAiToolCall toolCall = assistantMessage.toolCalls().get(0);
128+
if (!(toolCall instanceof OpenAiFunctionCall functionCall)) {
129+
throw new IllegalArgumentException(
130+
"Expected a function call, but got: %s".formatted(assistantMessage));
131+
}
132+
133+
final WeatherMethod.Request arguments =
134+
parseJson(functionCall.getArguments(), WeatherMethod.Request.class);
135+
final WeatherMethod.Response weatherMethod = WeatherMethod.getCurrentWeather(arguments);
109136

110-
return OpenAiClient.forModel(GPT_4O_MINI).chatCompletion(request);
137+
messages.add(OpenAiMessage.tool(weatherMethod.toString(), functionCall.getId()));
138+
139+
// Send back the results, and the model will incorporate them into its final response.
140+
return OpenAiClient.forModel(GPT_4O_MINI).chatCompletion(request.withMessages(messages));
141+
}
142+
143+
@Nonnull
144+
private static <T> T parseJson(@Nonnull final String rawJson, @Nonnull final Class<T> clazz) {
145+
try {
146+
return JACKSON.readValue(rawJson, clazz);
147+
} catch (JsonProcessingException e) {
148+
throw new IllegalArgumentException("Failed to parse tool call arguments: " + rawJson, e);
149+
}
150+
}
151+
152+
@Nonnull
153+
private static Map<String, Object> generateSchema(@Nonnull final Class<?> clazz) {
154+
final var jsonSchemaGenerator = new JsonSchemaGenerator(JACKSON);
155+
try {
156+
final var schema = jsonSchemaGenerator.generateSchema(clazz);
157+
return JACKSON.convertValue(schema, new TypeReference<>() {});
158+
} catch (JsonMappingException e) {
159+
throw new IllegalArgumentException("Could not generate schema for " + clazz.getName(), e);
160+
}
111161
}
112162

113163
/**

‎sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/services/WeatherMethod.java‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ record Response(double temp, Unit unit) {}
3535
@Nonnull
3636
@SuppressWarnings("unused")
3737
@Tool(description = "Get the weather in location")
38-
Response getCurrentWeather(@ToolParam @Nonnull final Request request) {
38+
static Response getCurrentWeather(@ToolParam @Nonnull final Request request) {
3939
final int temperature = request.location.hashCode() % 30;
4040
return new Response(temperature, request.unit);
4141
}

‎sample-code/spring-app/src/main/resources/static/index.html‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -569,12 +569,12 @@ <h5 class="mb-1">OpenAI</h5>
569569
</li>
570570
<li class="list-group-item">
571571
<div class="info-tooltip">
572-
<button type="submit" formaction="/chatCompletionTool"
572+
<button type="submit" formaction="/chatCompletionToolExecution"
573573
class="link-offset-2-hover link-underline link-underline-opacity-0 link-underline-opacity-75-hover endpoint">
574-
<code>/chatCompletionTool</code>
574+
<code>/chatCompletionToolExecution</code>
575575
</button>
576576
<div class="tooltip-content">
577-
Chat request to OpenAI with a tool.
577+
Chat request to OpenAI with an executed tool call.
578578
</div>
579579
</div>
580580
</li>

‎sample-code/spring-app/src/test/java/com/sap/ai/sdk/app/controllers/OpenAiTest.java‎

Lines changed: 9 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -73,16 +73,6 @@ void streamChatCompletion() {
7373
assertThat(totalOutput.getChoices().get(0).getContentFilterResults()).isNotNull();
7474
}
7575

76-
@Test
77-
void chatCompletionTools() {
78-
final var completion = service.chatCompletionTools(12);
79-
80-
final var message = completion.getChoices().get(0).getMessage();
81-
assertThat(message.getRole()).isEqualTo("assistant");
82-
assertThat(message.getToolCalls()).isNotNull();
83-
assertThat(message.getToolCalls().get(0).getFunction().getName()).isEqualTo("fibonacci");
84-
}
85-
8676
@Test
8777
void embedding() {
8878
final var embedding = service.embedding("Hello world");
@@ -101,4 +91,13 @@ void chatCompletionWithResource() {
10191
assertThat(message.getRole()).isEqualTo("assistant");
10292
assertThat(message.getContent()).isNotEmpty();
10393
}
94+
95+
@Test
96+
void chatCompletionToolExecution() {
97+
final var completion = service.chatCompletionToolExecution("Dubai", "°C");
98+
99+
String content = completion.getContent();
100+
101+
assertThat(content).contains("°C");
102+
}
104103
}

0 commit comments

Comments
 (0)