diff --git a/README.md b/README.md index 3e1eca70e..cea475a80 100644 --- a/README.md +++ b/README.md @@ -131,10 +131,11 @@ Please refer to [this documentation](docs/guides/ORCHESTRATION_CHAT_COMPLETION.m For more detailed information and advanced usage, please refer to the following: -- [Connecting to AI Core](docs/guides/CONNECTING_TO_AICORE.md) -- [Orchestration Chat Completion](docs/guides/ORCHESTRATION_CHAT_COMPLETION.md) -- [OpenAI Chat Completion](docs/guides/OPENAI_CHAT_COMPLETION.md) -- [AI Core Deployment](docs/guides/AI_CORE_DEPLOYMENT.md) +- [ Connecting to AI Core](docs/guides/CONNECTING_TO_AICORE.md) +- [ Orchestration Chat Completion](docs/guides/ORCHESTRATION_CHAT_COMPLETION.md) +- [ OpenAI Chat Completion](docs/guides/OPENAI_CHAT_COMPLETION.md) +- [ Spring AI Integration](docs/guides/SPRING_AI_INTEGRATION.md) +- [🧰 AI Core Deployment](docs/guides/AI_CORE_DEPLOYMENT.md) For updating versions, please refer to the [**Release Notes**](docs/release-notes/release-notes-0-to-14.md). diff --git a/docs/guides/AI_CORE_DEPLOYMENT.md b/docs/guides/AI_CORE_DEPLOYMENT.md index 3a6d8fe64..031c40f1c 100644 --- a/docs/guides/AI_CORE_DEPLOYMENT.md +++ b/docs/guides/AI_CORE_DEPLOYMENT.md @@ -66,7 +66,7 @@ In addition to the prerequisites above, we assume you have already set up the fo ``` -### Create a Deployment +## Create a Deployment Use the following code snippet to create a deployment in SAP AI Core: @@ -84,7 +84,7 @@ AiExecutionStatus status = deployment.getStatus(); Refer to the [DeploymentController.java](../../sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/controllers/DeploymentController.java) in our Spring Boot application for a complete example. -### Delete a Deployment +## Delete a Deployment ```java AiDeploymentCreationResponse deployment; // provided diff --git a/docs/guides/OPENAI_CHAT_COMPLETION.md b/docs/guides/OPENAI_CHAT_COMPLETION.md index 1a0cf2ca8..79f6c310e 100644 --- a/docs/guides/OPENAI_CHAT_COMPLETION.md +++ b/docs/guides/OPENAI_CHAT_COMPLETION.md @@ -87,7 +87,7 @@ In addition to the prerequisites above, we assume you have already set up the fo -### Simple chat completion +## Simple chat completion ```java var result = @@ -98,7 +98,7 @@ var result = String resultMessage = result.getContent(); ``` -### Using a Custom Resource Group +## Using a Custom Resource Group ```java var destination = new AiCoreService() @@ -107,7 +107,7 @@ var destination = new AiCoreService() OpenAiClient.withCustomDestination(destination); ``` -### Message history +## Message history ```java var systemMessage = @@ -124,7 +124,7 @@ String resultMessage = result.getContent(); See [an example in our Spring Boot application](../../sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/controllers/OpenAiController.java) -### Chat Completion with Specific Model Version +## Chat Completion with Specific Model Version By default, when no version is specified, the system selects one of the available deployments of the specified model, regardless of its version. To target a specific version, you can specify the model version along with the model. @@ -134,7 +134,7 @@ OpenAiChatCompletionOutput result = OpenAiClient.forModel(GPT_35_TURBO.withVersion("1106")).chatCompletion(request); ``` -### Chat completion with Custom Model +## Chat completion with Custom Model You can also use a custom OpenAI model for chat completion by creating an `OpenAiModel` object. @@ -145,11 +145,11 @@ OpenAiChatCompletionOutput result = Ensure that the custom model is deployed in SAP AI Core. -### Stream chat completion +## Stream chat completion It's possible to pass a stream of chat completion delta elements, e.g. from the application backend to the frontend in real-time. -#### Asynchronous Streaming +### Asynchronous Streaming This is a blocking example for streaming and printing directly to the console: @@ -168,7 +168,7 @@ try (Stream stream = client.streamChatCompletion(msg)) { } ``` -#### Aggregating Total Output +### Aggregating Total Output The following example is non-blocking and demonstrates how to aggregate the complete response. Any asynchronous library can be used, such as the classic Thread API. @@ -205,12 +205,10 @@ Integer tokensUsed = totalOutput.getUsage().getCompletionTokens(); System.out.println("Tokens used: " + tokensUsed); ``` -#### Spring Boot example - Please find [an example in our Spring Boot application](../../sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/controllers/OpenAiController.java). It shows the usage of Spring Boot's `ResponseBodyEmitter` to stream the chat completion delta messages to the frontend in real-time. -### Embedding +## Embedding Get the embeddings of a text input in list of float values: diff --git a/docs/guides/ORCHESTRATION_CHAT_COMPLETION.md b/docs/guides/ORCHESTRATION_CHAT_COMPLETION.md index 7f0d3be9f..635de0512 100644 --- a/docs/guides/ORCHESTRATION_CHAT_COMPLETION.md +++ b/docs/guides/ORCHESTRATION_CHAT_COMPLETION.md @@ -85,7 +85,7 @@ var config = new OrchestrationModuleConfig() Please also refer to [our sample code](../../sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/controllers/OrchestrationController.java) for this and all following code examples. -### Chat Completion +## Chat Completion Use the Orchestration service to generate a response to a user message: @@ -100,7 +100,7 @@ String messageResult = result.getContent(); In this example, the Orchestration service generates a response to the user message "Hello world! Why is this phrase so famous?". The LLM response is available as the first choice under the `result.getOrchestrationResult()` object. -### Chat completion with Templates +## Chat completion with Templates Use a prepared template and execute requests with by passing only the input parameters: @@ -117,7 +117,7 @@ var result = client.chatCompletion(prompt, configWithTemplate); In this case the template is defined with the placeholder `{{?language}}` which is replaced by the value `German` in the input parameters. -### Message history +## Message history Include a message history to maintain context in the conversation: @@ -134,7 +134,7 @@ var prompt = new OrchestrationPrompt(message).messageHistory(messagesHistory); var result = new OrchestrationClient().chatCompletion(prompt, config); ``` -### Chat completion filter +## Chat completion filter Apply content filtering to the chat completion: @@ -165,7 +165,7 @@ var configWithFilter = config.withInputFiltering(filterStrict).withOutputFilteri var result = new OrchestrationClient().chatCompletion(prompt, configWithFilter); ``` -#### Behavior of Input and Output Filters +### Behavior of Input and Output Filters - **Input Filter**: If the input message violates the filter policy, a `400 (Bad Request)` response will be received during the `chatCompletion` call. @@ -178,7 +178,7 @@ var result = You will find [some examples](../../sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/controllers/OrchestrationController.java) in our Spring Boot application demonstrating response handling with filters. -### Data masking +## Data masking Use the data masking module to anonymize personal information in the input: @@ -201,7 +201,7 @@ var result = In this example, the input will be masked before the call to the LLM and will remain masked in the output. -### Grounding +## Grounding Use the grounding module to provide additional context to the AI model. @@ -232,11 +232,11 @@ Use the grounding module to provide additional context to the AI model. In this example, the AI model is provided with additional context in the form of grounding information. Note, that it is necessary to provide the grounding input via one or more input variables. -### Stream chat completion +## Stream chat completion It's possible to pass a stream of chat completion delta elements, e.g. from the application backend to the frontend in real-time. -#### Asynchronous Streaming +### Asynchronous Streaming This is a blocking example for streaming and printing directly to the console: @@ -253,12 +253,10 @@ try (Stream stream = client.streamChatCompletion(prompt, config)) { } ``` -#### Spring Boot example - Please find [an example in our Spring Boot application](../../sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/controllers/OrchestrationController.java). It shows the usage of Spring Boot's `ResponseBodyEmitter` to stream the chat completion delta messages to the frontend in real-time. -### Set model parameters +## Set model parameters Change your LLM configuration to add model parameters: @@ -272,7 +270,7 @@ OrchestrationAiModel customGPT4O = .withVersion("2024-05-13"); ``` -### Using a Configuration from AI Launchpad +## Using a Configuration from AI Launchpad In case you have created a configuration in AI Launchpad, you can copy or download the configuration as JSON and use it directly in your code: diff --git a/docs/guides/SPRING_AI_INTEGRATION.md b/docs/guides/SPRING_AI_INTEGRATION.md new file mode 100644 index 000000000..963bcc2bc --- /dev/null +++ b/docs/guides/SPRING_AI_INTEGRATION.md @@ -0,0 +1,52 @@ +# Spring AI Integration + +## Table of Contents + +- [Introduction](#introduction) +- [Orchestration Chat Completion](#orchestration-chat-completion) +- [Orchestration Masking](#orchestration-masking) + +## Introduction + +This guide provides examples of how to use our Spring AI integration with our clients in SAP AI Core +for chat completion tasks using the SAP AI SDK for Java. + +## Orchestration Chat Completion + +The Orchestration client is integrated in Spring AI classes: + +```java +ChatModel client = new OrchestrationChatModel(); +OrchestrationModuleConfig config = new OrchestrationModuleConfig().withLlmConfig(GPT_35_TURBO); +OrchestrationChatOptions opts = new OrchestrationChatOptions(config); + +Prompt prompt = new Prompt("What is the capital of France?", opts); +ChatResponse response = client.call(prompt); +``` + +Please +find [an example in our Spring Boot application](../../sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/controllers/SpringAiOrchestrationController.java). + +## Orchestration Masking + +Configure Orchestration modules withing Spring AI: + +```java +ChatModel client = new OrchestrationChatModel(); +OrchestrationModuleConfig config = new OrchestrationModuleConfig().withLlmConfig(GPT_35_TURBO); + +val masking = + DpiMasking.anonymization() + .withEntities(DPIEntities.EMAIL, DPIEntities.ADDRESS, DPIEntities.LOCATION); + +val opts = new OrchestrationChatOptions(config.withMaskingConfig(masking)); +val prompt = + new Prompt( + "Please write 'Hello World!' to me via email. My email address is foo.bar@baz.ai", + opts); + +ChatResponse response = client.call(prompt); +``` + +Please +find [an example in our Spring Boot application](../../sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/controllers/SpringAiOrchestrationController.java). diff --git a/docs/release-notes/release_notes.md b/docs/release-notes/release_notes.md index 910723bfa..7e7985c77 100644 --- a/docs/release-notes/release_notes.md +++ b/docs/release-notes/release_notes.md @@ -17,6 +17,7 @@ - Orchestration supports images as input in newly introduced `MultiChatMessage`. - `MultiChatMessage` also allows for multiple content items (text or image) in one object. - Grounding input can be masked with `DPIConfig`. +- [Integrate the Orchestration client in Spring AI.](../guides/ORCHESTRATION_CHAT_COMPLETION.md#spring-ai-integration) ### 📈 Improvements diff --git a/orchestration/pom.xml b/orchestration/pom.xml index 8d2a99c3d..f9916c9f6 100644 --- a/orchestration/pom.xml +++ b/orchestration/pom.xml @@ -31,11 +31,11 @@ ${project.basedir}/../ - 70% - 87% - 88% - 65% - 65% + 76% + 90% + 91% + 69% + 70% 100% @@ -54,7 +54,11 @@ com.sap.cloud.sdk.cloudplatform connectivity-apache-httpclient5 - + + org.springframework.ai + spring-ai-core + true + org.apache.httpcomponents.core5 httpcore5 diff --git a/orchestration/src/main/java/com/sap/ai/sdk/orchestration/OrchestrationClient.java b/orchestration/src/main/java/com/sap/ai/sdk/orchestration/OrchestrationClient.java index 3c6e79ca3..0f1a52e53 100644 --- a/orchestration/src/main/java/com/sap/ai/sdk/orchestration/OrchestrationClient.java +++ b/orchestration/src/main/java/com/sap/ai/sdk/orchestration/OrchestrationClient.java @@ -1,11 +1,10 @@ package com.sap.ai.sdk.orchestration; -import static com.sap.ai.sdk.core.JacksonConfiguration.getDefaultObjectMapper; +import static com.sap.ai.sdk.orchestration.OrchestrationJacksonConfiguration.getOrchestrationObjectMapper; import com.fasterxml.jackson.core.JsonProcessingException; import com.fasterxml.jackson.databind.JsonNode; import com.fasterxml.jackson.databind.ObjectMapper; -import com.fasterxml.jackson.databind.module.SimpleModule; import com.fasterxml.jackson.databind.node.ObjectNode; import com.google.common.annotations.Beta; import com.sap.ai.sdk.core.AiCoreService; @@ -13,12 +12,9 @@ import com.sap.ai.sdk.core.common.ClientResponseHandler; import com.sap.ai.sdk.core.common.ClientStreamingHandler; import com.sap.ai.sdk.core.common.StreamedDelta; -import com.sap.ai.sdk.orchestration.model.ChatMessagesInner; import com.sap.ai.sdk.orchestration.model.CompletionPostRequest; import com.sap.ai.sdk.orchestration.model.CompletionPostResponse; -import com.sap.ai.sdk.orchestration.model.LLMModuleResult; import com.sap.ai.sdk.orchestration.model.ModuleConfigs; -import com.sap.ai.sdk.orchestration.model.ModuleResultsOutputUnmaskingInner; import com.sap.ai.sdk.orchestration.model.OrchestrationConfig; import com.sap.cloud.sdk.cloudplatform.connectivity.ApacheHttpClient5Accessor; import com.sap.cloud.sdk.cloudplatform.connectivity.HttpDestination; @@ -41,25 +37,7 @@ public class OrchestrationClient { private static final String DEFAULT_SCENARIO = "orchestration"; - static final ObjectMapper JACKSON; - - static { - JACKSON = getDefaultObjectMapper(); - - // Add mix-ins - JACKSON.addMixIn(LLMModuleResult.class, JacksonMixins.LLMModuleResultMixIn.class); - JACKSON.addMixIn( - ModuleResultsOutputUnmaskingInner.class, - JacksonMixins.ModuleResultsOutputUnmaskingInnerMixIn.class); - - final var module = - new SimpleModule() - .addDeserializer( - ChatMessagesInner.class, - PolymorphicFallbackDeserializer.fromJsonSubTypes(ChatMessagesInner.class)) - .setMixInAnnotation(ChatMessagesInner.class, JacksonMixins.NoneTypeInfoMixin.class); - JACKSON.registerModule(module); - } + static final ObjectMapper JACKSON = getOrchestrationObjectMapper(); @Nonnull private final Supplier destinationSupplier; diff --git a/orchestration/src/main/java/com/sap/ai/sdk/orchestration/OrchestrationJacksonConfiguration.java b/orchestration/src/main/java/com/sap/ai/sdk/orchestration/OrchestrationJacksonConfiguration.java new file mode 100644 index 000000000..26fcb2e19 --- /dev/null +++ b/orchestration/src/main/java/com/sap/ai/sdk/orchestration/OrchestrationJacksonConfiguration.java @@ -0,0 +1,54 @@ +package com.sap.ai.sdk.orchestration; + +import static com.sap.ai.sdk.core.JacksonConfiguration.getDefaultObjectMapper; + +import com.fasterxml.jackson.databind.ObjectMapper; +import com.fasterxml.jackson.databind.module.SimpleModule; +import com.google.common.annotations.Beta; +import com.sap.ai.sdk.orchestration.model.ChatMessagesInner; +import com.sap.ai.sdk.orchestration.model.LLMModuleResult; +import com.sap.ai.sdk.orchestration.model.ModuleResultsOutputUnmaskingInner; +import javax.annotation.Nonnull; +import lombok.AccessLevel; +import lombok.NoArgsConstructor; +import lombok.val; + +/** + * Internal utility class for getting a default object mapper with preset configuration. + * + * @since 1.2.0 + */ +@Beta +@NoArgsConstructor(access = AccessLevel.NONE) +public class OrchestrationJacksonConfiguration { + + /** + * Default object mapper used for JSON de-/serialization. Only intended for internal usage + * within this SDK. Largely follows the defaults set by Spring. + * + * @return A new object mapper with the default configuration. + * @see Jackson2ObjectMapperBuilder + */ + @Nonnull + @Beta + public static ObjectMapper getOrchestrationObjectMapper() { + + val jackson = getDefaultObjectMapper(); + + // Add mix-ins + jackson.addMixIn(LLMModuleResult.class, JacksonMixins.LLMModuleResultMixIn.class); + jackson.addMixIn( + ModuleResultsOutputUnmaskingInner.class, + JacksonMixins.ModuleResultsOutputUnmaskingInnerMixIn.class); + + final var module = + new SimpleModule() + .addDeserializer( + ChatMessagesInner.class, + PolymorphicFallbackDeserializer.fromJsonSubTypes(ChatMessagesInner.class)) + .setMixInAnnotation(ChatMessagesInner.class, JacksonMixins.NoneTypeInfoMixin.class); + jackson.registerModule(module); + return jackson; + } +} diff --git a/orchestration/src/main/java/com/sap/ai/sdk/orchestration/spring/OrchestrationChatModel.java b/orchestration/src/main/java/com/sap/ai/sdk/orchestration/spring/OrchestrationChatModel.java new file mode 100644 index 000000000..469a56086 --- /dev/null +++ b/orchestration/src/main/java/com/sap/ai/sdk/orchestration/spring/OrchestrationChatModel.java @@ -0,0 +1,78 @@ +package com.sap.ai.sdk.orchestration.spring; + +import com.google.common.annotations.Beta; +import com.sap.ai.sdk.orchestration.AssistantMessage; +import com.sap.ai.sdk.orchestration.OrchestrationClient; +import com.sap.ai.sdk.orchestration.OrchestrationPrompt; +import com.sap.ai.sdk.orchestration.SystemMessage; +import com.sap.ai.sdk.orchestration.UserMessage; +import java.util.List; +import java.util.Map; +import java.util.function.Function; +import javax.annotation.Nonnull; +import lombok.RequiredArgsConstructor; +import lombok.extern.slf4j.Slf4j; +import lombok.val; +import org.springframework.ai.chat.messages.Message; +import org.springframework.ai.chat.model.ChatModel; +import org.springframework.ai.chat.model.ChatResponse; +import org.springframework.ai.chat.prompt.Prompt; + +/** + * Spring AI integration for the orchestration service. + * + * @since 1.2.0 + */ +@Beta +@Slf4j +@RequiredArgsConstructor +public class OrchestrationChatModel implements ChatModel { + @Nonnull private OrchestrationClient client; + + /** + * Default constructor. + * + * @since 1.2.0 + */ + public OrchestrationChatModel() { + this.client = new OrchestrationClient(); + } + + @Nonnull + @Override + public ChatResponse call(@Nonnull final Prompt prompt) { + + if (prompt.getOptions() instanceof OrchestrationChatOptions options) { + + val orchestrationPrompt = toOrchestrationPrompt(prompt); + val response = client.chatCompletion(orchestrationPrompt, options.getConfig()); + return new OrchestrationSpringChatResponse(response); + } + throw new IllegalArgumentException( + "Please add OrchestrationChatOptions to the Prompt: new Prompt(\"message\", new OrchestrationChatOptions(config))"); + } + + @Nonnull + private OrchestrationPrompt toOrchestrationPrompt(@Nonnull final Prompt prompt) { + val messages = toOrchestrationMessages(prompt.getInstructions()); + return new OrchestrationPrompt(Map.of(), messages); + } + + @Nonnull + private static com.sap.ai.sdk.orchestration.Message[] toOrchestrationMessages( + @Nonnull final List messages) { + final Function mapper = + msg -> + switch (msg.getMessageType()) { + case SYSTEM: + yield new SystemMessage(msg.getText()); + case USER: + yield new UserMessage(msg.getText()); + case ASSISTANT: + yield new AssistantMessage(msg.getText()); + case TOOL: + throw new IllegalArgumentException("Tool messages are not supported"); + }; + return messages.stream().map(mapper).toArray(com.sap.ai.sdk.orchestration.Message[]::new); + } +} diff --git a/orchestration/src/main/java/com/sap/ai/sdk/orchestration/spring/OrchestrationChatOptions.java b/orchestration/src/main/java/com/sap/ai/sdk/orchestration/spring/OrchestrationChatOptions.java new file mode 100644 index 000000000..5d4a2e59c --- /dev/null +++ b/orchestration/src/main/java/com/sap/ai/sdk/orchestration/spring/OrchestrationChatOptions.java @@ -0,0 +1,178 @@ +package com.sap.ai.sdk.orchestration.spring; + +import static com.sap.ai.sdk.orchestration.OrchestrationAiModel.Parameter.FREQUENCY_PENALTY; +import static com.sap.ai.sdk.orchestration.OrchestrationAiModel.Parameter.MAX_TOKENS; +import static com.sap.ai.sdk.orchestration.OrchestrationAiModel.Parameter.PRESENCE_PENALTY; +import static com.sap.ai.sdk.orchestration.OrchestrationAiModel.Parameter.TEMPERATURE; +import static com.sap.ai.sdk.orchestration.OrchestrationAiModel.Parameter.TOP_P; +import static com.sap.ai.sdk.orchestration.OrchestrationJacksonConfiguration.getOrchestrationObjectMapper; + +import com.fasterxml.jackson.databind.ObjectMapper; +import com.google.common.annotations.Beta; +import com.sap.ai.sdk.orchestration.OrchestrationModuleConfig; +import com.sap.ai.sdk.orchestration.model.LLMModuleConfig; +import java.util.List; +import java.util.Map; +import java.util.Objects; +import javax.annotation.Nonnull; +import javax.annotation.Nullable; +import lombok.AccessLevel; +import lombok.Data; +import lombok.Getter; +import lombok.Setter; +import lombok.val; +import org.springframework.ai.chat.prompt.ChatOptions; + +/** + * Configuration to be used for orchestration requests. + * + * @since 1.2.0 + */ +@Beta +@Data +@Getter(AccessLevel.NONE) +@Setter(AccessLevel.NONE) +public class OrchestrationChatOptions implements ChatOptions { + + private static final ObjectMapper JACKSON = getOrchestrationObjectMapper(); + + @Getter(AccessLevel.PUBLIC) + @Setter(AccessLevel.PUBLIC) + @Nonnull + OrchestrationModuleConfig config; + + /** + * Returns the model to use for the chat. + * + * @return the model to use for the chat + * @see com.sap.ai.sdk.orchestration.OrchestrationAiModel + */ + @Nonnull + @Override + public String getModel() { + return getLlmConfigNonNull().getModelName(); + } + + /** + * Returns the model version to use for the chat. "latest" by default. + * + * @return the model version to use for the chat. + */ + @Nonnull + public String getModelVersion() { + return getLlmConfigNonNull().getModelVersion(); + } + + /** + * Returns the frequency penalty to use for the chat. + * + * @return the frequency penalty to use for the chat + */ + @Nullable + @Override + public Double getFrequencyPenalty() { + return getLlmConfigParam(FREQUENCY_PENALTY.getName()); + } + + /** + * Returns the maximum number of tokens to use for the chat. + * + * @return the maximum number of tokens to use for the chat + */ + @Nullable + @Override + public Integer getMaxTokens() { + return getLlmConfigParam(MAX_TOKENS.getName()); + } + + /** + * Returns the presence penalty to use for the chat. + * + * @return the presence penalty to use for the chat + */ + @Nullable + @Override + public Double getPresencePenalty() { + return getLlmConfigParam(PRESENCE_PENALTY.getName()); + } + + /** + * Returns the stop sequences to use for the chat. + * + * @return the stop sequences to use for the chat + */ + @Nullable + @Override + public List getStopSequences() { + return getLlmConfigParam("stop_sequences"); + } + + /** + * Returns the temperature to use for the chat. + * + * @return the temperature to use for the chat + */ + @Nullable + @Override + public Double getTemperature() { + return getLlmConfigParam(TEMPERATURE.getName()); + } + + /** + * Returns the top K to use for the chat. + * + * @return the top K to use for the chat + */ + @Nullable + @Override + public Integer getTopK() { + return getLlmConfigParam("top_k"); + } + + /** + * Returns the top P to use for the chat. + * + * @return the top P to use for the chat + */ + @Nullable + @Override + public Double getTopP() { + return getLlmConfigParam(TOP_P.getName()); + } + + /** + * Returns a copy of this {@link OrchestrationChatOptions}. + * + * @return a copy of this {@link OrchestrationChatOptions} + */ + @SuppressWarnings("unchecked") // The same suppress is in DefaultChatOptions + @Nonnull + @Override + public T copy() { + // note: this is a shallow copy + val copyConfig = + new OrchestrationModuleConfig() + .withTemplateConfig(config.getTemplateConfig()) + .withFilteringConfig(config.getFilteringConfig()) + .withLlmConfig(config.getLlmConfig()) + .withMaskingConfig(config.getMaskingConfig()) + .withGroundingConfig(config.getGroundingConfig()); + return (T) new OrchestrationChatOptions(copyConfig); + } + + @SuppressWarnings("unchecked") // getModelParams() returns Object, it should return Map + @Nullable + private T getLlmConfigParam(@Nonnull final String param) { + if (getLlmConfigNonNull().getModelParams() instanceof Map) { + return ((Map) getLlmConfigNonNull().getModelParams()).get(param); + } + return null; + } + + @Nonnull + private LLMModuleConfig getLlmConfigNonNull() { + return Objects.requireNonNull( + config.getLlmConfig(), + "LLM config is not set. Please set it: new OrchestrationChatOptions(new OrchestrationModuleConfig().withLlmConfig(...))"); + } +} diff --git a/orchestration/src/main/java/com/sap/ai/sdk/orchestration/spring/OrchestrationSpringChatResponse.java b/orchestration/src/main/java/com/sap/ai/sdk/orchestration/spring/OrchestrationSpringChatResponse.java new file mode 100644 index 000000000..fc61697fc --- /dev/null +++ b/orchestration/src/main/java/com/sap/ai/sdk/orchestration/spring/OrchestrationSpringChatResponse.java @@ -0,0 +1,85 @@ +package com.sap.ai.sdk.orchestration.spring; + +import com.google.common.annotations.Beta; +import com.sap.ai.sdk.orchestration.OrchestrationChatResponse; +import com.sap.ai.sdk.orchestration.model.LLMChoice; +import com.sap.ai.sdk.orchestration.model.LLMModuleResultSynchronous; +import com.sap.ai.sdk.orchestration.model.TokenUsage; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import javax.annotation.Nonnull; +import lombok.EqualsAndHashCode; +import lombok.Value; +import lombok.val; +import org.springframework.ai.chat.messages.AssistantMessage; +import org.springframework.ai.chat.metadata.ChatResponseMetadata; +import org.springframework.ai.chat.metadata.DefaultUsage; +import org.springframework.ai.chat.model.ChatResponse; +import org.springframework.ai.chat.model.Generation; + +/** + * Response from the orchestration service in a Spring AI {@link ChatResponse}. + * + * @since 1.2.0 + */ +@Beta +@Value +@EqualsAndHashCode(callSuper = true) +public class OrchestrationSpringChatResponse extends ChatResponse { + + OrchestrationChatResponse orchestrationResponse; + + OrchestrationSpringChatResponse(@Nonnull final OrchestrationChatResponse orchestrationResponse) { + super( + toGenerations( + (LLMModuleResultSynchronous) + orchestrationResponse.getOriginalResponse().getOrchestrationResult()), + toChatResponseMetadata( + (LLMModuleResultSynchronous) + orchestrationResponse.getOriginalResponse().getOrchestrationResult())); + this.orchestrationResponse = orchestrationResponse; + } + + @Nonnull + static List toGenerations(@Nonnull final LLMModuleResultSynchronous result) { + return result.getChoices().stream() + .map(OrchestrationSpringChatResponse::toAssistantMessage) + .map(Generation::new) + .toList(); + } + + @Nonnull + static AssistantMessage toAssistantMessage(@Nonnull final LLMChoice choice) { + final Map metadata = new HashMap<>(); + metadata.put("finish_reason", choice.getFinishReason()); + metadata.put("index", choice.getIndex()); + if (!choice.getLogprobs().isEmpty()) { + metadata.put("logprobs", choice.getLogprobs()); + } + return new AssistantMessage(choice.getMessage().getContent(), metadata); + } + + @Nonnull + static ChatResponseMetadata toChatResponseMetadata( + @Nonnull final LLMModuleResultSynchronous orchestrationResult) { + val metadataBuilder = ChatResponseMetadata.builder(); + + metadataBuilder + .id(orchestrationResult.getId()) + .model(orchestrationResult.getModel()) + .keyValue("object", orchestrationResult.getObject()) + .keyValue("created", orchestrationResult.getCreated()) + .usage(toDefaultUsage(orchestrationResult.getUsage())); + + return metadataBuilder.build(); + } + + @Nonnull + private static DefaultUsage toDefaultUsage(@Nonnull final TokenUsage usage) { + return new DefaultUsage( + usage.getPromptTokens().longValue(), + usage.getCompletionTokens().longValue(), + usage.getTotalTokens().longValue()); + } +} diff --git a/orchestration/src/test/java/com/sap/ai/sdk/orchestration/spring/OrchestrationChatModelTest.java b/orchestration/src/test/java/com/sap/ai/sdk/orchestration/spring/OrchestrationChatModelTest.java new file mode 100644 index 000000000..7af22441b --- /dev/null +++ b/orchestration/src/test/java/com/sap/ai/sdk/orchestration/spring/OrchestrationChatModelTest.java @@ -0,0 +1,71 @@ +package com.sap.ai.sdk.orchestration.spring; + +import static com.github.tomakehurst.wiremock.client.WireMock.aResponse; +import static com.github.tomakehurst.wiremock.client.WireMock.post; +import static com.github.tomakehurst.wiremock.client.WireMock.stubFor; +import static com.github.tomakehurst.wiremock.client.WireMock.urlPathEqualTo; +import static com.sap.ai.sdk.orchestration.OrchestrationAiModel.GPT_35_TURBO_16K; +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.assertThatThrownBy; + +import com.github.tomakehurst.wiremock.junit5.WireMockRuntimeInfo; +import com.github.tomakehurst.wiremock.junit5.WireMockTest; +import com.sap.ai.sdk.orchestration.OrchestrationClient; +import com.sap.ai.sdk.orchestration.OrchestrationModuleConfig; +import com.sap.cloud.sdk.cloudplatform.connectivity.ApacheHttpClient5Accessor; +import com.sap.cloud.sdk.cloudplatform.connectivity.ApacheHttpClient5Cache; +import com.sap.cloud.sdk.cloudplatform.connectivity.DefaultHttpDestination; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.ai.chat.prompt.Prompt; + +@WireMockTest +public class OrchestrationChatModelTest { + + private static OrchestrationChatModel client; + private static OrchestrationModuleConfig config; + private static Prompt prompt; + + @BeforeEach + void setup(WireMockRuntimeInfo server) { + final DefaultHttpDestination destination = + DefaultHttpDestination.builder(server.getHttpBaseUrl()).build(); + client = new OrchestrationChatModel(new OrchestrationClient(destination)); + config = new OrchestrationModuleConfig().withLlmConfig(GPT_35_TURBO_16K); + prompt = + new Prompt( + "Hello World! Why is this phrase so famous?", new OrchestrationChatOptions(config)); + ApacheHttpClient5Accessor.setHttpClientCache(ApacheHttpClient5Cache.DISABLED); + } + + @Test + void testCompletion() { + stubFor( + post(urlPathEqualTo("/completion")) + .willReturn( + aResponse() + .withBodyFile("templatingResponse.json") + .withHeader("Content-Type", "application/json"))); + final var result = client.call(prompt); + + assertThat(result).isNotNull(); + assertThat(result.getResult().getOutput().getContent()).isNotEmpty(); + } + + @Test + void testThrowsOnMissingChatOptions() { + assertThatThrownBy(() -> client.call(new Prompt("test"))) + .isExactlyInstanceOf(IllegalArgumentException.class) + .hasMessageContaining("Please add OrchestrationChatOptions to the Prompt"); + } + + @Test + void testThrowsOnMissingLlmConfig() { + OrchestrationChatOptions emptyConfig = + new OrchestrationChatOptions(new OrchestrationModuleConfig()); + + assertThatThrownBy(() -> client.call(new Prompt("test", emptyConfig))) + .isExactlyInstanceOf(IllegalStateException.class) + .hasMessageContaining("LLM config is required"); + } +} diff --git a/orchestration/src/test/java/com/sap/ai/sdk/orchestration/spring/OrchestrationChatOptionsTest.java b/orchestration/src/test/java/com/sap/ai/sdk/orchestration/spring/OrchestrationChatOptionsTest.java new file mode 100644 index 000000000..428dd1adf --- /dev/null +++ b/orchestration/src/test/java/com/sap/ai/sdk/orchestration/spring/OrchestrationChatOptionsTest.java @@ -0,0 +1,77 @@ +package com.sap.ai.sdk.orchestration.spring; + +import static com.sap.ai.sdk.orchestration.OrchestrationAiModel.GEMINI_1_5_FLASH; +import static com.sap.ai.sdk.orchestration.OrchestrationAiModel.Parameter.FREQUENCY_PENALTY; +import static com.sap.ai.sdk.orchestration.OrchestrationAiModel.Parameter.MAX_TOKENS; +import static com.sap.ai.sdk.orchestration.OrchestrationAiModel.Parameter.PRESENCE_PENALTY; +import static com.sap.ai.sdk.orchestration.OrchestrationAiModel.Parameter.TEMPERATURE; +import static com.sap.ai.sdk.orchestration.OrchestrationAiModel.Parameter.TOP_P; +import static org.assertj.core.api.Assertions.assertThat; + +import com.sap.ai.sdk.orchestration.OrchestrationAiModel; +import com.sap.ai.sdk.orchestration.OrchestrationModuleConfig; +import java.util.List; +import org.junit.jupiter.api.Test; + +class OrchestrationChatOptionsTest { + + static final OrchestrationAiModel CUSTOM_LLM = + GEMINI_1_5_FLASH + .withParam(FREQUENCY_PENALTY, 0.5) + .withParam(MAX_TOKENS, 100) + .withParam(PRESENCE_PENALTY, 0.5) + .withParam("stop_sequences", List.of("\n")) + .withParam(TEMPERATURE, 0.5) + .withParam("top_k", 50) + .withParam(TOP_P, 0.5); + + private static void assertCustomLLM(OrchestrationChatOptions opts) { + assertThat(opts.getModel()).isEqualTo(GEMINI_1_5_FLASH.getName()); + assertThat(opts.getModelVersion()).isEqualTo(GEMINI_1_5_FLASH.getVersion()); + assertThat(opts.getFrequencyPenalty()).isEqualTo(0.5); + assertThat(opts.getMaxTokens()).isEqualTo(100); + assertThat(opts.getPresencePenalty()).isEqualTo(0.5); + assertThat(opts.getStopSequences()).containsExactly("\n"); + assertThat(opts.getTemperature()).isEqualTo(0.5); + assertThat(opts.getTopK()).isEqualTo(50); + assertThat(opts.getTopP()).isEqualTo(0.5); + } + + @Test + void testParametersAreInherited() { + var opts = + new OrchestrationChatOptions( + new OrchestrationModuleConfig().withLlmConfig(GEMINI_1_5_FLASH)); + + assertThat(opts.getModel()).isEqualTo(GEMINI_1_5_FLASH.getName()); + assertThat(opts.getModelVersion()).isEqualTo(GEMINI_1_5_FLASH.getVersion()); + } + + @Test + void testCustomParametersAreInherited() { + var opts = + new OrchestrationChatOptions(new OrchestrationModuleConfig().withLlmConfig(CUSTOM_LLM)); + + assertCustomLLM(opts); + } + + @Test + void testCopy() { + var opts = + new OrchestrationChatOptions( + new OrchestrationModuleConfig().withLlmConfig(GEMINI_1_5_FLASH)); + + var copy = (OrchestrationChatOptions) opts.copy(); + assertThat(copy.getModel()).isEqualTo(GEMINI_1_5_FLASH.getName()); + assertThat(copy.getModelVersion()).isEqualTo(GEMINI_1_5_FLASH.getVersion()); + } + + @Test + void testCustomCopy() { + var opts = + new OrchestrationChatOptions(new OrchestrationModuleConfig().withLlmConfig(CUSTOM_LLM)); + + var copy = (OrchestrationChatOptions) opts.copy(); + assertCustomLLM(copy); + } +} diff --git a/orchestration/src/test/java/com/sap/ai/sdk/orchestration/spring/OrchestrationChatResponseTest.java b/orchestration/src/test/java/com/sap/ai/sdk/orchestration/spring/OrchestrationChatResponseTest.java new file mode 100644 index 000000000..5c2e10d82 --- /dev/null +++ b/orchestration/src/test/java/com/sap/ai/sdk/orchestration/spring/OrchestrationChatResponseTest.java @@ -0,0 +1,54 @@ +package com.sap.ai.sdk.orchestration.spring; + +import static org.assertj.core.api.Assertions.assertThat; + +import com.sap.ai.sdk.orchestration.model.ChatMessage; +import com.sap.ai.sdk.orchestration.model.LLMChoice; +import com.sap.ai.sdk.orchestration.model.LLMModuleResultSynchronous; +import com.sap.ai.sdk.orchestration.model.TokenUsage; +import java.util.List; +import org.junit.jupiter.api.Test; +import org.springframework.ai.chat.messages.AssistantMessage; + +class OrchestrationChatResponseTest { + + @Test + void testToAssistantMessage() { + var choice = + LLMChoice.create() + .index(0) + .message(ChatMessage.create().role("assistant").content("Hello, world!")) + .finishReason("stop"); + + AssistantMessage message = OrchestrationSpringChatResponse.toAssistantMessage(choice); + + assertThat(message.getContent()).isEqualTo("Hello, world!"); + assertThat(message.getMetadata()).containsEntry("finish_reason", "stop"); + assertThat(message.getMetadata()).containsEntry("index", 0); + } + + @Test + void testToChatResponseMetadata() { + var moduleResult = + LLMModuleResultSynchronous.create() + .id("test-id") + ._object("test-object") + .created(123456789) + .model("test-model") + .choices(List.of()) + .usage(TokenUsage.create().completionTokens(20).promptTokens(10).totalTokens(30)); + + var metadata = OrchestrationSpringChatResponse.toChatResponseMetadata(moduleResult); + + assertThat(metadata.getId()).isEqualTo("test-id"); + assertThat(metadata.getModel()).isEqualTo("test-model"); + assertThat(metadata.get("object")).isEqualTo("test-object"); + assertThat(metadata.get("created")).isEqualTo(123456789); + + var usage = metadata.getUsage(); + + assertThat(usage.getPromptTokens()).isEqualTo(10L); + assertThat(usage.getGenerationTokens()).isEqualTo(20L); + assertThat(usage.getTotalTokens()).isEqualTo(30L); + } +} diff --git a/pom.xml b/pom.xml index 090aeb7f1..97daab9b6 100644 --- a/pom.xml +++ b/pom.xml @@ -64,8 +64,11 @@ 2.1.3 3.5.2 6.2.1 + 1.0.0-M5 3.1.0 5.15.2 + + 1.14.2 2.44.1 false @@ -104,6 +107,11 @@ spring-web ${springframework.version} + + org.springframework.ai + spring-ai-core + ${spring-ai.version} + io.github.cdimascio dotenv-java @@ -114,6 +122,27 @@ jackson-module-parameter-names 2.18.2 + + + io.micrometer + micrometer-core + ${micrometer.version} + + + io.micrometer + micrometer-observation + ${micrometer.version} + + + org.springframework + spring-beans + ${springframework.version} + + + org.springframework + spring-context + ${springframework.version} + org.junit.jupiter @@ -175,6 +204,24 @@ test + + + + true + + spring-milestones + Spring Milestones + https://repo.spring.io/milestone + + + + false + + spring-snapshots + Spring Snapshots + https://repo.spring.io/snapshot + + @@ -633,11 +680,12 @@ https://gitbox.apache.org/repos/asf?p=maven-pmd-plugin.git;a=blob_plain;f=src/ma Apache Software License - Version 2.0|Apache License Version 2.0|Apache 2.0| The Apache License, Version 2.0|Apache License, Version 2.0|Apache-2.0| - The Apache Software License, Version 2.0 - The MIT License|MIT License|The MIT License (MIT)|MIT - The BSD 3-Clause License|BSD License 3 + The Apache Software License, Version 2.0|Apache License 2.0 + The MIT License|MIT License|The MIT License (MIT)|MIT|MIT-0 + The BSD 3-Clause License|BSD License 3|The BSD License|BSD-3-Clause|BSD-2-Clause|BSD licence Eclipse Distribution License - v 1.0|EDL 1.0 Eclipse Public License v2.0|EPL 2.0 + Public Domain|Public Domain, per Creative Commons CC0 diff --git a/sample-code/spring-app/pom.xml b/sample-code/spring-app/pom.xml index 19e57d4de..8611807e3 100644 --- a/sample-code/spring-app/pom.xml +++ b/sample-code/spring-app/pom.xml @@ -61,6 +61,10 @@ com.sap.cloud.sdk.cloudplatform cloudplatform-core + + org.springframework.ai + spring-ai-core + org.springframework.boot spring-boot-autoconfigure @@ -70,12 +74,6 @@ org.springframework.boot spring-boot ${spring-boot.version} - - - org.springframework - spring-context - - org.springframework diff --git a/sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/controllers/SpringAiOrchestrationController.java b/sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/controllers/SpringAiOrchestrationController.java new file mode 100644 index 000000000..e9594481a --- /dev/null +++ b/sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/controllers/SpringAiOrchestrationController.java @@ -0,0 +1,66 @@ +package com.sap.ai.sdk.app.controllers; + +import static com.sap.ai.sdk.orchestration.OrchestrationJacksonConfiguration.getOrchestrationObjectMapper; + +import com.fasterxml.jackson.core.JsonProcessingException; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.sap.ai.sdk.app.services.SpringAiOrchestrationService; +import com.sap.ai.sdk.orchestration.spring.OrchestrationSpringChatResponse; +import javax.annotation.Nullable; +import lombok.val; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.http.ResponseEntity; +import org.springframework.web.bind.annotation.GetMapping; +import org.springframework.web.bind.annotation.RequestHeader; +import org.springframework.web.bind.annotation.RequestMapping; +import org.springframework.web.bind.annotation.RestController; + +@SuppressWarnings("unused") +@RestController +@RequestMapping("/spring-ai-orchestration") +class SpringAiOrchestrationController { + @Autowired private SpringAiOrchestrationService service; + private static final ObjectMapper MAPPER = getOrchestrationObjectMapper(); + + @GetMapping("/completion") + ResponseEntity completion( + @Nullable @RequestHeader(value = "accept", required = false) final String accept) + throws JsonProcessingException { + val response = (OrchestrationSpringChatResponse) service.completion(); + + if ("application/json".equals(accept)) { + return ResponseEntity.ok() + .body( + MAPPER.writeValueAsString(response.getOrchestrationResponse().getOriginalResponse())); + } + return ResponseEntity.ok(response.getResult().getOutput().getContent()); + } + + @GetMapping("/template") + ResponseEntity template( + @Nullable @RequestHeader(value = "accept", required = false) final String accept) + throws JsonProcessingException { + val response = (OrchestrationSpringChatResponse) service.template(); + + if ("application/json".equals(accept)) { + return ResponseEntity.ok() + .body( + MAPPER.writeValueAsString(response.getOrchestrationResponse().getOriginalResponse())); + } + return ResponseEntity.ok(response.getResult().getOutput().getContent()); + } + + @GetMapping("/masking") + ResponseEntity masking( + @Nullable @RequestHeader(value = "accept", required = false) final String accept) + throws JsonProcessingException { + val response = (OrchestrationSpringChatResponse) service.masking(); + + if ("application/json".equals(accept)) { + return ResponseEntity.ok() + .body( + MAPPER.writeValueAsString(response.getOrchestrationResponse().getOriginalResponse())); + } + return ResponseEntity.ok(response.getResult().getOutput().getContent()); + } +} diff --git a/sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/services/SpringAiOrchestrationService.java b/sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/services/SpringAiOrchestrationService.java new file mode 100644 index 000000000..0219af8c8 --- /dev/null +++ b/sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/services/SpringAiOrchestrationService.java @@ -0,0 +1,76 @@ +package com.sap.ai.sdk.app.services; + +import static com.sap.ai.sdk.orchestration.OrchestrationAiModel.GPT_35_TURBO; + +import com.sap.ai.sdk.orchestration.DpiMasking; +import com.sap.ai.sdk.orchestration.OrchestrationModuleConfig; +import com.sap.ai.sdk.orchestration.model.DPIEntities; +import com.sap.ai.sdk.orchestration.spring.OrchestrationChatModel; +import com.sap.ai.sdk.orchestration.spring.OrchestrationChatOptions; +import java.util.Map; +import javax.annotation.Nonnull; +import lombok.val; +import org.springframework.ai.chat.model.ChatModel; +import org.springframework.ai.chat.model.ChatResponse; +import org.springframework.ai.chat.prompt.Prompt; +import org.springframework.ai.chat.prompt.PromptTemplate; +import org.springframework.stereotype.Service; + +/** Service class for the Orchestration service */ +@Service +public class SpringAiOrchestrationService { + private final ChatModel client = new OrchestrationChatModel(); + private final OrchestrationModuleConfig config = + new OrchestrationModuleConfig().withLlmConfig(GPT_35_TURBO); + private final OrchestrationChatOptions defaultOptions = new OrchestrationChatOptions(config); + + /** + * Chat request to OpenAI through the Orchestration service with a simple prompt. + * + * @return the assistant response object + */ + @Nonnull + public ChatResponse completion() { + val prompt = new Prompt("What is the capital of France?", defaultOptions); + + return client.call(prompt); + } + + /** + * Chat request to OpenAI through the Orchestration service with a template. + * + * @return the assistant response object + */ + @Nonnull + public ChatResponse template() { + val template = new PromptTemplate("{input}"); + val prompt = template.create(Map.of("input", "What is the capital of France?"), defaultOptions); + + return client.call(prompt); + } + + /** + * Let the orchestration service evaluate the feedback on the AI SDK provided by a hypothetical + * user. Anonymize any names given as they are not relevant for judging the sentiment of the + * feedback. + * + * @link SAP AI + * Core: Orchestration - Data Masking + * @return the assistant response object + */ + @Nonnull + public ChatResponse masking() { + val masking = + DpiMasking.anonymization() + .withEntities(DPIEntities.EMAIL, DPIEntities.ADDRESS, DPIEntities.LOCATION); + + val opts = new OrchestrationChatOptions(config.withMaskingConfig(masking)); + val prompt = + new Prompt( + "Please write 'Hello World!' to me via email. My email address is foo.bar@baz.ai", + opts); + + return client.call(prompt); + } +} diff --git a/sample-code/spring-app/src/main/resources/static/AI-SDK-Logo.svg b/sample-code/spring-app/src/main/resources/static/AI-SDK-Logo.svg new file mode 100644 index 000000000..7dfe5d8b0 --- /dev/null +++ b/sample-code/spring-app/src/main/resources/static/AI-SDK-Logo.svg @@ -0,0 +1,269 @@ + +image/svg+xml + diff --git a/sample-code/spring-app/src/main/resources/static/BTP-Cockpit-Logo.png b/sample-code/spring-app/src/main/resources/static/BTP-Cockpit-Logo.png new file mode 100644 index 000000000..6d8abb0d8 Binary files /dev/null and b/sample-code/spring-app/src/main/resources/static/BTP-Cockpit-Logo.png differ diff --git a/sample-code/spring-app/src/main/resources/static/Open-AI-Logo.svg b/sample-code/spring-app/src/main/resources/static/Open-AI-Logo.svg new file mode 100644 index 000000000..066a6119f --- /dev/null +++ b/sample-code/spring-app/src/main/resources/static/Open-AI-Logo.svg @@ -0,0 +1,11 @@ + + + + + + + + + + + diff --git a/sample-code/spring-app/src/main/resources/static/Orchestration-Logo.png b/sample-code/spring-app/src/main/resources/static/Orchestration-Logo.png new file mode 100644 index 000000000..7e50946dd Binary files /dev/null and b/sample-code/spring-app/src/main/resources/static/Orchestration-Logo.png differ diff --git a/sample-code/spring-app/src/main/resources/static/index.html b/sample-code/spring-app/src/main/resources/static/index.html index 0d2511c38..a3cda0a49 100644 --- a/sample-code/spring-app/src/main/resources/static/index.html +++ b/sample-code/spring-app/src/main/resources/static/index.html @@ -139,12 +139,15 @@

Response

-

Java AI SDK Application

+
+ AI SDK Logo +

Java AI SDK Application

+
-

AI Core

+

🧰 AI Core

The AI Core API provides tools to manage the lifecycle of your own AI scenarios, including artifacts, pipeline execution, and scalable deployments for training and inference. For more information, check the AI Core documentation.
@@ -243,9 +246,12 @@
Configurations
-

Orchestration

+
+ Orchestration Logo +

Orchestration

+
The Orchestration API offers functionality for enhancing your LLM calls with Templating, Filtering, Data Masking, Grounding and more. - For more information, check the Orchestration documentation. + For more information, check the Orchestration documentation
@@ -627,4 +692,4 @@
OpenAI
- \ No newline at end of file + diff --git a/sample-code/spring-app/src/test/java/com/sap/ai/sdk/app/controllers/SpringAiOrchestrationTest.java b/sample-code/spring-app/src/test/java/com/sap/ai/sdk/app/controllers/SpringAiOrchestrationTest.java new file mode 100644 index 000000000..220f45d5a --- /dev/null +++ b/sample-code/spring-app/src/test/java/com/sap/ai/sdk/app/controllers/SpringAiOrchestrationTest.java @@ -0,0 +1,33 @@ +package com.sap.ai.sdk.app.controllers; + +import static org.assertj.core.api.Assertions.assertThat; + +import com.sap.ai.sdk.app.services.SpringAiOrchestrationService; +import org.junit.jupiter.api.Test; +import org.springframework.ai.chat.model.ChatResponse; + +public class SpringAiOrchestrationTest { + + SpringAiOrchestrationService service = new SpringAiOrchestrationService(); + + @Test + void testCompletion() { + ChatResponse response = service.completion(); + assertThat(response).isNotNull(); + assertThat(response.getResult().getOutput().getContent()).contains("Paris"); + } + + @Test + void testTemplate() { + ChatResponse response = service.template(); + assertThat(response).isNotNull(); + assertThat(response.getResult().getOutput().getContent()).isNotEmpty(); + } + + @Test + void testMasking() { + ChatResponse response = service.masking(); + assertThat(response).isNotNull(); + assertThat(response.getResult().getOutput().getContent()).isNotEmpty(); + } +}