Merge remote-tracking branch 'origin/main' into openai/generated-model

Author: a-d

docs/guides/ORCHESTRATION_CHAT_COMPLETION.md

@@ -108,7 +108,8 @@ Use a prepared template and execute requests with by passing only the input para
 
 ```java
 var template = Message.user("Reply with 'Orchestration Service is working!' in {{?language}}");
-var templatingConfig = TemplatingModuleConfig.create().template(template);
+var templatingConfig =
+        TemplateConfig.create().withTemplate(List.of(template.createChatMessage()));
 var configWithTemplate = config.withTemplateConfig(templatingConfig);
 
 var inputParams = Map.of("language", "German");
@@ -119,6 +120,22 @@ var result = client.chatCompletion(prompt, configWithTemplate);
 
 In this case the template is defined with the placeholder `{{?language}}` which is replaced by the value `German` in the input parameters.
 
+Alternatively, you can use already prepared templates from the [Prompt Registry](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/prompt-registry) of SAP AI Core instead of passing a template in the request yourself.
+
+```java
+var template = TemplateConfig.reference().byId("21cb1358-0bf1-4f43-870b-00f14d0f9f16");
+var configWithTemplate = config.withTemplateConfig(template);
+
+var inputParams = Map.of("language", "Italian", "input", "cloud ERP systems");
+var prompt = new OrchestrationPrompt(inputParams);
+
+var result = client.chatCompletion(prompt, configWithTemplate);
+```
+
+A prompt template can be referenced either by ID as above, or by using a combination of name, scenario, and version. For details on storing a template in the Prompt Registry, refer to [this guide](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-prompt-template-imperative).
+
+You can find [some examples](https://github.com/SAP/ai-sdk-java/tree/main/sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/services/OrchestrationService.java) in our Spring Boot application demonstrating using templates from Prompt Registry.
+
 ## Message history
 
 Include a message history to maintain context in the conversation:
@@ -210,6 +227,10 @@ In this example, the input will be masked before the call to the LLM and will re
 
 Use the grounding module to provide additional context to the AI model. 
 
+### Vector Data Repository
+
+One way to provide grounding is by using a vector data repository. This can be done as follows.
+
 ```java
 // optional filter for collections
 var documentMetadata =
@@ -233,7 +254,37 @@ var result = client.chatCompletion(prompt, configWithGrounding);
 
 In this example, the AI model is provided with additional context in the form of grounding information.
 
-`Grounding.create()` is by default a document grounding service with a vector data repository.
+### Grounding via *help.sap.com*
+
+You can also use grounding based on *help.sap.com* for convenient SAP specific grounding. This can be achieved as follows.
+
+```java
+var groundingHelpSapCom =
+        DocumentGroundingFilter.create()
+                .dataRepositoryType(DataRepositoryType.HELP_SAP_COM);
+var groundingConfig = Grounding.create().filters(groundingHelpSapCom);
+var configWithGrounding = config.withGrounding(groundingConfig);
+
+var prompt = groundingConfig.createGroundingPrompt("What is a fuzzy search?");
+var response = client.chatCompletion(prompt, configWithGrounding);
+```
+
+Please find [an example in our Spring Boot application](../../sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/services/OrchestrationService.java).
+
+### Mask Grounding
+
+You can also mask both the grounding information and the prompt message:
+
+```java
+var maskingConfig =
+    DpiMasking.anonymization()
+        .withEntities(DPIEntities.SENSITIVE_DATA)
+        .withMaskGroundingEnabled()
+        .withAllowList(List.of("SAP", "Joule"));
+var maskedGroundingConfig = groundingConfig.withMaskingConfig(maskingConfig);
+
+var result = client.chatCompletion(prompt, maskedGroundingConfig);
+```
 
 Please find [an example in our Spring Boot application](../../sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/services/OrchestrationService.java).
 

docs/guides/SPRING_AI_INTEGRATION.md

@@ -7,6 +7,7 @@
 - [Orchestration Masking](#orchestration-masking)
 - [Stream chat completion](#stream-chat-completion)
 - [Tool Calling](#tool-calling)
+- [Chat Memory](#chat-memory)
 
 ## Introduction
 
@@ -137,3 +138,26 @@ ChatResponse response = client.call(prompt);
 
 Please find [an example in our Spring Boot application](../../sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/services/SpringAiOrchestrationService.java).
 
+## Chat Memory
+
+Create a Spring AI `ChatClient` from our `OrchestrationChatModel` and add a chat memory advisor like so:
+
+```java
+ChatModel client = new OrchestrationChatModel();
+OrchestrationModuleConfig config = new OrchestrationModuleConfig().withLlmConfig(GPT_35_TURBO);
+OrchestrationChatOptions opts = new OrchestrationChatOptions(config);
+
+val memory = new InMemoryChatMemory();
+val advisor = new MessageChatMemoryAdvisor(memory);
+val cl = ChatClient.builder(client).defaultAdvisors(advisor).build();
+
+Prompt prompt1 = new Prompt("What is the capital of France?", defaultOptions);
+String content1 = cl.prompt(prompt1).call().content();
+// content1 is "Paris"
+
+Prompt prompt2 = new Prompt("And what is the typical food there?", defaultOptions);
+String content2 = cl.prompt(prompt2).call().content();
+// chat memory will remember that the user is inquiring about France.
+```
+
+Please find [an example in our Spring Boot application](../../sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/services/SpringAiOrchestrationService.java).

docs/release-notes/release_notes.md

@@ -12,7 +12,10 @@
 
 ### ✨ New Functionality
 
--
+- [Orchestration] [Add Spring AI Chat Memory support](https://github.com/SAP/ai-sdk-java/tree/main/docs/guides/SPRING_AI_INTEGRATION.md#chat-memory)
+- [Orchestration] [Prompt templates can be consumed from registry.](https://github.com/SAP/ai-sdk-java/tree/main/docs/guides/ORCHESTRATION_CHAT_COMPLETION.md#Chat-completion-with-Templates)
+- [Orchestration] [Masking is now available on grounding.](https://github.com/SAP/ai-sdk-java/tree/main/docs/guides/ORCHESTRATION_CHAT_COMPLETION.md#mask-grounding)
+- [Orchestration] [Grounding via *help.sap.com* is enabled.](https://github.com/SAP/ai-sdk-java/tree/main/docs/guides/ORCHESTRATION_CHAT_COMPLETION.md#grounding)
 
 ### 📈 Improvements
 

foundation-models/openai/src/main/java/com/sap/ai/sdk/foundationmodels/openai/OpenAiError.java

@@ -8,7 +8,6 @@
 import lombok.AccessLevel;
 import lombok.AllArgsConstructor;
 import lombok.Value;
-import lombok.experimental.Delegate;
 
 /**
  * Represents an error response from the OpenAI API.
@@ -20,7 +19,6 @@
 @AllArgsConstructor(onConstructor = @__({@JsonCreator}), access = AccessLevel.PROTECTED)
 public class OpenAiError implements ClientError {
   /** The original error response from the OpenAI API. */
-  @Delegate(types = {ClientError.class})
   ErrorResponse originalResponse;
 
   /**

orchestration/pom.xml

@@ -31,10 +31,10 @@
   </developers>
   <properties>
     <project.rootdir>${project.basedir}/../</project.rootdir>
-    <coverage.complexity>82%</coverage.complexity>
-    <coverage.line>93%</coverage.line>
+    <coverage.complexity>83%</coverage.complexity>
+    <coverage.line>94%</coverage.line>
     <coverage.instruction>94%</coverage.instruction>
-    <coverage.branch>76%</coverage.branch>
+    <coverage.branch>77%</coverage.branch>
     <coverage.method>93%</coverage.method>
     <coverage.class>100%</coverage.class>
   </properties>

orchestration/src/main/java/com/sap/ai/sdk/orchestration/ConfigToRequestTransformer.java

@@ -5,6 +5,7 @@
 import com.sap.ai.sdk.orchestration.model.ModuleConfigs;
 import com.sap.ai.sdk.orchestration.model.OrchestrationConfig;
 import com.sap.ai.sdk.orchestration.model.Template;
+import com.sap.ai.sdk.orchestration.model.TemplateRef;
 import com.sap.ai.sdk.orchestration.model.TemplatingModuleConfig;
 import io.vavr.control.Option;
 import java.util.ArrayList;
@@ -47,6 +48,9 @@ static TemplatingModuleConfig toTemplateModuleConfig(
      * In this case, the request will fail, since the templating module will try to resolve the parameter.
      * To be fixed with https://github.tools.sap/AI/llm-orchestration/issues/662
      */
+    if (config instanceof TemplateRef) {
+      return config;
+    }
     val template = config instanceof Template t ? t : Template.create().template();
     val messages = template.getTemplate();
     val responseFormat = template.getResponseFormat();

orchestration/src/main/java/com/sap/ai/sdk/orchestration/DpiMasking.java

@@ -5,6 +5,7 @@
 import static com.sap.ai.sdk.orchestration.model.DPIConfig.TypeEnum.SAP_DATA_PRIVACY_INTEGRATION;
 
 import com.sap.ai.sdk.orchestration.model.DPIConfig;
+import com.sap.ai.sdk.orchestration.model.DPIConfigMaskGroundingInput;
 import com.sap.ai.sdk.orchestration.model.DPIEntities;
 import com.sap.ai.sdk.orchestration.model.DPIEntityConfig;
 import com.sap.ai.sdk.orchestration.model.MaskingProviderConfig;
@@ -16,6 +17,7 @@
 import lombok.Getter;
 import lombok.RequiredArgsConstructor;
 import lombok.Value;
+import lombok.With;
 import lombok.val;
 
 /**
@@ -31,6 +33,8 @@
 public class DpiMasking implements MaskingProvider {
   @Nonnull DPIConfig.MethodEnum maskingMethod;
   @Nonnull List<DPIEntities> entities;
+  @With boolean maskGroundingInput;
+  @Nonnull List<String> allowList;
 
   /**
    * Build a configuration applying anonymization.
@@ -65,7 +69,7 @@ public static class Builder {
      *
      * @param entity An entity type to mask (required)
      * @param entities Additional entity types to mask (optional)
-     * @return A configured {@link DpiMasking} instance
+     * @return A new {@link DpiMasking} instance
      * @see DPIEntities
      */
     @Nonnull
@@ -74,17 +78,30 @@ public DpiMasking withEntities(
       val entitiesList = new ArrayList<DPIEntities>();
       entitiesList.add(entity);
       entitiesList.addAll(Arrays.asList(entities));
-      return new DpiMasking(maskingMethod, entitiesList);
+      return new DpiMasking(maskingMethod, entitiesList, false, List.of());
     }
   }
 
+  /**
+   * Set words that should not be masked.
+   *
+   * @param allowList List of strings that should not be masked
+   * @return A new {@link DpiMasking} instance
+   */
+  @Nonnull
+  public DpiMasking withAllowList(@Nonnull final List<String> allowList) {
+    return new DpiMasking(maskingMethod, entities, maskGroundingInput, allowList);
+  }
+
   @Nonnull
   @Override
   public MaskingProviderConfig createConfig() {
     val entitiesDTO = entities.stream().map(it -> DPIEntityConfig.create().type(it)).toList();
     return DPIConfig.create()
         .type(SAP_DATA_PRIVACY_INTEGRATION)
         .method(maskingMethod)
-        .entities(entitiesDTO);
+        .entities(entitiesDTO)
+        .maskGroundingInput(DPIConfigMaskGroundingInput.create().enabled(maskGroundingInput))
+        .allowlist(allowList);
   }
 }

orchestration/src/main/java/com/sap/ai/sdk/orchestration/Grounding.java

@@ -85,6 +85,11 @@ public GroundingModuleConfig createConfig() {
             .outputParam("groundingContext")
             .filters(filters);
 
+    if (filters.contains(
+        DocumentGroundingFilter.create().dataRepositoryType(DataRepositoryType.HELP_SAP_COM))) {
+      groundingConfigConfig.setMetadataParams(null);
+    }
+
     return GroundingModuleConfig.create()
         .type(documentGroundingService)
         .config(groundingConfigConfig);

orchestration/src/main/java/com/sap/ai/sdk/orchestration/OrchestrationError.java

@@ -4,10 +4,10 @@
 import com.google.common.annotations.Beta;
 import com.sap.ai.sdk.core.common.ClientError;
 import com.sap.ai.sdk.orchestration.model.ErrorResponse;
+import javax.annotation.Nonnull;
 import lombok.AccessLevel;
 import lombok.AllArgsConstructor;
 import lombok.Value;
-import lombok.experimental.Delegate;
 
 /**
  * Orchestration error response.
@@ -18,6 +18,17 @@
 @Value
 @Beta
 public class OrchestrationError implements ClientError {
-  @Delegate(types = {ClientError.class})
   ErrorResponse originalResponse;
+
+  /**
+   * Gets the error message from the contained original response.
+   *
+   * @return the error message
+   */
+  @Nonnull
+  public String getMessage() {
+    return originalResponse.getCode() == 500
+        ? originalResponse.getMessage() + " located in " + originalResponse.getLocation()
+        : originalResponse.getMessage();
+  }
 }

orchestration/src/main/java/com/sap/ai/sdk/orchestration/model/AzureContentSafety.java

@@ -1,6 +1,6 @@
 /*
  * Internal Orchestration Service API
- * SAP AI Core - Orchestration Service API
+ * Orchestration is an inference service which provides common additional capabilities for business AI scenarios, such as content filtering and data masking. At the core of the service is the LLM module which allows for an easy, harmonized access to the language models of gen AI hub. The service is designed to be modular and extensible, allowing for the addition of new modules in the future. Each module can be configured independently and at runtime, allowing for a high degree of flexibility in the orchestration of AI services.
  *
  *
  *