-
Notifications
You must be signed in to change notification settings - Fork 16
feat: Orchestration Grounding convenience #293
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 9 commits
986924e
50f0f45
ec28d01
e0cd777
078df19
2b090de
d594557
beaeeff
ecfb112
7b839a6
d27b9e0
8ab0ddd
d072452
8ca2f66
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -206,31 +206,31 @@ In this example, the input will be masked before the call to the LLM and will re | |
| Use the grounding module to provide additional context to the AI model. | ||
|
|
||
| ```java | ||
| var message = | ||
| Message.user( | ||
| "{{?groundingInput}} Use the following information as additional context: {{?groundingOutput}}"); | ||
| var prompt = | ||
| new OrchestrationPrompt(Map.of("groundingInput", "What does Joule do?"), message); | ||
|
|
||
| var filterInner = | ||
| DocumentGroundingFilter.create().id("someID").dataRepositoryType(DataRepositoryType.VECTOR); | ||
| var groundingConfigConfig = | ||
| GroundingModuleConfigConfig.create() | ||
| .inputParams(List.of("groundingInput")) | ||
| .outputParam("groundingOutput") | ||
| .addFiltersItem(filterInner); | ||
|
|
||
| var groundingConfig = | ||
| GroundingModuleConfig.create() | ||
| .type(GroundingModuleConfig.TypeEnum.DOCUMENT_GROUNDING_SERVICE) | ||
| .config(groundingConfigConfig); | ||
| var configWithGrounding = config.withGroundingConfig(groundingConfig); | ||
|
|
||
| var result = | ||
| new OrchestrationClient().chatCompletion(prompt, configWithGrounding); | ||
| // optional filter for collections | ||
| var documentMetadata = | ||
| SearchDocumentKeyValueListPair.create() | ||
| .key("my-collection") | ||
| .value("value") | ||
| .selectMode(List.of(SearchSelectOptionEnum.IGNORE_IF_KEY_ABSENT)); | ||
| // optional filter for document chunks | ||
| var databaseFilter = | ||
| DocumentGroundingFilter.create() | ||
| .id("") | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. (Minor/Comment) According to spec this identifier shall be unique per request. See specGroundingModuleConfig:
type: object
required:
- type
- config
additionalProperties: false
properties:
type:
...
config:
type: object
required:
- input_params
- output_param
additionalProperties: false
properties:
filters:
type: array
items:
oneOf:
- $ref: "#/components/schemas/DocumentGroundingFilter"
description: Document grounding service filters to be used
...
DocumentGroundingFilter:
type: object
required:
- id
- data_repository_type
additionalProperties: false
properties:
id:
$ref: "#/components/schemas/GroundingFilterId"
...
GroundingFilterId:
title: Id
description: Identifier of this SearchFilter - unique per request.
GroundingFilterSearchConfiguration:
...With the proposed convenience API we are
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Therefore:
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I modified the API to allow 1+ filters. |
||
| .dataRepositoryType(DataRepositoryType.VECTOR) | ||
| .documentMetadata(List.of(documentMetadata)); | ||
CharlesDuboisSAP marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| var groundingConfig = Grounding.create().filter(databaseFilter); | ||
| var prompt = groundingConfig.createGroundingPrompt("What does Joule do?"); | ||
| var configWithGrounding = config.withGrounding(groundingConfig); | ||
|
|
||
| var result = client.chatCompletion(prompt, configWithGrounding); | ||
| ``` | ||
|
|
||
| In this example, the AI model is provided with additional context in the form of grounding information. Note, that it is necessary to provide the grounding input via one or more input variables. | ||
| In this example, the AI model is provided with additional context in the form of grounding information. | ||
|
|
||
| `Grounding.create()` is by default a document grounding service with a vector data repository. | ||
|
|
||
| Please find [an example in our Spring Boot application](../../sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/services/OrchestrationService.java). | ||
|
|
||
| ## Stream chat completion | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,74 @@ | ||
| package com.sap.ai.sdk.orchestration; | ||
|
|
||
| import com.google.common.annotations.Beta; | ||
| import com.sap.ai.sdk.orchestration.model.DataRepositoryType; | ||
| import com.sap.ai.sdk.orchestration.model.DocumentGroundingFilter; | ||
| import com.sap.ai.sdk.orchestration.model.GroundingModuleConfig; | ||
| import com.sap.ai.sdk.orchestration.model.GroundingModuleConfig.TypeEnum; | ||
| import com.sap.ai.sdk.orchestration.model.GroundingModuleConfigConfig; | ||
| import java.util.List; | ||
| import java.util.Map; | ||
| import javax.annotation.Nonnull; | ||
| import lombok.Setter; | ||
| import lombok.experimental.Accessors; | ||
| import lombok.val; | ||
|
|
||
| /** | ||
| * Grounding integrates external, contextually relevant, domain-specific, or real-time data into AI | ||
| * processes. This data supplements the natural language processing capabilities of pre-trained | ||
| * models, which are trained on general material. | ||
| * | ||
| * @link <a href="https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/grounding">SAP AI | ||
| * Core: Orchestration - Grounding</a> | ||
| */ | ||
| @Beta | ||
| @Setter(onMethod_ = {@Nonnull}) | ||
| @Accessors(fluent = true) | ||
| public class Grounding implements GroundingProvider { | ||
|
|
||
| private DocumentGroundingFilter filter = | ||
| DocumentGroundingFilter.create().id("").dataRepositoryType(DataRepositoryType.VECTOR); | ||
| private TypeEnum documentGroundingService = TypeEnum.DOCUMENT_GROUNDING_SERVICE; | ||
|
|
||
| /** | ||
| * Create a new default grounding provider. | ||
| * | ||
| * <p>It is by default a document grounding service with a vector data repository. | ||
| * | ||
| * @return The grounding provider. | ||
| */ | ||
| @Nonnull | ||
| public static Grounding create() { | ||
| return new Grounding(); | ||
| } | ||
|
|
||
| /** | ||
| * Create a prompt with grounding parameters included in the message. | ||
| * | ||
| * <p>It uses the inputParams {@code userMessage} for the user message and {@code | ||
| * groundingContext} for the grounding context. | ||
| * | ||
| * @param message The user message. | ||
| * @return The prompt with grounding. | ||
| */ | ||
| @Nonnull | ||
| public OrchestrationPrompt createGroundingPrompt(@Nonnull final String message) { | ||
| return new OrchestrationPrompt( | ||
| Map.of("userMessage", message), | ||
| Message.user( | ||
| "{{?userMessage}} Use the following information as additional context: {{?groundingContext}}")); | ||
| } | ||
|
|
||
| @Nonnull | ||
| @Override | ||
| public GroundingModuleConfig createConfig() { | ||
| val groundingConfigConfig = | ||
| GroundingModuleConfigConfig.create() | ||
| .inputParams(List.of("userMessage")) | ||
| .outputParam("groundingContext") | ||
| .addFiltersItem(filter); | ||
| return GroundingModuleConfig.create() | ||
| .type(documentGroundingService) | ||
| .config(groundingConfigConfig); | ||
| } | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| package com.sap.ai.sdk.orchestration; | ||
|
|
||
| import com.sap.ai.sdk.orchestration.model.GroundingModuleConfig; | ||
| import javax.annotation.Nonnull; | ||
|
|
||
| /** | ||
| * Interface for grounding configurations. | ||
| * | ||
| * @link <a href="https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/grounding">SAP AI | ||
| * Core: Orchestration - Grounding</a> | ||
| */ | ||
| @FunctionalInterface | ||
| public interface GroundingProvider { | ||
CharlesDuboisSAP marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| /** | ||
| * Create a grounding configuration. | ||
| * | ||
| * @return the grounding configuration | ||
| */ | ||
| @Nonnull | ||
| GroundingModuleConfig createConfig(); | ||
| } | ||
Uh oh!
There was an error while loading. Please reload this page.