diff --git a/best-practices/generative-ai/document-grounding/README.md b/best-practices/generative-ai/document-grounding/README.md new file mode 100644 index 00000000..48182d09 --- /dev/null +++ b/best-practices/generative-ai/document-grounding/README.md @@ -0,0 +1,12 @@ +# Demo for SAP Document Grounding Service +Document Grounding service provides a simplified way of performing Retrieval Augmented Generation on your domain data. There are several out-of-the-box data connectors supported with this service to process source files. This section provides guidance and examples for connecting to these sources and different approaches to perform RAG. + +Article related to this topic: [SAP Document Grounding Service](TODO) + +## Language-Specific Examples +This repository includes examples in multiple programming languages to demonstrate these best practices: +* Python: Python implementation for accessing AI models +* TODO + +Getting Started +Each language-specific folder contains its own README with detailed setup and usage instructions. Choose the implementation that best matches your technology stack to get started. \ No newline at end of file diff --git a/best-practices/generative-ai/document-grounding/python/Document_AI_S3_API.ipynb b/best-practices/generative-ai/document-grounding/python/Document_AI_S3_API.ipynb new file mode 100644 index 00000000..9249a6a3 --- /dev/null +++ b/best-practices/generative-ai/document-grounding/python/Document_AI_S3_API.ipynb @@ -0,0 +1,434 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Document Grounding on S3 data using Using Document Management APIs\n", + "\n", + "Purpose: Ground LLM responses on your enterprise data stored in S3, with SAP Document Grounding service using Using Document Management APIs. The tutorial demonstrates different steps to set up and implement the grounding service with S3 as a data source.\n", + "\n", + "\n", + "The process consists of three steps: \n", + "* Step 1: Upload Documents to S3 bucket \n", + "* Step 2: Create Data Repository pointing to S3 bucket \n", + "* Step 3: Retrieve most similar documents from Data Repository based on input query and generate augmented answer\n", + "\n", + "\n", + "**Pre-requisites:**\n", + "* Object Store (S3) and its credentials\n", + "\n", + "\n", + "**Step 1:**\n", + "Push your document(s) to the S3 bucket. \n", + "\n", + "**Step 2:**\n", + "* Create Generic Secret on AI Launchpad with S3 object store credentials.\n", + "* Use Document Management Pipelines API to create a Data Repository from S3 bucket.\n", + "\n", + "**Step 3:**\n", + "* Use Document Management Retrieval API to fetch most similar documents from the Data Repository\n", + "* [Optional] Use Gen AI Hub SDK to access an LLM to create answer using the retrieved documents as a context.\n", + "\n", + "\n", + "In this tutorial, we will:\n", + "1. Create Access Token.\n", + "2. Create Repository using Pipelines API.\n", + "3. Retrieve documents using Retrieval API.\n", + "3. [OPTIONAL] Use GPT-4o model to generate answer" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 1: Data Load to S3" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Obtain Object Store Credentials** \n", + "* Download S3 credentials from BTP cockpit > BTP subaccount > Space > Instances > ObjectStore > Credentials\n", + "\n", + "**AWS CLI Installation and Configuration** \n", + "* Install AWS CLI: Download and install the AWS Command Line Interface from the official AWS documentation [Link](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html).\n", + "* Verify Installation: Check your installation by running:\n", + "``` sh\n", + "aws --version\n", + "```\n", + "* Configure AWS CLI: Run the configuration command and pass corresponding values from Object Store credentials.\n", + "``` sh\n", + "aws configure\n", + "```\n", + "\n", + "**Push Documents to S3** \n", + "You can push your documents to S3 bucket. You can optionally put the documents to a sub-directory in the bucket as well.\n", + "\n", + "S3 CLI commands: \n", + "``` sh\n", + "aws s3 cp sample.pdf s3://your-bucket-name/sample.pdf\n", + "OR\n", + "aws s3 cp . s3://your-bucket-name/ --recursive\n", + "OR\n", + "aws s3 cp . s3://your-bucket-name// --recursive\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 2: Create Data Repository using Pipelines API" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Step 2.1: Create Generic Secret \n", + "Create Generic Secret key collection on AI Launchpad with S3 Object Store credentials. [SAP Help](https://help.sap.com/docs/ai-launchpad/sap-ai-launchpad/grounding-management)." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Step 2.2: Generate Access Token\n", + "\n", + "Create Access Token using the AI Core credentials." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "True" + ] + }, + "execution_count": 1, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "import os\n", + "from dotenv import load_dotenv\n", + "load_dotenv(override=True)" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Access token obtained successfully.\n" + ] + } + ], + "source": [ + "import requests\n", + "\n", + "# Replace these with your actual service key details\n", + "client_id = os.getenv(\"AICORE_CLIENT_ID\")\n", + "client_secret = os.getenv(\"AICORE_CLIENT_SECRET\")\n", + "auth_url = os.getenv(\"AICORE_AUTH_URL\")\n", + "\n", + "# Prepare the payload and headers\n", + "payload = {\n", + " \"grant_type\": \"client_credentials\"\n", + "}\n", + "headers = {\n", + " \"Content-Type\": \"application/x-www-form-urlencoded\"\n", + "}\n", + "\n", + "# Make the POST request to obtain the token\n", + "response = requests.post(auth_url, data=payload, headers=headers, auth=(client_id, client_secret))\n", + "\n", + "# Check if the request was successful\n", + "if response.status_code == 200:\n", + " access_token = response.json().get(\"access_token\")\n", + " print(\"Access token obtained successfully.\")\n", + "else:\n", + " print(f\"Failed to obtain access token: {response.status_code} - {response.text}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Step 2.3: Create Data Repository using Pipeline API\n", + "\n", + "As a pre-requisite, the S3 credentials are added as a generic secret on AI Core. You need to replace the generic secret key name with yours in the following code.\n", + "\n", + "Optionally, specify the path in S3 bucket from where the documents are to be considered for grounding." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Status Code: 201\n", + "Response: {\"pipelineId\": \"13ce3a20-26a8-455c-a06e-1343be4a8bf7\"}\n", + "\n" + ] + } + ], + "source": [ + "AI_API_URL = r\"https://api.ai.prod.eu-central-1.aws.ml.hana.ondemand.com\" # Update your AI_API_URL as per the aws region\n", + "url = f\"{AI_API_URL}/v2/lm/document-grounding/pipelines\"\n", + "\n", + "headers = {\"Authorization\": f\"Bearer {access_token}\",\n", + " \"AI-Resource-Group\": \"default\", # update your resource group name here\n", + " \"Content-Type\": \"application/json\"}\n", + "\n", + "payload = {\n", + " \"type\": \"S3\",\n", + " \"configuration\": {\n", + " \"destination\": \"ai-best-practices-grounding-s3-b2\", # Update your S3 generic secret key name here\n", + " \"s3\": {\n", + " \"includePaths\": [\"/new_papers/\"] # Sub-directory path (also called as perfix) in S3 bucket to consider for fetching data from\n", + " }\n", + " }\n", + "}\n", + "\n", + "response = requests.post(url, headers=headers, json=payload)\n", + "\n", + "print(\"Status Code:\", response.status_code)\n", + "print(\"Response:\", response.text)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Step 2.4: Verify created Data Repository\n", + "\n", + "**Get the details for newly created Data Repository**\n", + " \n", + "Go to Grounding Management page on AI Launchpad and note the ID of Data Repository for the pipeline ID that you just created in above step." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Status Code: 200\n", + "Response: {\"id\": \"b27f2f08-d6e2-4907-920d-7d08035c4de8\", \"title\": \"pipeline-13ce3a20-26a8-455c-a06e-1343be4a8bf7-collection\", \"metadata\": [{\"key\": \"pipeline\", \"value\": [\"13ce3a20-26a8-455c-a06e-1343be4a8bf7\"]}, {\"key\": \"type\", \"value\": [\"custom\"]}, {\"key\": \"pipelineType\", \"value\": [\"S3\"]}], \"type\": \"vector\"}\n", + "\n" + ] + } + ], + "source": [ + "AI_API_URL = r\"https://api.ai.prod.eu-central-1.aws.ml.hana.ondemand.com\" # Replace with your AI API URL\n", + "repository_id = \"b27f2f08-d6e2-4907-920d-7d08035c4de8\" # Paste Data Repository ID from Grounding Management page on AI Launchpad\n", + "\n", + "url = f\"{AI_API_URL}/v2/lm/document-grounding/retrieval/dataRepositories/{repository_id}\"\n", + "\n", + "\n", + "headers = {\"Authorization\": f\"Bearer {access_token}\",\n", + " \"AI-Resource-Group\": \"default\", # Update your resource group name here\n", + " \"Content-Type\": \"application/json\"}\n", + "\n", + "response = requests.get(url, headers=headers)\n", + "\n", + "print(\"Status Code:\", response.status_code)\n", + "print(\"Response:\", response.text)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 3: Retrieve Similar Documents\n", + "\n", + "Add ID of your data repository in the following cell. If want to include more data repositories, you can add them in the list as well.\n", + "\n", + "Optionally, specify the path in S3 bucket from where the documents are to be considered grounding." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Status Code: 200\n", + "The concept of receptive field is important for understanding and diagnosing how deep CNNs work.\n", + "Since anywhere in an input image outside the receptive field of a unit does not affect the value of that\n", + "unit, it is necessary to carefully control the receptive field, to ensure that it covers the entire relevant\n", + "image region. In many tasks, especially dense prediction tasks like semantic image segmentation,\n", + "stereo and optical flow estimation, where we make a prediction for each single pixel in the input image,\n", + "it is critical for each output pixel to have a big receptive field, such that no important information is\n", + "left out when making the prediction.\n", + "The receptive field size of a unit can be increased in a number of ways. One option is to stack more\n", + "layers to make the network deeper, which increases the receptive field size linearly by theory, as\n", + "each extra layer increases the receptive field size by the kernel size. Sub-sampling on the other hand\n", + "takes up a fraction of the full theoretical receptive field. Empirical results echoed the theory we\n", + "established. We believe this is just the start of the study of effective receptive field, which provides a\n", + "new angle to understand deep CNNs. In the future we hope to study more about what factors impact\n", + "effective receptive field in practice and how we can gain more control over them.\n", + "8\n" + ] + } + ], + "source": [ + "url = f\"{AI_API_URL}/v2/lm/document-grounding/retrieval/search\"\n", + "\n", + "headers = {\n", + " \"AI-Resource-Group\": \"default\",\n", + " \"Authorization\": f\"Bearer {access_token}\",\n", + " \"Content-Type\": \"application/json\"\n", + "}\n", + "\n", + "payload = {\n", + " \"query\": \"What is efficient receptive field?\",\n", + " \"filters\": [\n", + " {\n", + " \"id\": \"string\",\n", + " \"searchConfiguration\": {\n", + " \"maxChunkCount\": 2\n", + " },\n", + " \"dataRepositories\": [\"b27f2f08-d6e2-4907-920d-7d08035c4de8\"], # Specify your repository ID(s)\n", + " \"dataRepositoryType\": \"vector\"\n", + " }\n", + " ]\n", + "}\n", + "\n", + "response = requests.post(url, headers=headers, json=payload)\n", + "\n", + "print(\"Status Code:\", response.status_code)\n", + "response_text = response.text\n", + "\n", + "import json\n", + "\n", + "# Parse the JSON string into a dictionary\n", + "response_dict = json.loads(response_text)\n", + "retrieved_docs = [] \n", + "# Loop through and print each \"content\"\n", + "for result in response_dict.get(\"results\", []):\n", + " for res in result.get(\"results\", []):\n", + " for document in res.get(\"dataRepository\", {}).get(\"documents\", []):\n", + " for chunk in document.get(\"chunks\", []):\n", + " retrieved_docs.append(chunk.get(\"content\", \"\"))\n", + "\n", + "for doc in retrieved_docs:\n", + " print(doc)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 3.1 [OPTIONAL]: Augment answer generation with retrieved documents" + ] + }, + { + "cell_type": "code", + "execution_count": 33, + "metadata": {}, + "outputs": [], + "source": [ + "context = ' '.join([c for c in retrieved_docs])\n", + "\n", + "query = \"What is receptive field?\"" + ] + }, + { + "cell_type": "code", + "execution_count": 34, + "metadata": {}, + "outputs": [], + "source": [ + "prompt = f\"\"\"\n", + "Use the following context information to answer to user's query.\n", + "Here is some context: {context}\n", + "\n", + "Based on the above context, answer the following query:\n", + "{query}\n", + "\n", + "The answer tone has to be very professional in nature.\n", + "\n", + "If you don't know the answer, politely say that you don't know, don't try to make up an answer.\n", + "\"\"\"" + ] + }, + { + "cell_type": "code", + "execution_count": 35, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "In the context of deep convolutional neural networks (CNNs), the receptive field refers to the specific portion of the input image that affects the output of a particular unit or neuron within the network. Essentially, it constitutes the region in which information influences the activation of that unit. The concept of the receptive field is crucial for understanding how deep CNNs operate and is particularly important in tasks involving dense predictions like semantic image segmentation, stereo vision, and optical flow estimation, where predictions need to be made for individual pixels within an image.\n", + "\n", + "To ensure that CNNs efficiently cover the necessary image regions, the receptive field size must be carefully managed. Increasing the receptive field allows for more comprehensive capture of relevant information needed for accurate predictions. This can be achieved by adding more layers, which theoretically increases the receptive field size linearly, as each layer adds to the receptive field proportional to the kernel size. However, techniques like sub-sampling can reduce the effective receptive field.\n", + "\n", + "Understanding and controlling the receptive field size is a developing area in the study of CNNs, with ongoing research into the factors that affect the effective receptive field and how to optimize its control.\n", + "\n", + "\n" + ] + } + ], + "source": [ + "from gen_ai_hub.proxy.native.openai import chat\n", + "\n", + "messages = [\n", + " {\"role\": \"system\", \"content\": \"You are an intelligent assistant.\"},\n", + " {\"role\": \"user\", \"content\": prompt}\n", + "]\n", + "\n", + "kwargs = dict(model_name=\"gpt-4o\", messages=messages)\n", + "\n", + "response = chat.completions.create(**kwargs)\n", + "\n", + "print(response.choices[0].message.content)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.11" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/best-practices/generative-ai/document-grounding/python/Document_AI_S3_SDK.ipynb b/best-practices/generative-ai/document-grounding/python/Document_AI_S3_SDK.ipynb new file mode 100644 index 00000000..fe556455 --- /dev/null +++ b/best-practices/generative-ai/document-grounding/python/Document_AI_S3_SDK.ipynb @@ -0,0 +1,294 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Document Grounding on S3 data using SAP Cloud SDK for AI (Python)\n", + "\n", + "Purpose: Ground LLM responses on your enterprise data stored in S3, with SAP Document Grounding service using Ochestration module of SAP Cloud SDK (Python). The tutorial demonstrates different steps to set up and implement the grounding service with S3 as a data source.\n", + "\n", + "\n", + "The process consists of three steps: \n", + "* Step 1: Upload Documents to S3 bucket \n", + "* Step 2: Create Data Repository pointing to S3 bucket \n", + "* Step 3: Retrieve most similar documents from Data Repository based on input query and generate augmented answer\n", + "\n", + "\n", + "**Pre-requisites:**\n", + "* Object Store (S3)\n", + "* Orchestration service is deployed on AI Core. [SAP Help](https://help.sap.com/docs/ai-launchpad/sap-ai-launchpad/create-deployment-for-orchestration?q=orchestration)\n", + "\n", + "\n", + "**Step 1:**\n", + "Push your document(s) to the S3 bucket. \n", + "\n", + "**Step 2:**\n", + "Create Generic Secret and a data repository using AI Launchpad with S3 as a source.\n", + "\n", + "**Step 3:**\n", + "Once a data repository is created from S3, use the SDK to retrieve similar documents and complete answer generation.\n", + "\n", + "\n", + "In this tutorial, we will:\n", + "1. Initialize the Orchestration Service and configure the LLM.\n", + "2. Define a prompt template for combining the user query with retrieved context.\n", + "3. Configure grounding with vector data repositories.\n", + "4. Run the orchestration pipeline and get the grounded response." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 1: Data Load to S3" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Obtain Object Store Credentials** \n", + "* Download S3 credentials from BTP cockpit > BTP subaccount > Space > Instances > ObjectStore > Credentials\n", + "\n", + "**AWS CLI Installation and Configuration** \n", + "* Install AWS CLI: Download and install the AWS Command Line Interface from the official AWS documentation [Link](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html).\n", + "* Verify Installation: Check your installation by running:\n", + "``` sh\n", + "aws --version\n", + "```\n", + "* Configure AWS CLI: Run the configuration command and pass corresponding values from Object Store credentials.\n", + "``` sh\n", + "aws configure\n", + "```\n", + "\n", + "**Push Documents to S3** \n", + "You can push your documents to S3 bucket. You can optionally put the documents to a sub-directory in the bucket as well.\n", + "\n", + "S3 CLI commands: \n", + "``` sh\n", + "aws s3 cp sample.pdf s3://your-bucket-name/sample.pdf\n", + "OR\n", + "aws s3 cp . s3://your-bucket-name/ --recursive\n", + "OR\n", + "aws s3 cp . s3://your-bucket-name// --recursive\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 2: Create Data Repository on AI Launchpad\n", + "\n", + "Create Generic Secret key collection and a Data Repository using that on AI Launchpad. [SAP Help Documentation](https://help.sap.com/docs/ai-launchpad/sap-ai-launchpad/grounding-management).\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 3: Retrieve Documents from Data Repository" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Step 3.1: Initialize Orchestration Service and LLM\n", + "\n", + "First, we connect to the **Orchestration Service** using the service URL of your deployment.\n", + "We also define the **LLM** (in this case `gpt-4o`) with required parameters such as temperature.\n", + "\n", + "Replace the placeholder `` in the URL with your deployment ID." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "from dotenv import load_dotenv\n", + "load_dotenv(override=True)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from gen_ai_hub.orchestration.service import OrchestrationService\n", + "from gen_ai_hub.orchestration.models.config import OrchestrationConfig\n", + "from gen_ai_hub.orchestration.models.document_grounding import (GroundingModule, GroundingType, DataRepositoryType,\n", + " GroundingFilterSearch, DocumentGrounding,DocumentGroundingFilter)\n", + "from gen_ai_hub.orchestration.models.llm import LLM\n", + "orchestration_service_url = \"\"\n", + "orchestration_service = OrchestrationService(api_url=orchestration_service_url)\n", + "\n", + "llm = LLM(\n", + " name=\"gpt-4o\",\n", + " parameters={\n", + " 'temperature': 0.0,\n", + " }\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Step 3.2: Define Prompt Template\n", + "\n", + "Here, we define a **prompt template** that instructs the model to:\n", + "- Take the retrieved `Context` from the document grounding service.\n", + "- Use it along with the user’s `Question`.\n", + "\n", + "This ensures the LLM’s answers are **grounded in enterprise data** rather than generic knowledge." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from gen_ai_hub.orchestration.models.message import SystemMessage, UserMessage\n", + "from gen_ai_hub.orchestration.models.template import Template, TemplateValue\n", + "\n", + "prompt = Template(messages=[\n", + " SystemMessage(\"You are a helpful assistant. Use the given Context information for providing answer to a given query.\"),\n", + " UserMessage(\"\"\"Context: {{ ?grounding_response }}\n", + " Question: {{ ?query }}\n", + " \"\"\"),\n", + "])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Step 3.3: Configure Document Grounding\n", + "\n", + "Next, we configure **document grounding** by linking a **vector data repository**. This ensures that relevant enterprise documents are searched and retrieved for the query.\n", + "\n", + "- `data_repositories`: List of repository IDs.\n", + "- `max_chunk_count`: Controls how many chunks of relevant documents are retrieved.\n", + "\n", + "Replace the placeholder (``) with your actual repository ID." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "filters = [DocumentGroundingFilter(id=\"vector\",\n", + " data_repositories=[\"\"],\n", + " search_config=GroundingFilterSearch(max_chunk_count=3),\n", + " data_repository_type=DataRepositoryType.VECTOR.value\n", + " )]\n", + "\n", + "grounding_config = GroundingModule(\n", + " type=GroundingType.DOCUMENT_GROUNDING_SERVICE.value,\n", + " config=DocumentGrounding(input_params=[\"query\"], output_param=\"grounding_response\", filters=filters)\n", + " )\n", + "\n", + "config = OrchestrationConfig(template=prompt, llm=llm, grounding=grounding_config)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Step 3.4: Run the Orchestration Pipeline\n", + "\n", + "Finally, we execute the orchestration pipeline by passing the query. The LLM combines the query with **retrieved context from documents** and returns a **grounded answer**.\n", + "\n", + "You should see the response printed below once you run the last cell." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from functools import partial\n", + "\n", + "def _run_query(query: str, *, _service=orchestration_service, _config=config):\n", + " response = _service.run(\n", + " config=_config,\n", + " template_values=[TemplateValue(\"query\", query)]\n", + " )\n", + " print(response.orchestration_result.choices[0].message.content)\n", + "\n", + "# Use partial so you can call it simply with a query\n", + "run_query = partial(_run_query)" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "The effective receptive field is a concept introduced to better understand how deep convolutional neural networks (CNNs) process input images. While the theoretical receptive field of a unit in a CNN is determined by the network's architecture and is supposed to cover a certain area of the input image, the effective receptive field is the actual area that significantly influences the unit's output. \n", + "\n", + "Empirical studies have shown that the effective receptive field is smaller than the theoretical receptive field and follows a Gaussian distribution. This means that the central part of the theoretical receptive field has a stronger influence on the unit's output, while the outer parts contribute less. Understanding the effective receptive field is important for tasks like semantic image segmentation, stereo, and optical flow estimation, where each output pixel needs to respond to a sufficiently large area of the input image to capture all relevant information.\n", + "\n", + "The study of effective receptive fields provides insights into how deep CNNs work and suggests ways to potentially control and optimize them, such as adjusting network architecture, using nonlinear activations, dropout, sub-sampling, and skip connections.\n" + ] + } + ], + "source": [ + "run_query(\"what is effective receptive field?\")" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "The SAP Document AI Embedded Edition refers to a version of the SAP Document AI solution that is integrated within a specific workspace or environment. This edition allows users to utilize the capabilities of SAP Document AI directly within their existing systems or workflows, providing seamless access to document information extraction processes. It enables the automation of extracting relevant information from business documents and matching them to enrichment data records, all within the embedded workspace.\n" + ] + } + ], + "source": [ + "run_query(\"what is Document AI Embedded?\")" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.11" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/best-practices/generative-ai/document-grounding/python/Document_AI_Vector_API.ipynb b/best-practices/generative-ai/document-grounding/python/Document_AI_Vector_API.ipynb new file mode 100644 index 00000000..4d6a4808 --- /dev/null +++ b/best-practices/generative-ai/document-grounding/python/Document_AI_Vector_API.ipynb @@ -0,0 +1,631 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Document Grounding on using Vector APIs of Document Grounding Management\n", + "\n", + "Purpose: Ground LLM responses on your enterprise data, with SAP Document Grounding service using Vector APIs with complete control over chunking and metadata handling process. The tutorial demonstrates different steps to set up and implement the grounding service using Vector APIs for your documents.\n", + "\n", + "\n", + "The process consists of three steps: \n", + "* Step 1: Create Data Repository \n", + "* Step 2: Prepare the data \n", + "* Step 3: Ingest data using Vector APIs \n", + "* Step 4: Retrieve most similar documents from Data Repository based on input query and generate augmented answer\n", + "\n", + "\n", + "**Step 1:**\n", + "* Generate Access token.\n", + "* Create a Data Repository (also called as Collection) \n", + "\n", + "**Step 2:**\n", + "* Create chunks and metadata to add as Documents in the newly create Data Repository\n", + "\n", + "**Step 3:**\n", + "* Use Vector API to add the add the documents.\n", + "\n", + "**Step 4:**\n", + "* Use Document Management Retrieval API to fetch most similar documents from the Data Repository\n", + "* [Optional] Use Gen AI Hub SDK to access an LLM to create answer using the retrieved documents as a context.\n", + "\n", + "**Step 5: [OPTIONAL]**\n", + "* Use GPT-4o model to generate answer" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 1: Create Data Repository using Vector API" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Step 1.2: Generate Access Token\n", + "\n", + "Create Access Token using the AI Core credentials." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "True" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "import os\n", + "from dotenv import load_dotenv\n", + "load_dotenv(override=True)" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Access token obtained successfully.\n" + ] + } + ], + "source": [ + "import requests\n", + "\n", + "# Replace these with your actual service key details\n", + "client_id = os.getenv(\"AICORE_CLIENT_ID\")\n", + "client_secret = os.getenv(\"AICORE_CLIENT_SECRET\")\n", + "auth_url = os.getenv(\"AICORE_AUTH_URL\")\n", + "\n", + "# Prepare the payload and headers\n", + "payload = {\n", + " \"grant_type\": \"client_credentials\"\n", + "}\n", + "headers = {\n", + " \"Content-Type\": \"application/x-www-form-urlencoded\"\n", + "}\n", + "\n", + "# Make the POST request to obtain the token\n", + "response = requests.post(auth_url, data=payload, headers=headers, auth=(client_id, client_secret))\n", + "\n", + "# Check if the request was successful\n", + "if response.status_code == 200:\n", + " access_token = response.json().get(\"access_token\")\n", + " print(\"Access token obtained successfully.\")\n", + "else:\n", + " print(f\"Failed to obtain access token: {response.status_code} - {response.text}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Step 1.2: Create a Data Repository\n", + "\n", + "Create a data repository for your knowledge base using Vector API. Response code 202 denotes that the Repository (or called as Collection) is created successfully." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 32, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "import requests\n", + "\n", + "AI_API_URL = r\"https://api.ai.prod.eu-central-1.aws.ml.hana.ondemand.com\" # Update your AI_API_URL as per the aws region\n", + "url = f\"{AI_API_URL}/v2/lm/document-grounding/vector/collections\"\n", + "\n", + "body={\n", + " \"title\": \"bp-dg-vector-data-repo\", # Give a name to the Data Repository (Collection)\n", + " \"embeddingConfig\": {\n", + " \"modelName\": \"text-embedding-ada-002\" # Mention the name of embeddings model\n", + " }\n", + "}\n", + "\n", + "headers = {\"Authorization\": f\"Bearer {access_token}\",\n", + " \"AI-Resource-Group\": \"default\", # Mention the name of your resource group\n", + " \"Content-Type\": \"application/json\"}\n", + "response = requests.post(url, headers=headers,json=body)\n", + "response" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Step 1.3: Get the Collection ID\n", + "\n", + "Go the AI Launchpad and note the colletion ID of the newly created Document Repository" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "collection_id = \"28e4a470-75f7-4ca0-8d30-1fb9687e73b1\" # Update with your collection id obtained from AI Launchpad" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 2: Data Preparation" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The Vector APIs expects the data payload in the following format. Hence while chunking and associating metadata, it is convenient to create output structure tailored as per the payload schema. Moreover, dumping the data in JSONL would help in debugging and manual adjustment if required as well.\n", + "\n", + "Note: \n", + "* In each API call, we can push all chunks of a single document only.\n", + "* When creating Collection from an S3 bucket, the default metadatas contains 'id' as key and document name as value. To maintain consistency, I suggest, while creating a collection using Vector API, we also create similar metadata as well." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Documents written to documents.jsonl in JSONL format (one PDF per line)\n" + ] + } + ], + "source": [ + "import os\n", + "import json\n", + "import fitz # PyMuPDF\n", + "from langchain.docstore.document import Document\n", + "from langchain.text_splitter import RecursiveCharacterTextSplitter\n", + "\n", + "# Path to the directory containing PDF files\n", + "pdf_directory = '../sample_files/' # Update this to your folder path\n", + "\n", + "# Output JSONL file\n", + "output_file = 'documents.jsonl'\n", + "\n", + "# List all PDF files in the directory\n", + "pdf_files = [f for f in os.listdir(pdf_directory) if f.endswith('.pdf')]\n", + "\n", + "# Initialize the text splitter\n", + "text_splitter = RecursiveCharacterTextSplitter(\n", + " chunk_size=1000,\n", + " chunk_overlap=100\n", + ")\n", + "\n", + "with open(output_file, 'w', encoding='utf-8') as f_out:\n", + " for pdf_file in pdf_files:\n", + " file_path = os.path.join(pdf_directory, pdf_file)\n", + " \n", + " # Open the PDF file with PyMuPDF\n", + " doc = fitz.open(file_path)\n", + " full_text = \"\"\n", + " \n", + " # Extract text from each page\n", + " for page_num in range(len(doc)):\n", + " page = doc.load_page(page_num)\n", + " full_text += page.get_text()\n", + " \n", + " doc.close()\n", + " \n", + " # Create unique metadata\n", + " doc_meta = os.path.basename(file_path)\n", + " \n", + " document = Document(\n", + " page_content=full_text,\n", + " metadata={\n", + " 'source': pdf_file,\n", + " 'author': f\"Author of {pdf_file}\",\n", + " 'category': f\"Category_{os.path.splitext(pdf_file)[0]}\"\n", + " }\n", + " )\n", + " \n", + " # Split into chunks\n", + " chunks = text_splitter.split_documents([document])\n", + " \n", + " # Build the document entry\n", + " doc_entry = {\n", + " \"documents\": [\n", + " {\n", + " \"metadata\": [\n", + " {\n", + " \"key\": \"id\",\n", + " \"value\": [doc_meta] # I suggest to keep this key-value pair as default one for consistency with S3 collection\n", + " },\n", + " {\n", + " \"key\": \"url\",\n", + " \"value\": [f\"http://example.com/{doc_meta}\"] # Similarly add your metadatas\n", + " }\n", + " ],\n", + " \"chunks\": []\n", + " }\n", + " ]\n", + " }\n", + " \n", + " for idx, chunk in enumerate(chunks, start=1):\n", + " chunk_entry = {\n", + " \"content\": chunk.page_content,\n", + " \"metadata\": [\n", + " {\n", + " \"key\": \"index\",\n", + " \"value\": [str(idx)]\n", + " }\n", + " ]\n", + " }\n", + " doc_entry[\"documents\"][0][\"chunks\"].append(chunk_entry)\n", + " \n", + " # Write the document entry as one line in JSONL format\n", + " f_out.write(json.dumps(doc_entry, ensure_ascii=False) + '\\n')\n", + "\n", + "print(f\"Documents written to {output_file} in JSONL format (one PDF per line)\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 3: Data Ingestion using Vector API" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Read the payload corresponding to each document from documents.jsonl file and use Vector API to add the records." + ] + }, + { + "cell_type": "code", + "execution_count": 39, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Uploading Documents: 0%| | 0/6 [00:00