diff --git a/README.md b/README.md index 5eeabbbf61..373fcf65cd 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,25 @@ -# ChatGPT-like app with your data using Azure OpenAI and Azure AI Search (Python) + + +# RAG chat app with Azure OpenAI and Azure AI Search (Python) + +This solution creates a ChatGPT-like frontend experience over your own documents using RAG (Retrieval Augmented Generation). It uses Azure OpenAI Service to access GPT models, and Azure AI Search for data indexing and retrieval. This solution's backend is written in Python. There are also [**JavaScript**](https://aka.ms/azai/js/code), [**.NET**](https://aka.ms/azai/net/code), and [**Java**](https://aka.ms/azai/java/code) samples based on this one. Learn more about [developing AI apps using Azure AI Services](https://aka.ms/azai). diff --git a/docs/appservice.md b/docs/appservice.md index 0fbba03279..4d1701e04c 100644 --- a/docs/appservice.md +++ b/docs/appservice.md @@ -1,4 +1,4 @@ -# Debugging the app on App Service +# RAG chat: Debugging the app on App Service When you run `azd up` or `azd deploy`, it deploys your application to App Service, and displays the deployed endpoint in the console. diff --git a/docs/azd.md b/docs/azd.md index 16112660b2..0ffb37a294 100644 --- a/docs/azd.md +++ b/docs/azd.md @@ -1,4 +1,4 @@ -# Deploying with the Azure Developer CLI +# RAG chat: Deploying with the Azure Developer CLI This guide includes advanced topics that are not necessary for a basic deployment. If you are new to the project, please consult the main [README](../README.md#deploying) for steps on deploying the project. diff --git a/docs/azure_app_service.md b/docs/azure_app_service.md index 88f80318a8..a902f1d7f2 100644 --- a/docs/azure_app_service.md +++ b/docs/azure_app_service.md @@ -1,9 +1,9 @@ -# Deploying on Azure App Service +# RAG chat: Deploying on Azure App Service Due to [a limitation](https://github.com/Azure/azure-dev/issues/2736) of the Azure Developer CLI (`azd`), there can be only one host option in the [azure.yaml](../azure.yaml) file. By default, `host: containerapp` is used and `host: appservice` is commented out. -To deploy to Azure Container Apps, please follow the following steps: +To deploy to Azure App Service, please follow the following steps: 1. Comment out `host: containerapp` and uncomment `host: appservice` in the [azure.yaml](../azure.yaml) file. diff --git a/docs/azure_container_apps.md b/docs/azure_container_apps.md index 635919aee2..be7b2a56b5 100644 --- a/docs/azure_container_apps.md +++ b/docs/azure_container_apps.md @@ -1,4 +1,4 @@ -# Deploying on Azure Container Apps +# RAG chat: Deploying on Azure Container Apps Due to [a limitation](https://github.com/Azure/azure-dev/issues/2736) of the Azure Developer CLI (`azd`), there can be only one host option in the [azure.yaml](../azure.yaml) file. By default, `host: containerapp` is used and `host: appservice` is commented out. diff --git a/docs/customization.md b/docs/customization.md index a43506a29d..cb1601fafc 100644 --- a/docs/customization.md +++ b/docs/customization.md @@ -1,4 +1,4 @@ -# Customizing the Chat App +# RAG chat: Customizing the chat app This guide provides more details for customizing the Chat App. diff --git a/docs/data_ingestion.md b/docs/data_ingestion.md index 787df72f61..1d317b6345 100644 --- a/docs/data_ingestion.md +++ b/docs/data_ingestion.md @@ -1,21 +1,24 @@ -# Indexing documents for the Chat App +# RAG chat: Data ingestion -This guide provides more details for using the `prepdocs` script to index documents for the Chat App. +The [azure-search-openai-demo](/) project can set up a full RAG chat app on Azure AI Search and OpenAI so that you can chat on custom data, like internal enterprise data or domain-specific knowledge sets. For full instructions on setting up the project, consult the [main README](/README.md), and then return here for detailed instructions on the data ingestion component. + +The chat app provides two ways to ingest data: manual indexing and integrated vectorization. This document explains the differences between the two approaches and provides an overview of the manual indexing process. - [Supported document formats](#supported-document-formats) -- [Overview of the manual indexing process](#overview-of-the-manual-indexing-process) +- [Manual indexing process](#manual-indexing-process) - [Chunking](#chunking) - [Categorizing data for enhanced search](#enhancing-search-functionality-with-data-categorization) - [Indexing additional documents](#indexing-additional-documents) - [Removing documents](#removing-documents) -- [Overview of Integrated Vectorization](#overview-of-integrated-vectorization) +- [Integrated Vectorization](#integrated-vectorization) - [Indexing of additional documents](#indexing-of-additional-documents) - [Removal of documents](#removal-of-documents) - [Scheduled indexing](#scheduled-indexing) +- [Debugging tips](#debugging-tips) ## Supported document formats -In order to ingest a document format, we need a tool that can turn it into text. By default, use Azure Document Intelligence (DI in the table below), but we also have local parsers for several formats. The local parsers are not as sophisticated as Azure Document Intelligence, but they can be used to decrease charges. +In order to ingest a document format, we need a tool that can turn it into text. By default, the manual indexing uses Azure Document Intelligence (DI in the table below), but we also have local parsers for several formats. The local parsers are not as sophisticated as Azure Document Intelligence, but they can be used to decrease charges. | Format | Manual indexing | Integrated Vectorization | | ------ | ------------------------------------ | ------------------------ | @@ -29,7 +32,7 @@ In order to ingest a document format, we need a tool that can turn it into text. The Blob indexer used by the Integrated Vectorization approach also supports a few [additional formats](https://learn.microsoft.com/azure/search/search-howto-indexing-azure-blob-storage#supported-document-formats). -## Overview of the manual indexing process +## Manual indexing process The [`prepdocs.py`](../app/backend/prepdocs.py) script is responsible for both uploading and indexing documents. The typical usage is to call it using `scripts/prepdocs.sh` (Mac/Linux) or `scripts/prepdocs.ps1` (Windows), as these scripts will set up a Python virtual environment and pass in the required parameters based on the current `azd` environment. You can pass additional arguments directly to the script, for example `scripts/prepdocs.ps1 --removeall`. Whenever `azd up` or `azd provision` is run, the script is called automatically. @@ -42,12 +45,6 @@ The script uses the following steps to index documents: 3. Split the PDFs into chunks of text. 4. Upload the chunks to Azure AI Search. If using vectors (the default), also compute the embeddings and upload those alongside the text. -### Enhancing search functionality with data categorization - -To enhance search functionality, categorize data during the ingestion process with the `--category` argument, for example `scripts/prepdocs.ps1 --category ExampleCategoryName`. This argument specifies the category to which the data belongs, enabling you to filter search results based on these categories. - -After running the script with the desired category, ensure these categories are added to the 'Include Category' dropdown list. This can be found in the developer settings of [`Chat.tsx`](../app/frontend/src/pages/chat/Chat.tsx) and [`Ask.tsx`](../app/frontend/src/pages/ask/Ask.tsx). The default option for this dropdown is "All". By including specific categories, you can refine your search results more effectively. - ### Chunking We're often asked why we need to break up the PDFs into chunks when Azure AI Search supports searching large documents. @@ -56,6 +53,12 @@ Chunking allows us to limit the amount of information we send to OpenAI due to t If needed, you can modify the chunking algorithm in `scripts/prepdocslib/textsplitter.py`. +### Enhancing search functionality with data categorization + +To enhance search functionality, categorize data during the ingestion process with the `--category` argument, for example `scripts/prepdocs.ps1 --category ExampleCategoryName`. This argument specifies the category to which the data belongs, enabling you to filter search results based on these categories. + +After running the script with the desired category, ensure these categories are added to the 'Include Category' dropdown list. This can be found in the developer settings of [`Chat.tsx`](../app/frontend/src/pages/chat/Chat.tsx) and [`Ask.tsx`](../app/frontend/src/pages/ask/Ask.tsx). The default option for this dropdown is "All". By including specific categories, you can refine your search results more effectively. + ### Indexing additional documents To upload more PDFs, put them in the data/ folder and run `./scripts/prepdocs.sh` or `./scripts/prepdocs.ps1`. @@ -70,7 +73,7 @@ To remove all documents, use `scripts/prepdocs.sh --removeall` or `scripts/prepd You can also remove individual documents by using the `--remove` flag. Open either `scripts/prepdocs.sh` or `scripts/prepdocs.ps1` and replace `/data/*` with `/data/YOUR-DOCUMENT-FILENAME-GOES-HERE.pdf`. Then run `scripts/prepdocs.sh --remove` or `scripts/prepdocs.ps1 --remove`. -## Overview of Integrated Vectorization +## Integrated Vectorization Azure AI Search includes an [integrated vectorization feature](https://techcommunity.microsoft.com/t5/ai-azure-ai-services-blog/announcing-the-public-preview-of-integrated-vectorization-in/ba-p/3960809#:~:text=Integrated%20vectorization%20is%20a%20new%20feature%20of%20Azure,pull-indexers%2C%20and%20vectorization%20of%20text%20queries%20through%20vectorizers), a cloud-based approach to data ingestion. Integrated vectorization takes care of document format cracking, data extraction, chunking, vectorization, and indexing, all with Azure technologies. @@ -99,3 +102,30 @@ The Azure AI Search indexer will take care of removing those documents from the ### Scheduled indexing If you would like the indexer to run automatically, you can set it up to [run on a schedule](https://learn.microsoft.com/azure/search/search-howto-schedule-indexers). + +## Debugging tips + +If you are not sure if a file successfully uploaded, you can query the index from the Azure Portal or from the REST API. Open the index and paste the queries below into the search bar. + +To see all the filenames uploaded to the index: + +```json +{ + "search": "*", + "count": true, + "top": 1, + "facets": ["sourcefile"] +} +``` + +To search for specific filenames: + +```json +{ + "search": "*", + "count": true, + "top": 1, + "filter": "sourcefile eq 'employee_handbook.pdf'", + "facets": ["sourcefile"] +} +``` diff --git a/docs/deploy_existing.md b/docs/deploy_existing.md index 26628b83a4..b43208a69b 100644 --- a/docs/deploy_existing.md +++ b/docs/deploy_existing.md @@ -1,5 +1,5 @@ -# Deploying with existing Azure resources +# RAG chat: Deploying with existing Azure resources If you already have existing Azure resources, or if you want to specify the exact name of new Azure Resource, you can do so by setting `azd` environment values. You should set these values before running `azd up`. Once you've set them, return to the [deployment steps](../README.md#deploying). diff --git a/docs/deploy_features.md b/docs/deploy_features.md index 455fa975a0..64ee402979 100644 --- a/docs/deploy_features.md +++ b/docs/deploy_features.md @@ -1,5 +1,5 @@ -# Enabling optional features +# RAG chat: Enabling optional features This document covers optional features that can be enabled in the deployed Azure resources. You should typically enable these features before running `azd up`. Once you've set them, return to the [deployment steps](../README.md#deploying). diff --git a/docs/deploy_lowcost.md b/docs/deploy_lowcost.md index 2c41a0cc43..809e87a807 100644 --- a/docs/deploy_lowcost.md +++ b/docs/deploy_lowcost.md @@ -1,4 +1,4 @@ -# Deploying with minimal costs +# RAG chat: Deploying with minimal costs This AI RAG chat application is designed to be easily deployed using the Azure Developer CLI, which provisions the infrastructure according to the Bicep files in the `infra` folder. Those files describe each of the Azure resources needed, and configures their SKU (pricing tier) and other parameters. Many Azure services offer a free tier, but the infrastructure files in this project do *not* default to the free tier as there are often limitations in that tier. diff --git a/docs/deploy_private.md b/docs/deploy_private.md index 8863868df1..35160df628 100644 --- a/docs/deploy_private.md +++ b/docs/deploy_private.md @@ -1,5 +1,25 @@ - -# Deploying with private access + + +# RAG chat: Deploying with private access + +The [azure-search-openai-demo](/) project can set up a full RAG chat app on Azure AI Search and OpenAI so that you can chat on custom data, like internal enterprise data or domain-specific knowledge sets. For full instructions on setting up the project, consult the [main README](/README.md), and then return here for detailed instructions on configuring private endpoints. โš ๏ธ This feature is not yet compatible with Azure Container Apps, so you will need to [deploy to Azure App Service](./azure_app_service.md) instead. diff --git a/docs/gpt4v.md b/docs/gpt4v.md index 8762aeba48..c69988d66f 100644 --- a/docs/gpt4v.md +++ b/docs/gpt4v.md @@ -1,4 +1,4 @@ -# Using GPT vision model with RAG approach +# RAG chat: Using GPT vision model with RAG approach This repository now includes an example of integrating a GPT Vision model with Azure AI Search. This feature enables indexing and searching images and graphs, such as financial documents, in addition to text-based content, and then sending the retrieved content to the GPT model for response generation. diff --git a/docs/localdev.md b/docs/localdev.md index 39959db690..b107ed91f8 100644 --- a/docs/localdev.md +++ b/docs/localdev.md @@ -1,4 +1,4 @@ -# Local development of Chat App +# RAG chat: Local development of chat app You can only run locally **after** having successfully run the `azd up` command. If you haven't yet, follow the steps in [Azure deployment](../README.md#azure-deployment) above. diff --git a/docs/login_and_acl.md b/docs/login_and_acl.md index 1af26dd249..77748a975c 100644 --- a/docs/login_and_acl.md +++ b/docs/login_and_acl.md @@ -1,4 +1,25 @@ -# Setting up optional login and document level access control + + +# RAG chat: Setting up optional login and document level access control + +The [azure-search-openai-demo](/) project can set up a full RAG chat app on Azure AI Search and OpenAI so that you can chat on custom data, like internal enterprise data or domain-specific knowledge sets. For full instructions on setting up the project, consult the [main README](/README.md), and then return here for detailed instructions on configuring login and access control. ## Table of Contents diff --git a/docs/other_samples.md b/docs/other_samples.md index ad24a9864d..2e678d5ae2 100644 --- a/docs/other_samples.md +++ b/docs/other_samples.md @@ -1,4 +1,4 @@ -# Alternative RAG chat samples +# RAG chat: Alternative RAG chat samples There are an increasingly large number of ways to build RAG chat apps. diff --git a/docs/productionizing.md b/docs/productionizing.md index 9d09cf8ff2..f1b957f626 100644 --- a/docs/productionizing.md +++ b/docs/productionizing.md @@ -1,4 +1,4 @@ -# Productionizing the Chat App +# RAG chat: Productionizing the app This sample is designed to be a starting point for your own production application, but you should do a thorough review of the security and performance before deploying diff --git a/samples/chat/README.md b/samples/chat/README.md deleted file mode 100644 index 39ce5fa75a..0000000000 --- a/samples/chat/README.md +++ /dev/null @@ -1,290 +0,0 @@ ---- -name: ChatGPT-like app with your data (Python) -description: Chat with your domain data using Azure OpenAI and Azure AI Search. -languages: -- python -- typescript -- bicep -- azdeveloper -products: -- azure-openai -- azure-cognitive-search -- azure-app-service -- azure -page_type: sample -urlFragment: azure-search-openai-demo ---- - - -# ChatGPT-like app with your data using Azure OpenAI and Azure AI Search (Python) - -This solution's backend is written in Python. There are also [**JavaScript**](https://aka.ms/azai/js/code), [**.NET**](https://aka.ms/azai/net/code), and [**Java**](https://aka.ms/azai/java/code) samples based on this one. Learn more about [developing AI apps using Azure AI Services](https://aka.ms/azai). - -## Table of Contents - -- [Features](#features) -- [Azure account requirements](#azure-account-requirements) -- [Azure deployment](#azure-deployment) - - [Cost estimation](#cost-estimation) - - [Project setup](#project-setup) - - [GitHub Codespaces](#github-codespaces) - - [VS Code Dev Containers](#vs-code-dev-containers) - - [Local environment](#local-environment) - - [Deploying](#deploying) - - [Deploying again](#deploying-again) -- [Sharing environments](#sharing-environments) -- [Using the app](#using-the-app) -- [Running locally](#running-locally) -- [Monitoring with Application Insights](#monitoring-with-application-insights) -- [Customizing the UI and data](#customizing-the-ui-and-data) -- [Productionizing](#productionizing) -- [Clean up](#clean-up) -- [Troubleshooting](#troubleshooting) -- [Resources](#resources) - -[![Open in GitHub Codespaces](https://img.shields.io/static/v1?style=for-the-badge&label=GitHub+Codespaces&message=Open&color=brightgreen&logo=github)](https://github.com/codespaces/new?hide_repo_select=true&ref=main&repo=599293758&machine=standardLinux32gb&devcontainer_path=.devcontainer%2Fdevcontainer.json&location=WestUs2) -[![Open in Dev Containers](https://img.shields.io/static/v1?style=for-the-badge&label=Dev%20Containers&message=Open&color=blue&logo=visualstudiocode)](https://vscode.dev/redirect?url=vscode://ms-vscode-remote.remote-containers/cloneInVolume?url=https://github.com/azure-samples/azure-search-openai-demo) - -This sample demonstrates a few approaches for creating ChatGPT-like experiences over your own data using the Retrieval Augmented Generation pattern. It uses Azure OpenAI Service to access a GPT model (gpt-35-turbo), and Azure AI Search for data indexing and retrieval. - -The repo includes sample data so it's ready to try end to end. In this sample application we use a fictitious company called Contoso Electronics, and the experience allows its employees to ask questions about the benefits, internal policies, as well as job descriptions and roles. - -![RAG Architecture](docs/images/appcomponents.png) - -## Features - -- Chat (multi-turn) and Q&A (single turn) interfaces -- Renders citations and thought process for each answer -- Includes settings directly in the UI to tweak the behavior and experiment with options -- Integrates Azure AI Search for indexing and retrieval of documents, with support for [many document formats](/docs/data_ingestion.md#supported-document-formats) as well as [integrated vectorization](/docs/data_ingestion.md#overview-of-integrated-vectorization) -- Optional usage of [GPT-4 with vision](/docs/gpt4v.md) to reason over image-heavy documents -- Optional addition of [speech input/output](/docs/deploy_features.md#enabling-speech-inputoutput) for accessibility -- Optional automation of [user login and data access](/docs/login_and_acl.md) via Microsoft Entra -- Performance tracing and monitoring with Application Insights - -![Chat screen](docs/images/chatscreen.png) - -[๐Ÿ“บ Watch a video overview of the app.](https://youtu.be/3acB0OWmLvM) - -## Azure account requirements - -**IMPORTANT:** In order to deploy and run this example, you'll need: - -- **Azure account**. If you're new to Azure, [get an Azure account for free](https://azure.microsoft.com/free/cognitive-search/) and you'll get some free Azure credits to get started. See [guide to deploying with the free trial](docs/deploy_lowcost.md). -- **Azure subscription with access enabled for the Azure OpenAI service**. You can request access with [this form](https://aka.ms/oaiapply). If your access request to Azure OpenAI service doesn't match the [acceptance criteria](https://learn.microsoft.com/legal/cognitive-services/openai/limited-access?context=%2Fazure%2Fcognitive-services%2Fopenai%2Fcontext%2Fcontext), you can use [OpenAI public API](https://platform.openai.com/docs/api-reference/introduction) instead. Learn [how to switch to an OpenAI instance](docs/deploy_existing.md#openaicom-openai). -- **Azure account permissions**: - - Your Azure account must have `Microsoft.Authorization/roleAssignments/write` permissions, such as [Role Based Access Control Administrator](https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#role-based-access-control-administrator-preview), [User Access Administrator](https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#user-access-administrator), or [Owner](https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#owner). If you don't have subscription-level permissions, you must be granted [RBAC](https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#role-based-access-control-administrator-preview) for an existing resource group and [deploy to that existing group](docs/deploy_existing.md#resource-group). - - Your Azure account also needs `Microsoft.Resources/deployments/write` permissions on the subscription level. - -## Azure deployment - -### Cost estimation - -Pricing varies per region and usage, so it isn't possible to predict exact costs for your usage. -However, you can try the [Azure pricing calculator](https://azure.com/e/d18187516e9e421e925b3b311eec8aae) for the resources below. - -- Azure App Service: Basic Tier with 1 CPU core, 1.75 GB RAM. Pricing per hour. [Pricing](https://azure.microsoft.com/pricing/details/app-service/linux/) -- Azure OpenAI: Standard tier, GPT and Ada models. Pricing per 1K tokens used, and at least 1K tokens are used per question. [Pricing](https://azure.microsoft.com/pricing/details/cognitive-services/openai-service/) -- Azure AI Document Intelligence: SO (Standard) tier using pre-built layout. Pricing per document page, sample documents have 261 pages total. [Pricing](https://azure.microsoft.com/pricing/details/form-recognizer/) -- Azure AI Search: Standard tier, 1 replica, free level of semantic search. Pricing per hour. [Pricing](https://azure.microsoft.com/pricing/details/search/) -- Azure Blob Storage: Standard tier with ZRS (Zone-redundant storage). Pricing per storage and read operations. [Pricing](https://azure.microsoft.com/pricing/details/storage/blobs/) -- Azure Monitor: Pay-as-you-go tier. Costs based on data ingested. [Pricing](https://azure.microsoft.com/pricing/details/monitor/) - -To reduce costs, you can switch to free SKUs for various services, but those SKUs have limitations. -See this guide on [deploying with minimal costs](docs/deploy_lowcost.md) for more details. - -โš ๏ธ To avoid unnecessary costs, remember to take down your app if it's no longer in use, -either by deleting the resource group in the Portal or running `azd down`. - -### Project setup - -You have a few options for setting up this project. -The easiest way to get started is GitHub Codespaces, since it will setup all the tools for you, -but you can also [set it up locally](#local-environment) if desired. - -#### GitHub Codespaces - -You can run this repo virtually by using GitHub Codespaces, which will open a web-based VS Code in your browser: - -[![Open in GitHub Codespaces](https://img.shields.io/static/v1?style=for-the-badge&label=GitHub+Codespaces&message=Open&color=brightgreen&logo=github)](https://github.com/codespaces/new?hide_repo_select=true&ref=main&repo=599293758&machine=standardLinux32gb&devcontainer_path=.devcontainer%2Fdevcontainer.json&location=WestUs2) - -Once the codespace opens (this may take several minutes), open a terminal window. - -#### VS Code Dev Containers - -A related option is VS Code Dev Containers, which will open the project in your local VS Code using the [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers): - -1. Start Docker Desktop (install it if not already installed) -1. Open the project: - [![Open in Dev Containers](https://img.shields.io/static/v1?style=for-the-badge&label=Dev%20Containers&message=Open&color=blue&logo=visualstudiocode)](https://vscode.dev/redirect?url=vscode://ms-vscode-remote.remote-containers/cloneInVolume?url=https://github.com/azure-samples/azure-search-openai-demo) -1. In the VS Code window that opens, once the project files show up (this may take several minutes), open a terminal window. - -#### Local environment - -1. Install the required tools: - - - [Azure Developer CLI](https://aka.ms/azure-dev/install) - - [Python 3.9, 3.10, or 3.11](https://www.python.org/downloads/) - - **Important**: Python and the pip package manager must be in the path in Windows for the setup scripts to work. - - **Important**: Ensure you can run `python --version` from console. On Ubuntu, you might need to run `sudo apt install python-is-python3` to link `python` to `python3`. - - [Node.js 18+](https://nodejs.org/download/) - - [Git](https://git-scm.com/downloads) - - [Powershell 7+ (pwsh)](https://github.com/powershell/powershell) - For Windows users only. - - **Important**: Ensure you can run `pwsh.exe` from a PowerShell terminal. If this fails, you likely need to upgrade PowerShell. - -2. Create a new folder and switch to it in the terminal. -3. Run this command to download the project code: - - ```shell - azd init -t azure-search-openai-demo - ``` - - Note that this command will initialize a git repository, so you do not need to clone this repository. - -### Deploying - -Follow these steps to provision Azure resources and deploy the application code: - -1. Login to your Azure account: - - ```shell - azd auth login - ``` - -1. Create a new azd environment: - - ```shell - azd env new - ``` - - Enter a name that will be used for the resource group. - This will create a new folder in the `.azure` folder, and set it as the active environment for any calls to `azd` going forward. -1. (Optional) This is the point where you can customize the deployment by setting environment variables, in order to [use existing resources](docs/deploy_existing.md), [enable optional features (such as auth or vision)](docs/deploy_features.md), or [deploy to free tiers](docs/deploy_lowcost.md). -1. Run `azd up` - This will provision Azure resources and deploy this sample to those resources, including building the search index based on the files found in the `./data` folder. - - **Important**: Beware that the resources created by this command will incur immediate costs, primarily from the AI Search resource. These resources may accrue costs even if you interrupt the command before it is fully executed. You can run `azd down` or delete the resources manually to avoid unnecessary spending. - - You will be prompted to select two locations, one for the majority of resources and one for the OpenAI resource, which is currently a short list. That location list is based on the [OpenAI model availability table](https://learn.microsoft.com/azure/cognitive-services/openai/concepts/models#model-summary-table-and-region-availability) and may become outdated as availability changes. -1. After the application has been successfully deployed you will see a URL printed to the console. Click that URL to interact with the application in your browser. -It will look like the following: - -!['Output from running azd up'](docs/images/endpoint.png) - -> NOTE: It may take 5-10 minutes after you see 'SUCCESS' for the application to be fully deployed. If you see a "Python Developer" welcome screen or an error page, then wait a bit and refresh the page. See [guide on debugging App Service deployments](docs/appservice.md). - -### Deploying again - -If you've only changed the backend/frontend code in the `app` folder, then you don't need to re-provision the Azure resources. You can just run: - -```azd deploy``` - -If you've changed the infrastructure files (`infra` folder or `azure.yaml`), then you'll need to re-provision the Azure resources. You can do that by running: - -```azd up``` - -## Sharing environments - -To give someone else access to a completely deployed and existing environment, -either you or they can follow these steps: - -1. Install the [Azure CLI](https://learn.microsoft.com/cli/azure/install-azure-cli) -1. Run `azd init -t azure-search-openai-demo` or clone this repository. -1. Run `azd env refresh -e {environment name}` - They will need the azd environment name, subscription ID, and location to run this command. You can find those values in your `.azure/{env name}/.env` file. This will populate their azd environment's `.env` file with all the settings needed to run the app locally. -1. Set the environment variable `AZURE_PRINCIPAL_ID` either in that `.env` file or in the active shell to their Azure ID, which they can get with `az ad signed-in-user show`. -1. Run `./scripts/roles.ps1` or `.scripts/roles.sh` to assign all of the necessary roles to the user. If they do not have the necessary permission to create roles in the subscription, then you may need to run this script for them. Once the script runs, they should be able to run the app locally. - -## Running locally - -You can only run locally **after** having successfully run the `azd up` command. If you haven't yet, follow the steps in [Azure deployment](#azure-deployment) above. - -1. Run `azd auth login` -2. Change dir to `app` -3. Run `./start.ps1` or `./start.sh` or run the "VS Code Task: Start App" to start the project locally. - -See more tips in [the local development guide](docs/localdev.md). - -## Using the app - -- In Azure: navigate to the Azure WebApp deployed by azd. The URL is printed out when azd completes (as "Endpoint"), or you can find it in the Azure portal. -- Running locally: navigate to 127.0.0.1:50505 - -Once in the web app: - -- Try different topics in chat or Q&A context. For chat, try follow up questions, clarifications, ask to simplify or elaborate on answer, etc. -- Explore citations and sources -- Click on "settings" to try different options, tweak prompts, etc. - -## Monitoring with Application Insights - -By default, deployed apps use Application Insights for the tracing of each request, along with the logging of errors. - -To see the performance data, go to the Application Insights resource in your resource group, click on the "Investigate -> Performance" blade and navigate to any HTTP request to see the timing data. -To inspect the performance of chat requests, use the "Drill into Samples" button to see end-to-end traces of all the API calls made for any chat request: - -![Tracing screenshot](docs/images/transaction-tracing.png) - -To see any exceptions and server errors, navigate to the "Investigate -> Failures" blade and use the filtering tools to locate a specific exception. You can see Python stack traces on the right-hand side. - -You can also see chart summaries on a dashboard by running the following command: - -```shell -azd monitor -``` - -## Customizing the UI and data - -Once you successfully deploy the app, you can start customizing it for your needs: changing the text, tweaking the prompts, and replacing the data. Consult the [app customization guide](docs/customization.md) as well as the [data ingestion guide](docs/data_ingestion.md) for more details. - -## Productionizing - -This sample is designed to be a starting point for your own production application, -but you should do a thorough review of the security and performance before deploying -to production. Read through our [productionizing guide](docs/productionizing.md) for more details. - -## Clean up - -To clean up all the resources created by this sample: - -1. Run `azd down` -2. When asked if you are sure you want to continue, enter `y` -3. When asked if you want to permanently delete the resources, enter `y` - -The resource group and all the resources will be deleted. - -## Troubleshooting - -Here are the most common failure scenarios and solutions: - -1. The subscription (`AZURE_SUBSCRIPTION_ID`) doesn't have access to the Azure OpenAI service. Please ensure `AZURE_SUBSCRIPTION_ID` matches the ID specified in the [OpenAI access request process](https://aka.ms/oai/access). - -1. You're attempting to create resources in regions not enabled for Azure OpenAI (e.g. East US 2 instead of East US), or where the model you're trying to use isn't enabled. See [this matrix of model availability](https://aka.ms/oai/models). - -1. You've exceeded a quota, most often number of resources per region. See [this article on quotas and limits](https://aka.ms/oai/quotas). - -1. You're getting "same resource name not allowed" conflicts. That's likely because you've run the sample multiple times and deleted the resources you've been creating each time, but are forgetting to purge them. Azure keeps resources for 48 hours unless you purge from soft delete. See [this article on purging resources](https://learn.microsoft.com/azure/cognitive-services/manage-resources?tabs=azure-portal#purge-a-deleted-resource). - -1. You see `CERTIFICATE_VERIFY_FAILED` when the `prepdocs.py` script runs. That's typically due to incorrect SSL certificates setup on your machine. Try the suggestions in this [StackOverflow answer](https://stackoverflow.com/questions/35569042/ssl-certificate-verify-failed-with-python3/43855394#43855394). - -1. After running `azd up` and visiting the website, you see a '404 Not Found' in the browser. Wait 10 minutes and try again, as it might be still starting up. Then try running `azd deploy` and wait again. If you still encounter errors with the deployed app, consult the [guide on debugging App Service deployments](docs/appservice.md). Please file an issue if the logs don't help you resolve the error. - -## Resources - -- [Additional documentation for this app](docs/README.md) -- [๐Ÿ“– Revolutionize your Enterprise Data with ChatGPT: Next-gen Apps w/ Azure OpenAI and AI Search](https://aka.ms/entgptsearchblog) -- [๐Ÿ“– Azure AI Search](https://learn.microsoft.com/azure/search/search-what-is-azure-search) -- [๐Ÿ“– Azure OpenAI Service](https://learn.microsoft.com/azure/cognitive-services/openai/overview) -- [๐Ÿ“– Comparing Azure OpenAI and OpenAI](https://learn.microsoft.com/azure/cognitive-services/openai/overview#comparing-azure-openai-and-openai/) -- [๐Ÿ“– Access Control in Generative AI applications with Azure Cognitive Search](https://techcommunity.microsoft.com/t5/ai-azure-ai-services-blog/access-control-in-generative-ai-applications-with-azure/ba-p/3956408) -- [๐Ÿ“บ Quickly build and deploy OpenAI apps on Azure, infused with your own data](https://www.youtube.com/watch?v=j8i-OM5kwiY) -- [๐Ÿ“บ AI Chat App Hack series](https://www.youtube.com/playlist?list=PL5lwDBUC0ag6_dGZst5m3G72ewfwXLcXV) - -### Getting help - -This is a sample built to demonstrate the capabilities of modern Generative AI apps and how they can be built in Azure. -For help with deploying this sample, please post in [GitHub Issues](/issues). If you're a Microsoft employee, you can also post in [our Teams channel](https://aka.ms/azai-python-help). - -This repository is supported by the maintainers, _not_ by Microsoft Support, -so please use the support mechanisms described above, and we will do our best to help you out. - -### Note - ->Note: The PDF documents used in this demo contain information generated using a language model (Azure OpenAI Service). The information contained in these documents is only for demonstration purposes and does not reflect the opinions or beliefs of Microsoft. Microsoft makes no representations or warranties of any kind, express or implied, about the completeness, accuracy, reliability, suitability or availability with respect to the information contained in this document. All rights reserved to Microsoft. diff --git a/samples/data-ingestion/README.md b/samples/data-ingestion/README.md deleted file mode 100644 index 1d3926a19a..0000000000 --- a/samples/data-ingestion/README.md +++ /dev/null @@ -1,112 +0,0 @@ ---- -name: ChatGPT + Enterprise data + data ingestion -description: Prepare your documents for a RAG chat app by ingesting them into Azure AI Search using either a Python script or integrated vectorization. -languages: -- python -- typescript -- bicep -- azdeveloper -products: -- azure-openai -- azure-cognitive-search -- azure-app-service -- azure -page_type: sample -urlFragment: azure-search-openai-demo-data-ingestion ---- - - -# Indexing documents for the Chat App - -The [azure-search-openai-demo](/) project can set up a full RAG chat app on Azure AI Search and OpenAI so that you can chat on custom data, like internal enterprise data or domain-specific knowledge sets. For full instructions on setting up the project, consult the [main README](/README.md), and then return here for detailed instructions on the data ingestion component. - -This guide provides more details for using the `prepdocs` script to index documents for the Chat App. - -- [Supported document formats](#supported-document-formats) -- [Overview of the manual indexing process](#overview-of-the-manual-indexing-process) - - [Chunking](#chunking) - - [Indexing additional documents](#indexing-additional-documents) - - [Removing documents](#removing-documents) -- [Overview of Integrated Vectorization](#overview-of-integrated-vectorization) - - [Indexing of additional documents](#indexing-of-additional-documents) - - [Removal of documents](#removal-of-documents) - - [Scheduled indexing](#scheduled-indexing) - -## Supported document formats - -In order to ingest a document format, we need a tool that can turn it into text. By default, use Azure Document Intelligence (DI in the table below), but we also have local parsers for several formats. The local parsers are not as sophisticated as Azure Document Intelligence, but they can be used to decrease charges. - -| Format | Manual indexing | Integrated Vectorization | -| ------ | ------------------------------------ | ------------------------ | -| PDF | Yes (DI or local with PyPDF) | Yes | -| HTML | Yes (DI or local with BeautifulSoup) | Yes | -| DOCX, PPTX, XLSX | Yes (DI) | Yes | -| Images (JPG, PNG, BPM, TIFF, HEIFF)| Yes (DI) | Yes | -| TXT | Yes (Local) | Yes | -| JSON | Yes (Local) | Yes | -| CSV | Yes (Local) | Yes | - -The Blob indexer used by the Integrated Vectorization approach also supports a few [additional formats](https://learn.microsoft.com/azure/search/search-howto-indexing-azure-blob-storage#supported-document-formats). - -## Overview of the manual indexing process - -The [`prepdocs.py`](/app/backend/prepdocs.py) script is responsible for both uploading and indexing documents. The typical usage is to call it using `scripts/prepdocs.sh` (Mac/Linux) or `scripts/prepdocs.ps1` (Windows), as these scripts will set up a Python virtual environment and pass in the required parameters based on the current `azd` environment. Whenever `azd up` or `azd provision` is run, the script is called automatically. - -![Diagram of the indexing process](/docs/images/diagram_prepdocs.png) - -The script uses the following steps to index documents: - -1. If it doesn't yet exist, create a new index in Azure AI Search. -2. Upload the PDFs to Azure Blob Storage. -3. Split the PDFs into chunks of text. -4. Upload the chunks to Azure AI Search. If using vectors (the default), also compute the embeddings and upload those alongside the text. - -### Chunking - -We're often asked why we need to break up the PDFs into chunks when Azure AI Search supports searching large documents. - -Chunking allows us to limit the amount of information we send to OpenAI due to token limits. By breaking up the content, it allows us to easily find potential chunks of text that we can inject into OpenAI. The method of chunking we use leverages a sliding window of text such that sentences that end one chunk will start the next. This allows us to reduce the chance of losing the context of the text. - -If needed, you can modify the chunking algorithm in `scripts/prepdocslib/textsplitter.py`. - -### Indexing additional documents - -To upload more PDFs, put them in the data/ folder and run `./scripts/prepdocs.sh` or `./scripts/prepdocs.ps1`. - -A [recent change](https://github.com/Azure-Samples/azure-search-openai-demo/pull/835) added checks to see what's been uploaded before. The prepdocs script now writes an .md5 file with an MD5 hash of each file that gets uploaded. Whenever the prepdocs script is re-run, that hash is checked against the current hash and the file is skipped if it hasn't changed. - -### Removing documents - -You may want to remove documents from the index. For example, if you're using the sample data, you may want to remove the documents that are already in the index before adding your own. - -To remove all documents, use the `--removeall` flag. Open either `scripts/prepdocs.sh` or `scripts/prepdocs.ps1` and add `--removeall` to the command at the bottom of the file. Then run the script as usual. - -You can also remove individual documents by using the `--remove` flag. Open either `scripts/prepdocs.sh` or `scripts/prepdocs.ps1`, add `--remove` to the command at the bottom of the file, and replace `/data/*` with `/data/YOUR-DOCUMENT-FILENAME-GOES-HERE.pdf`. Then run the script as usual. - -## Overview of Integrated Vectorization - -Azure AI search recently introduced an [integrated vectorization feature in preview mode](https://techcommunity.microsoft.com/t5/ai-azure-ai-services-blog/announcing-the-public-preview-of-integrated-vectorization-in/ba-p/3960809#:~:text=Integrated%20vectorization%20is%20a%20new%20feature%20of%20Azure,pull-indexers%2C%20and%20vectorization%20of%20text%20queries%20through%20vectorizers). This feature is a cloud-based approach to data ingestion, which takes care of document format cracking, data extraction, chunking, vectorization, and indexing, all with Azure technologies. - -See [this notebook](https://github.com/Azure/azure-search-vector-samples/blob/main/demo-python/code/integrated-vectorization/azure-search-integrated-vectorization-sample.ipynb) to understand the process of setting up integrated vectorization. -We have integrated that code into our `prepdocs` script, so you can use it without needing to understand the details. - -This feature cannot be used on existing index. You need to create a new index or drop and recreate an existing index. -In the newly created index schema, a new field 'parent_id' is added. This is used internally by the indexer to manage life cycle of chunks. - -This feature is not supported in the free SKU for Azure AI Search. - -### Indexing of additional documents - -To add additional documents to the index, first upload them to your data source (Blob storage, by default). -Then navigate to the Azure portal, find the index, and run it. -The Azure AI Search indexer will identify the new documents and ingest them into the index. - -### Removal of documents - -To remove documents from the index, remove them from your data source (Blob storage, by default). -Then navigate to the Azure portal, find the index, and run it. -The Azure AI Search indexer will take care of removing those documents from the index. - -### Scheduled indexing - -If you would like the indexer to run automatically, you can set it up to [run on a schedule](https://learn.microsoft.com/azure/search/search-howto-schedule-indexers). diff --git a/samples/document-security/README.md b/samples/document-security/README.md deleted file mode 100644 index 5f6bde0edd..0000000000 --- a/samples/document-security/README.md +++ /dev/null @@ -1,305 +0,0 @@ ---- -name: ChatGPT + Enterprise data + document security -description: This guide demonstrates how to add an optional login and document level access control system to a RAG chat app. This system can be used to restrict access to indexed data to specific users. -languages: -- python -- typescript -- bicep -- azdeveloper -products: -- azure-openai -- azure-cognitive-search -- azure-app-service -- azure -page_type: sample -urlFragment: azure-search-openai-demo-document-security ---- - - -# Setting up optional login and document level access control - -The [azure-search-openai-demo](/) project can set up a full RAG chat app on Azure AI Search and OpenAI so that you can chat on custom data, like internal enterprise data or domain-specific knowledge sets. For full instructions on setting up the project, consult the [main README](/README.md), and then return here for detailed instructions on configuring login and access control. - -## Table of Contents - -- [Requirements](#requirements) -- [Setting up Microsoft Entra applications](#setting-up-microsoft-entra-applications) - - [Automatic Setup](#automatic-setup) - - [Manual Setup](#manual-setup) - - [Server App](#server-app) - - [Client App](#client-app) - - [Configure Server App Known Client Applications](#configure-server-app-known-client-applications) - - [Testing](#testing) - - [Programmatic Access With Authentication](#programmatic-access-with-authentication) - - [Troubleshooting](#troubleshooting) -- [Adding data with document level access control](#adding-data-with-document-level-access-control) - - [Using the Add Documents API](#using-the-add-documents-api) - - [Azure Data Lake Storage Gen2 and prepdocs](#azure-data-lake-storage-gen2-setup) -- [Environment variables reference](#environment-variables-reference) - - [Authentication behavior by environment](#authentication-behavior-by-environment) - -This guide demonstrates how to add an optional login and document level access control system to the sample. This system can be used to restrict access to indexed data to specific users based on what [Microsoft Entra groups](https://learn.microsoft.com/entra/fundamentals/how-to-manage-groups) they are a part of, or their [user object id](https://learn.microsoft.com/partner-center/find-ids-and-domain-names#find-the-user-object-id). - -![AppLoginArchitecture](/docs/images/applogincomponents.png) - -## Requirements - -**IMPORTANT:** In order to add optional login and document level access control, you'll need the following in addition to the normal sample requirements - -- **Azure account permissions**: Your Azure account must have [permission to manage applications in Microsoft Entra](https://learn.microsoft.com/entra/identity/role-based-access-control/permissions-reference#cloud-application-administrator). - -## Setting up Microsoft Entra applications - -Two Microsoft Entra applications must be registered in order to make the optional login and document level access control system work correctly. One app is for the client UI. The client UI is implemented as a [single page application](https://learn.microsoft.com/entra/identity-platform/scenario-spa-app-registration). The other app is for the API server. The API server uses a [confidential client](https://learn.microsoft.com/entra/identity-platform/msal-client-applications) to call the [Microsoft Graph API](https://learn.microsoft.com/graph/use-the-api). - -### Automatic Setup - -The easiest way to setup the two apps is to use the `azd` CLI. We've written scripts that will automatically create the two apps and configure them for use with the sample. To trigger the automatic setup, run the following commands: - -1. Run `azd env set AZURE_USE_AUTHENTICATION true` to enable the login UI and use App Service authentication by default. -1. Ensure access control is enabled on your search index. If your index doesn't exist yet, run prepdocs with `AZURE_USE_AUTHENTICATION` set to `true`. If your index already exists, run `python ./scripts/manageacl.py --acl-action enable_acls`. -1. (Optional) To require access control when using the app, run `azd env set AZURE_ENFORCE_ACCESS_CONTROL true`. Authentication is always required to search on documents with access control assigned, regardless of if unauthenticated access is enabled or not. -1. (Optional) To allow authenticated users to search on documents that have no access controls assigned, even when access control is required, run `azd env set AZURE_ENABLE_GLOBAL_DOCUMENT_ACCESS true`. -1. (Optional) To allow unauthenticated users to use the app, even when access control is enforced, run `azd env set AZURE_ENABLE_UNAUTHENTICATED_ACCESS true`. `AZURE_ENABLE_GLOBAL_DOCUMENT_ACCESS` should also be set to true if you want unauthenticated users to be able to search on documents with no access control. -1. Run `azd env set AZURE_AUTH_TENANT_ID ` to set the tenant ID associated with authentication. -1. If your auth tenant ID is different from your currently logged in tenant ID, run `azd auth login --tenant-id ` to login to the authentication tenant simultaneously. -1. Run `azd up` to deploy the app. - -### Manual Setup - -The following instructions explain how to setup the two apps using the Azure Portal. - -#### Server App - -- Sign in to the [Azure portal](https://portal.azure.com/). -- Select the Microsoft Entra ID service. -- In the left hand menu, select **Application Registrations**. -- Select **New Registration**. - - In the **Name** section, enter a meaningful application name. This name will be displayed to users of the app, for example `Azure Search OpenAI Chat API`. - - Under **Supported account types**, select **Accounts in this organizational directory only**. -- Select **Register** to create the application -- In the app's registration screen, find the **Application (client) ID**. - - Run the following `azd` command to save this ID: `azd env set AZURE_SERVER_APP_ID `. - -- Microsoft Entra supports three types of credentials to authenticate an app using the [client credentials](https://learn.microsoft.com/entra/identity-platform/v2-oauth2-client-creds-grant-flow): passwords (app secrets), certificates, and federated identity credentials. For a higher level of security, either [certificates](https://learn.microsoft.com/entra/identity-platform/howto-create-self-signed-certificate) or federated identity credentials are recommended. This sample currently uses an app secret for ease of provisioning. - -- Select **Certificates & secrets** in the left hand menu. -- In the **Client secrets** section, select **New client secret**. - - Type a description, for example `Azure Search OpenAI Chat Key`. - - Select one of the available key durations. - - The generated key value will be displayed after you select **Add**. - - Copy the generated key value and run the following `azd` command to save this ID: `azd env set AZURE_SERVER_APP_SECRET `. -- Select **API Permissions** in the left hand menu. By default, the [delegated `User.Read`](https://learn.microsoft.com/graph/permissions-reference#user-permissions) permission should be present. This permission is required to read the signed-in user's profile to get the security information used for document level access control. If this permission is not present, it needs to be added to the application. - - Select **Add a permission**, and then **Microsoft Graph**. - - Select **Delegated permissions**. - - Search for and and select `User.Read`. - - Select **Add permissions**. -- Select **Expose an API** in the left hand menu. The server app works by using the [On Behalf Of Flow](https://learn.microsoft.com/entra/identity-platform/v2-oauth2-on-behalf-of-flow#protocol-diagram), which requires the server app to expose at least 1 API. - - The application must define a URI to expose APIs. Select **Add** next to **Application ID URI**. - - By default, the Application ID URI is set to `api://`. Accept the default by selecting **Save**. - - Under **Scopes defined by this API**, select **Add a scope**. - - Fill in the values as indicated: - - For **Scope name**, use **access_as_user**. - - For **Who can consent?**, select **Admins and users**. - - For **Admin consent display name**, type **Access Azure Search OpenAI Chat API**. - - For **Admin consent description**, type **Allows the app to access Azure Search OpenAI Chat API as the signed-in user.**. - - For **User consent display name**, type **Access Azure Search OpenAI Chat API**. - - For **User consent description**, type **Allow the app to access Azure Search OpenAI Chat API on your behalf**. - - Leave **State** set to **Enabled**. - - Select **Add scope** at the bottom to save the scope. -- (Optional) Enable group claims. Include which Microsoft Entra groups the user is part of as part of the login in the [optional claims](https://learn.microsoft.com/entra/identity-platform/optional-claims). The groups are used for [optional security filtering](https://learn.microsoft.com/azure/search/search-security-trimming-for-azure-search) in the search results. - - In the left hand menu, select **Token configuration** - - Under **Optional claims**, select **Add groups claim** - - Select which [group types](https://learn.microsoft.com/entra/identity/hybrid/connect/how-to-connect-fed-group-claims) to include in the claim. Note that a [overage claim](https://learn.microsoft.com/entra/identity-platform/access-token-claims-reference#groups-overage-claim) will be emitted if the user is part of too many groups. In this case, the API server will use the [Microsoft Graph](https://learn.microsoft.com/graph/api/user-list-memberof?view=graph-rest-*0&tabs=http) to list the groups the user is part of instead of relying on the groups in the claim. - - Select **Add** to save your changes - -#### Client App - -- Sign in to the [Azure portal](https://portal.azure.com/). -- Select the Microsoft Entra ID service. -- In the left hand menu, select **Application Registrations**. -- Select **New Registration**. - - In the **Name** section, enter a meaningful application name. This name will be displayed to users of the app, for example `Azure Search OpenAI Chat Web App`. - - Under **Supported account types**, select **Accounts in this organizational directory only**. - - Under `Redirect URI (optional)` section, select `Single-page application (SPA)` in the combo-box and enter the following redirect URI: - - If you are running the sample locally, use `http://localhost:50505/redirect`. - - If you are running the sample on Azure, use the endpoint provided by `azd up`: `https://.azurewebsites.net/redirect`. - - If you are running the sample from Github Codespaces, use the Codespaces endpoint: `https://-50505.app.github.dev/` -- Select **Register** to create the application -- In the app's registration screen, find the **Application (client) ID**. - - Run the following `azd` command to save this ID: `azd env set AZURE_CLIENT_APP_ID `. -- In the left hand menu, select **Authentication**. - - Under **Implicit grant and hybrid flows**, select **ID Tokens (used for implicit and hybrid flows)** - - Select **Save** -- In the left hand menu, select **API permissions**. You will add permission to access the **access_as_user** API on the server app. This permission is required for the [On Behalf Of Flow](https://learn.microsoft.com/entra/identity-platform/v2-oauth2-on-behalf-of-flow#protocol-diagram) to work. - - Select **Add a permission**, and then **My APIs**. - - In the list of applications, select your server application **Azure Search OpenAI Chat API** - - Ensure **Delegated permissions** is selected. - - In the **Select permissions** section, select the **access_as_user** permission - - Select **Add permissions**. - -#### Configure Server App Known Client Applications - -Consent from the user must be obtained for use of the client and server app. The client app can prompt the user for consent through a dialog when they log in. The server app has no ability to show a dialog for consent. Client apps can be [added to the list of known clients](https://learn.microsoft.com/entra/identity-platform/v2-oauth2-on-behalf-of-flow#gaining-consent-for-the-middle-tier-application) to access the server app, so a consent dialog is shown for the server app. - -- Navigate to the server app registration -- In the left hand menu, select **Manifest** -- Replace `"knownClientApplications": []` with `"knownClientApplications": [""]` -- Select **Save** - -#### Testing - -If you are running setup for the first time, ensure you have run `azd env set AZURE_ADLS_GEN2_STORAGE_ACCOUNT ` before running `azd up`. If you do not set this environment variable, your index will not be initialized with access control support when `prepdocs` is run for the first time. To manually enable access control in your index, use the [manual setup script](#azure-data-lake-storage-gen2-setup). - -Ensure you run `azd env set AZURE_USE_AUTHENTICATION` to enable the login UI once you have setup the two Microsoft Entra apps before you deploy or run the application. The login UI will not appear unless all [required environment variables](#environment-variables-reference) have been setup. - -In both the chat and ask a question modes, under **Developer settings** optional **Use oid security filter** and **Use groups security filter** checkboxes will appear. The oid (User ID) filter maps to the `oids` field in the search index and the groups (Group ID) filter maps to the `groups` field in the search index. If `AZURE_ENFORCE_ACCESS_CONTROL` has been set, then both the **Use oid security filter** and **Use groups security filter** options are always enabled and cannot be disabled. - -#### Programmatic Access with Authentication - -If you want to use the chat endpoint without the UI and still use authentication, you must disable [App Service built-in authentication](https://learn.microsoft.com/azure/app-service/overview-authentication-authorization) and use only the app's MSAL-based authentication flow. Ensure the `AZURE_DISABLE_APP_SERVICES_AUTHENTICATION` environment variable is set before deploying. - -Get an access token that can be used for calling the chat API using the following code: - -```python -from azure.identity import DefaultAzureCredential -import os - -token = DefaultAzureCredential().get_token(f"api://{os.environ['AZURE_SERVER_APP_ID']}/access_as_user", tenant_id=os.getenv('AZURE_AUTH_TENANT_ID', os.getenv('AZURE_TENANT_ID'))) - -print(token.token) -``` - -### Troubleshooting - -- If your primary tenant restricts the ability to create Entra applications, you'll need to use a separate tenant to create the Entra applications. You can create a new tenant by following [these instructions](https://learn.microsoft.com/entra/identity-platform/quickstart-create-new-tenant). Then run `azd env set AZURE_AUTH_TENANT_ID ` before running `azd up`. -- If any Entra apps need to be recreated, you can avoid redeploying the app by [changing the app settings in the portal](https://learn.microsoft.com/azure/app-service/configure-common?tabs=portal#configure-app-settings). Any of the [required environment variables](#environment-variables-reference) can be changed. Once the environment variables have been changed, restart the web app. -- It's possible a consent dialog will not appear when you log into the app for the first time. If this consent dialog doesn't appear, you will be unable to use the security filters because the API server app does not have permission to read your authorization information. A consent dialog can be forced to appear by adding `"prompt": "consent"` to the `loginRequest` property in [`authentication.py`](/app/backend/core/authentication.py) -- It's possible that your tenant admin has placed a restriction on consent to apps with [unverified publishers](https://learn.microsoft.com/entra/identity-platform/publisher-verification-overview). In this case, only admins may consent to the client and server apps, and normal user accounts are unable to use the login system until the admin consents on behalf of the entire organization. -- It's possible that your tenant admin requires [admin approval of all new apps](https://learn.microsoft.com/entra/identity/enterprise-apps/manage-consent-requests). Regardless of whether you select the delegated or admin permissions, the app will not work without tenant admin consent. See this guide for [granting consent to an app](https://learn.microsoft.com/entra/identity/enterprise-apps/grant-admin-consent?pivots=portal). - -## Adding data with document level access control - -The sample supports 2 main strategies for adding data with document level access control. - -- [Using the Add Documents API](#using-the-add-documents-api). Sample scripts are provided which use the Azure AI Search Service Add Documents API to directly manage access control information on _existing documents_ in the index. -- [Using prepdocs and Azure Data Lake Storage Gen 2](#azure-data-lake-storage-gen2-setup). Sample scripts are provided which set up an [Azure Data Lake Storage Gen 2](https://learn.microsoft.com/azure/storage/blobs/data-lake-storage-introduction) account, set the [access control information](https://learn.microsoft.com/azure/storage/blobs/data-lake-storage-access-control) on files and folders stored there, and ingest those documents into the search index with their access control information. - -### Using the Add Documents API - -Manually enable document level access control on a search index and manually set access control values using the [manageacl.py](/scripts/manageacl.py) script. - -Prior to running the script: - -- Run `azd up` or use `azd env set` to manually set the `AZURE_SEARCH_SERVICE` and `AZURE_SEARCH_INDEX` azd environment variables -- Activate the Python virtual environment for your shell session - -The script supports the following commands. All commands support `-v` for verbose logging. - -- `python ./scripts/manageacl.py --acl-action enable_acls`: Creates the required `oids` (User ID) and `groups` (Group IDs) [security filter](https://learn.microsoft.com/azure/search/search-security-trimming-for-azure-search) fields for document level access control on your index, as well as the `storageUrl` field for storing the Blob storage URL. Does nothing if these fields already exist. - - Example usage: - - ```shell - python ./scripts/manageacl.py -v --acl-action enable_acls - ``` - -- `python ./scripts/manageacl.py --acl-type [oids or groups]--acl-action view --url [https://url.pdf]`: Prints access control values associated with either User IDs or Group IDs for the document at the specified URL. - - Example to view all Group IDs: - - ```shell - python ./scripts/manageacl.py -v --acl-type groups --acl-action view --url https://st12345.blob.core.windows.net/content/Benefit_Options.pdf - ``` - -- `python ./scripts/manageacl.py --acl-type [oids or groups]--acl-action add --acl [ID of group or user] --url [https://url.pdf]`: Adds an access control value associated with either User IDs or Group IDs for the document at the specified URL. - - Example to add a Group ID: - - ```shell - python ./scripts/manageacl.py -v --acl-type groups --acl-action add --acl xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx --url https://st12345.blob.core.windows.net/content/Benefit_Options.pdf - ``` - -- `python ./scripts/manageacl.py --acl-type [oids or groups]--acl-action remove_all --url [https://url.pdf]`: Removes all access control values associated with either User IDs or Group IDs for a specific document. - - Example to remove all Group IDs: - - ```shell - python ./scripts/manageacl.py -v --acl-type groups --acl-action remove_all --url https://st12345.blob.core.windows.net/content/Benefit_Options.pdf - ``` - -- `python ./scripts/manageacl.py --url [https://url.pdf] --acl-type [oids or groups]--acl-action remove --acl [ID of group or user]`: Removes an access control value associated with either User IDs or Group IDs for a specific document. - - Example to remove a specific User ID: - - ```shell - python ./scripts/manageacl.py -v --acl-type oids --acl-action remove --acl xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx --url https://st12345.blob.core.windows.net/content/Benefit_Options.pdf - ``` - -### Azure Data Lake Storage Gen2 Setup - -[Azure Data Lake Storage Gen2](https://learn.microsoft.com/azure/storage/blobs/data-lake-storage-introduction) implements an [access control model](https://learn.microsoft.com/azure/storage/blobs/data-lake-storage-access-control) that can be used for document level access control. The [adlsgen2setup.py](/scripts/adlsgen2setup.py) script uploads the sample data included in the [data](./data) folder to a Data Lake Storage Gen2 storage account. The [Storage Blob Data Owner](https://learn.microsoft.com/azure/storage/blobs/data-lake-storage-access-control-model#role-based-access-control-azure-rbac) role is required to use the script. - -In order to use this script, an existing Data Lake Storage Gen2 storage account is required. Run `azd env set AZURE_ADLS_GEN2_STORAGE_ACCOUNT ` prior to running the script. - -Then run the script inside your Python environment: - -```shell -python /scripts/adlsgen2setup.py './data/*' --data-access-control './scripts/sampleacls.json' -v -``` - -The script performs the following steps: - -- Creates example [groups](https://learn.microsoft.com/entra/fundamentals/how-to-manage-groups) listed in the [sampleacls.json](/scripts/sampleacls.json) file. -- Creates a filesystem / container `gptkbcontainer` in the storage account. -- Creates the directories listed in the [sampleacls.json](/scripts/sampleacls.json) file. -- Uploads the sample PDFs referenced in the [sampleacls.json](/scripts/sampleacls.json) file into the appropriate directories. -- [Recursively sets Access Control Lists (ACLs)](https://learn.microsoft.com/azure/storage/blobs/data-lake-storage-acl-cli) using the information from the [sampleacls.json](/scripts/sampleacls.json) file. - -In order to use the sample access control, you need to join these groups in your Microsoft Entra tenant. - -Note that this optional script may not work in Codespaces if your administrator has applied a [Conditional Access policy](https://learn.microsoft.com/entra/identity/conditional-access/overview) to your tenant. - -#### Azure Data Lake Storage Gen2 Prep Docs - -Once a Data Lake Storage Gen2 storage account has been setup with sample data and access control lists, [prepdocs.py](/app/backend/prepdocs.py) can be used to automatically process PDFs in the storage account and store them with their [access control lists in the search index](https://learn.microsoft.com/azure/storage/blobs/data-lake-storage-access-control). - -To run this script with a Data Lake Storage Gen2 account, first set the following environment variables: - -- `AZURE_ADLS_GEN2_STORAGE_ACCOUNT`: Name of existing [Data Lake Storage Gen2 storage account](https://learn.microsoft.com/azure/storage/blobs/data-lake-storage-introduction). -- (Optional) `AZURE_ADLS_GEN2_FILESYSTEM`: Name of existing Data Lake Storage Gen2 filesystem / container in the storage account. If empty, `gptkbcontainer` is used. -- (Optional) `AZURE_ADLS_GEN2_FILESYSTEM_PATH`: Specific path in the Data Lake Storage Gen2 filesystem / container to process. Only PDFs contained in this path will be processed. - -Once the environment variables are set, run the script using the following command: `/scripts/prepdocs.ps1` or `/scripts/prepdocs.sh`. - -## Environment variables reference - -The following environment variables are used to setup the optional login and document level access control: - -- `AZURE_USE_AUTHENTICATION`: Enables Entra ID login and document level access control. Set to true before running `azd up`. -- `AZURE_ENFORCE_ACCESS_CONTROL`: Enforces Entra ID based login and document level access control on documents with access control assigned. Set to true before running `azd up`. If `AZURE_ENFORCE_ACCESS_CONTROL` is enabled and `AZURE_ENABLE_UNAUTHENTICATED_ACCESS` is not enabled, then authentication is required to use the app. -- `AZURE_ENABLE_GLOBAL_DOCUMENT_ACCESS`: Allows users to search on documents that have no access controls assigned -- `AZURE_ENABLE_UNAUTHENTICATED_ACCESS`: Allows unauthenticated users to access the chat app, even when `AZURE_ENFORCE_ACCESS_CONTROL` is enabled. `AZURE_ENABLE_GLOBAL_DOCUMENT_ACCESS` should be set to true to allow unauthenticated users to search on documents that have no access control assigned. Unauthenticated users cannot search on documents with access control assigned. -- `AZURE_DISABLE_APP_SERVICES_AUTHENTICATION`: Disables [use of built-in authentication for App Services](https://learn.microsoft.com/azure/app-service/overview-authentication-authorization). An authentication flow based on the MSAL SDKs is used instead. Useful when you want to provide programmatic access to the chat endpoints with authentication. -- `AZURE_SERVER_APP_ID`: (Required) Application ID of the Microsoft Entra app for the API server. -- `AZURE_SERVER_APP_SECRET`: [Client secret](https://learn.microsoft.com/entra/identity-platform/v2-oauth2-client-creds-grant-flow) used by the API server to authenticate using the Microsoft Entra server app. -- `AZURE_CLIENT_APP_ID`: Application ID of the Microsoft Entra app for the client UI. -- `AZURE_AUTH_TENANT_ID`: [Tenant ID](https://learn.microsoft.com/entra/fundamentals/how-to-find-tenant) associated with the Microsoft Entra tenant used for login and document level access control. Defaults to `AZURE_TENANT_ID` if not defined. -- `AZURE_ADLS_GEN2_STORAGE_ACCOUNT`: (Optional) Name of existing [Data Lake Storage Gen2 storage account](https://learn.microsoft.com/azure/storage/blobs/data-lake-storage-introduction) for storing sample data with [access control lists](https://learn.microsoft.com/azure/storage/blobs/data-lake-storage-access-control). Only used with the optional Data Lake Storage Gen2 [setup](#azure-data-lake-storage-gen2-setup) and [prep docs](#azure-data-lake-storage-gen2-prep-docs) scripts. -- `AZURE_ADLS_GEN2_FILESYSTEM`: (Optional) Name of existing [Data Lake Storage Gen2 filesystem](https://learn.microsoft.com/azure/storage/blobs/data-lake-storage-introduction) for storing sample data with [access control lists](https://learn.microsoft.com/azure/storage/blobs/data-lake-storage-access-control). Only used with the optional Data Lake Storage Gen2 [setup](#azure-data-lake-storage-gen2-setup) and [prep docs](#azure-data-lake-storage-gen2-prep-docs) scripts. -- `AZURE_ADLS_GEN2_FILESYSTEM_PATH`: (Optional) Name of existing path in a [Data Lake Storage Gen2 filesystem](https://learn.microsoft.com/azure/storage/blobs/data-lake-storage-introduction) for storing sample data with [access control lists](https://learn.microsoft.com/azure/storage/blobs/data-lake-storage-access-control). Only used with the optional Data Lake Storage Gen2 [prep docs](#azure-data-lake-storage-gen2-prep-docs) script. - -### Authentication behavior by environment - -This application uses an in-memory token cache. User sessions are only available in memory while the application is running. When the application server is restarted, all users will need to log-in again. - -The following table describes the impact of the `AZURE_USE_AUTHENTICATION` and `AZURE_ENFORCE_ACCESS_CONTROL` variables depending on the environment you are deploying the application in: - -| AZURE_USE_AUTHENTICATION | AZURE_ENFORCE_ACCESS_CONTROL | Environment | Default Behavior | -|-|-|-|-| -| True | False | App Services | Use integrated auth
Login page blocks access to app
User can opt-into access control in developer settings
Allows unrestricted access to sources | -| True | True | App Services | Use integrated auth
Login page blocks access to app
User must use access control | -| True | False | Local or Codespaces | Do not use integrated auth
Can use app without login
User can opt-into access control in developer settings
Allows unrestricted access to sources | -| True | True | Local or Codespaces | Do not use integrated auth
Cannot use app without login
Behavior is chat box is greyed out with default โ€œPlease login messageโ€
User must use login button to make chat box usable
User must use access control when logged in | -| False | False | All | No login or access control | -| False | True | All | Invalid setting | diff --git a/samples/private-endpoint/README.md b/samples/private-endpoint/README.md deleted file mode 100644 index baecedc03e..0000000000 --- a/samples/private-endpoint/README.md +++ /dev/null @@ -1,70 +0,0 @@ ---- -name: ChatGPT + Enterprise data + private endpoints -description: Configure access to a RAG chat app so that it's only accessible from private endpoints. -languages: -- python -- typescript -- bicep -- azdeveloper -products: -- azure-openai -- azure-cognitive-search -- azure-app-service -- azure -page_type: sample -urlFragment: azure-search-openai-demo-private-access ---- - - -# Deploying with private access - -The [azure-search-openai-demo](/) project can set up a full RAG chat app on Azure AI Search and OpenAI so that you can chat on custom data, like internal enterprise data or domain-specific knowledge sets. For full instructions on setting up the project, consult the [main README](/README.md), and then return here for detailed instructions on configuring private endpoints. - -## Before you begin - -Deploying with public access disabled adds additional cost to your deployment. Please see pricing for the following products: - -1. [Private Endpoints](https://azure.microsoft.com/pricing/details/private-link/) - 1. The exact number of private endpoints created depends on the [optional features](/docs/deploy_features.md) used. -1. [Private DNS Zones](https://azure.microsoft.com/pricing/details/dns/) -1. (Optional, but recommended)[Azure Virtual Machines](https://azure.microsoft.com/pricing/details/virtual-machines/windows/) -1. (Optional, but recommended)[Azure Bastion](https://azure.microsoft.com/pricing/details/azure-bastion/) - -## Environment variables controlling private access - -1. `AZURE_PUBLIC_NETWORK_ACCESS`: Controls the value of public network access on supported Azure resources. Valid values are 'Enabled' or 'Disabled'. - 1. When public network access is 'Enabled', Azure resources are open to the internet. - 1. When public network access is 'Disabled', Azure resources are only accessible over a virtual network. -1. `AZURE_USE_PRIVATE_ENDPOINT`: Controls deployment of [private endpoints](https://learn.microsoft.com/azure/private-link/private-endpoint-overview) which connect Azure resources to the virtual network. - 1. When set to 'true', ensures private endpoints are deployed for connectivity even when `AZURE_PUBLIC_NETWORK_ACCESS` is 'Disabled'. - 1. Note that private endpoints do not make the chat app accessible from the internet. Connections must be initiated from inside the virtual network. -1. `AZURE_PROVISION_VM`: Controls deployment of a [virtual machine](https://learn.microsoft.com/azure/virtual-machines/overview) and [Azure Bastion](https://learn.microsoft.com/azure/bastion/bastion-overview). Azure Bastion allows you to securely connect to the virtual machine, without being connected virtual network. Since the virtual machine is connected to the virtual network, you are able to access the chat app. - 1. You must set `AZURE_VM_USERNAME` and `AZURE_VM_PASSWORD` to provision the built-in administrator account with the virtual machine so you can log in through Azure Bastion. - 1. By default, a server version of Windows is used for the VM. If you need to [enroll your device in Microsoft Intune](https://learn.microsoft.com/mem/intune/user-help/enroll-windows-10-device), you should use a desktop version of Windows by setting the following environment variables: - - * `azd env set AZURE_VM_OS_PUBLISHER MicrosoftWindowsDesktop` - * `azd env set AZURE_VM_OS_OFFER Windows-11` - * `azd env set AZURE_VM_OS_VERSION win11-23h2-pro` - -## Recommended deployment strategy for private access - -1. Deploy the app with private endpoints enabled and public access enabled. - - ```shell - azd env set AZURE_USE_PRIVATE_ENDPOINT true - azd env set AZURE_PUBLIC_NETWORK_ACCESS Enabled - azd up - ``` - -1. Validate that you can connect to the chat app and it's working as expected from the internet. -1. Re-provision the app with public access disabled. - - ```shell - azd env set AZURE_PUBLIC_NETWORK_ACCESS Disabled - azd env set AZURE_PROVISION_VM true # Optional but recommended - azd env set AZURE_VM_USERNAME myadminusername # https://learn.microsoft.com/azure/virtual-machines/windows/faq#what-are-the-username-requirements-when-creating-a-vm- - azd env set AZURE_VM_PASSWORD mypassword # https://learn.microsoft.com/azure/virtual-machines/windows/faq#what-are-the-password-requirements-when-creating-a-vm- - azd provision - ``` - -1. Log into your new VM using [Azure Bastion](https://learn.microsoft.com/azure/bastion/tutorial-create-host-portal#connect). Validate the chat app is accessible from the virtual machine using a web browser.