diff --git a/README.md b/README.md index 3da325ea8..435356ecc 100644 --- a/README.md +++ b/README.md @@ -63,137 +63,148 @@ This system is intended for developing and deploying custom AI solutions for spe ![image](./documentation/images/readme/macae-architecture.png) +

+
+QUICK DEPLOY +

+### Prerequisites -### **How to install/deploy** +To deploy this solution accelerator, ensure you have access to an [Azure subscription](https://azure.microsoft.com/free/) with the necessary permissions to create **resource groups and resources**. Follow the steps in [Azure Account Set Up](./docs/AzureAccountSetUp.md) -This guide provides step-by-step instructions for deploying your application using Azure Container Registry (ACR) and Azure Container Apps. +Check the [Azure Products by Region](https://azure.microsoft.com/en-us/explore/global-infrastructure/products-by-region/table) page and select a **region** where the following services are available: -There are several ways to deploy the solution. You can deploy to run in Azure in one click, or manually, or you can deploy locally. +- Azure OpenAI Service +- Azure AI Search +- [Azure Semantic Search](./docs/AzureSemanticSearchRegion.md) -## Quick Deploy +### ⚠️ Important: Check Azure OpenAI Quota Availability -

+➡️ To ensure sufficient quota is available in your subscription, please follow **[Quota check instructions guide](./documentation/quota_check.md)** before you deploy the solution. -[![Deploy to Azure](https://aka.ms/deploytoazurebutton)](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2Fmicrosoft%2FMulti-Agent-Custom-Automation-Engine-Solution-Accelerator%2Frefs%2Fheads%2Fmain%2Fdeploy%2Fmacae-continer-oc.json) +| [![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/microsoft/Multi-Agent-Custom-Automation-Engine-Solution-Accelerator) | [![Open in Dev Containers](https://img.shields.io/static/v1?style=for-the-badge&label=Dev%20Containers&message=Open&color=blue&logo=visualstudiocode)](https://vscode.dev/redirect?url=vscode://ms-vscode-remote.remote-containers/cloneInVolume?url=https://github.com/microsoft/Multi-Agent-Custom-Automation-Engine-Solution-Accelerator) | +|---|---| -When Deployment is complete, follow steps in [Set Up Authentication in Azure App Service](./documentation/azure_app_service_auth_setup.md) to add app authentication to your web app running on Azure App Service -## Local Deployment -To run the solution site and API backend only locally for development and debugging purposes, See the [local deployment guide](./documentation/LocalDeployment.md). + -## Manual Azure Deployment -Manual Deployment differs from the ‘Quick Deploy’ option in that it will install an Azure Container Registry (ACR) service, and relies on the installer to build and push the necessary containers to this ACR. This allows you to build and push your own code changes and provides a sample solution you can customize based on your requirements. +### Configurable Deployment Settings + +When you start the deployment, most parameters will have **default values**, but you can update the below settings by following the steps [here](./documentation/CustomizingAzdParameters.md): + +| **Setting** | **Description** | **Default value** | +|------------|----------------| ------------| +| **Environment Name** | A **3-20 character alphanumeric value** used to generate a unique ID to prefix the resources. | macaetemplate | +| **Cosmos Location** | A **less busy** region for **CosmosDB**, useful in case of availability constraints. | eastus2 | +| **Deployment Type** | Select from a drop-down list. | Global Standard | +| **GPT Model** | Choose from **gpt-4o** | gpt-4o | +| **GPT Model Deployment Capacity** | Configure capacity for **GPT models**. | 50k | -### Prerequisites -- Current Azure CLI installed - You can update to the latest version using ```az upgrade``` -- Azure account with appropriate permissions -- Docker installed +### [Optional] Quota Recommendations +By default, the **Gpt-4o model capacity** in deployment is set to **50k tokens**, so we recommend +> **For Global Standard | GPT-4o - the capacity to at least 50k tokens for optimal performance.** -### Deploy the Azure Services -All of the necessary Azure services can be deployed using the /deploy/macae.bicep script. This script will require the following parameters: +To adjust quota settings if required, follow these [steps](./documentation/AzureGPTQuotaSettings.md) -``` -az login -az account set --subscription -az group create --name --location -``` -To deploy the script you can use the Azure CLI. -``` -az deployment group create \ - --resource-group \ - --template-file \ - --name -``` +### Deployment Options +Pick from the options below to see step-by-step instructions for: GitHub Codespaces, VS Code Dev Containers, Local Environments, and Bicep deployments. -Note: if you are using windows with PowerShell, the continuation character (currently ‘\’) should change to the tick mark (‘`’). +
+ Deploy in GitHub Codespaces -The template will require you fill in locations for Cosmos and OpenAI services. This is to avoid the possibility of regional quota errors for either of these resources. +### GitHub Codespaces -### Create the Containers -#### Get admin credentials from ACR +You can run this solution using GitHub Codespaces. The button will open a web-based VS Code instance in your browser: -Retrieve the admin credentials for your Azure Container Registry (ACR): +1. Open the solution accelerator (this may take several minutes): -```sh -az acr credential show \ ---name \ ---resource-group -``` + [![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/microsoft/Multi-Agent-Custom-Automation-Engine-Solution-Accelerator) +2. Accept the default values on the create Codespaces page +3. Open a terminal window if it is not already open +4. Continue with the [deploying steps](#deploying) -#### Login to ACR +
-Login to your Azure Container Registry: +
+ Deploy in VS Code -```sh -az acr login --name -``` + ### VS Code Dev Containers -#### Build and push the image +You can run this solution in VS Code Dev Containers, which will open the project in your local VS Code using the [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers): -Build the frontend and backend Docker images and push them to your Azure Container Registry. Run the following from the src/backend and the src/frontend directory contexts: +1. Start Docker Desktop (install it if not already installed) +2. Open the project: -```sh -az acr build \ ---registry \ ---resource-group \ ---image . -``` + [![Open in Dev Containers](https://img.shields.io/static/v1?style=for-the-badge&label=Dev%20Containers&message=Open&color=blue&logo=visualstudiocode)](https://vscode.dev/redirect?url=vscode://ms-vscode-remote.remote-containers/cloneInVolume?url=https://github.com/microsoft/Multi-Agent-Custom-Automation-Engine-Solution-Accelerator) + +3. In the VS Code window that opens, once the project files show up (this may take several minutes), open a terminal window. +4. Continue with the [deploying steps](#deploying) + +
+ +
+ Deploy in your local environment + +### Local environment + +To run the solution site and API backend only locally for development and debugging purposes, See the [local deployment guide](./documentation/LocalDeployment.md). -### Add images to the Container APP and Web App services +
-To add your newly created backend image: -- Navigate to the Container App Service in the Azure portal -- Click on Application/Containers in the left pane -- Click on the "Edit and deploy" button in the upper left of the containers pane -- In the "Create and deploy new revision" page, click on your container image 'backend'. This will give you the option of reconfiguring the container image, and also has an Environment variables tab -- Change the properties page to - - point to your Azure Container registry with a private image type and your image name (e.g. backendmacae:latest) - - under "Authentication type" select "Managed Identity" and choose the 'mace-containerapp-pull'... identity setup in the bicep template -- In the environment variables section add the following (each with a 'Manual entry' source): +### Manual Azure Deployment +Manual Deployment differs from the ‘Quick Deploy’ option in that it will install an Azure Container Registry (ACR) service, and relies on the installer to build and push the necessary containers to this ACR. This allows you to build and push your own code changes and provides a sample solution you can customize based on your requirements. - name: 'COSMOSDB_ENDPOINT' - value: \ +See the [local deployment guide](./documentation/ManualAzureDeployment.md). - name: 'COSMOSDB_DATABASE' - value: 'autogen' - Note: To change the default, you will need to create the database in Cosmos - - name: 'COSMOSDB_CONTAINER' - value: 'memory' - name: 'AZURE_OPENAI_ENDPOINT' - value: +### Deploying - name: 'AZURE_OPENAI_DEPLOYMENT_NAME' - value: 'gpt-4o' +Once you've opened the project in [Codespaces](#github-codespaces) or in [Dev Containers](#vs-code-dev-containers) or [locally](#local-environment), you can deploy it to Azure following the following steps. - name: 'AZURE_OPENAI_API_VERSION' - value: '2024-08-01-preview' - Note: Version should be updated based on latest available +To change the azd parameters from the default values, follow the steps [here](./documentation/CustomizingAzdParameters.md). - name: 'FRONTEND_SITE_NAME' - value: 'https://.azurewebsites.net' - name: 'APPLICATIONINSIGHTS_CONNECTION_STRING' - value: +1. Login to Azure: -- Click 'Save' and deploy your new revision + ```shell + azd auth login + ``` -To add the new container to your website run the following: + #### To authenticate with Azure Developer CLI (`azd`), use the following command with your **Tenant ID**: -``` -az webapp config container set --resource-group \ ---name \ ---container-image-name \ ---container-registry-url -``` + ```sh + azd auth login --tenant-id + ``` +2. Provision and deploy all the resources: -### Add the Entra identity provider to the Azure Web App -To add the identity provider, please follow the steps outlined in [Set Up Authentication in Azure App Service](./documentation/azure_app_service_auth_setup.md) + ```shell + azd up + ``` + +3. Provide an `azd` environment name (like "macaeapp") +4. Select a subscription from your Azure account, and select a location which has quota for all the resources. + * This deployment will take *7-10 minutes* to provision the resources in your account and set up the solution with sample data. + * If you get an error or timeout with deployment, changing the location can help, as there may be availability constraints for the resources. +5. Open the [Azure Portal](https://portal.azure.com/), go to the deployed resource group, find the App Service and get the app URL from `Default domain`. + +6. You can now delete the resources by running `azd down`, if you are done trying out the application. + + +

+Additional Steps +

+ +1. **Add App Authentication** + + Follow steps in [App Authentication](./documentation/azure_app_service_auth_setup.md) to configure authenitcation in app service. + + Note: Authentication changes can take up to 10 minutes + +2. **Deleting Resources After a Failed Deployment** + + Follow steps in [Delete Resource Group](./documentation/DeleteResourceGroup.md) If your deployment fails and you need to clean up the resources. ### Run locally and debug @@ -209,35 +220,64 @@ Note that you can configure the name of the Cosmos database in the configuration If you are using VSCode, you can use the debug configuration shown in the [local deployment guide](./documentation/LocalDeployment.md). -## Supporting documentation +## Sample Questions + +To help you get started, here are some [Sample Questions](./documentation/SampleQuestions.md) you can follow once your application is up and running. + +

+
+Responsible AI Transparency FAQ +

+Please refer to [Transparency FAQ](./documentation/TRANSPARENCY_FAQ.md) for responsible AI transparency details of this solution accelerator. -### +

+Supporting documentation +

### How to customize This solution is designed to be easily customizable. You can modify the front end site, or even build your own front end and attach to the backend API. You can further customize the backend by adding your own agents with their own specific capabilities. Deeper technical information to aid in this customization can be found in this [document](./documentation/CustomizeSolution.md). -### Additional resources +### Costs + +Pricing varies per region and usage, so it isn't possible to predict exact costs for your usage. +The majority of the Azure resources used in this infrastructure are on usage-based pricing tiers. +However, Azure Container Registry has a fixed cost per registry per day. + +You can try the [Azure pricing calculator](https://azure.microsoft.com/en-us/pricing/calculator) for the resources: + +* Azure AI Foundry: Free tier. [Pricing](https://azure.microsoft.com/pricing/details/ai-studio/) +* Azure AI Services: S0 tier, defaults to gpt-4o. Pricing is based on token count. [Pricing](https://azure.microsoft.com/pricing/details/cognitive-services/) +* Azure Container App: Consumption tier with 0.5 CPU, 1GiB memory/storage. Pricing is based on resource allocation, and each month allows for a certain amount of free usage. [Pricing](https://azure.microsoft.com/pricing/details/container-apps/) +* Azure Container Registry: Basic tier. [Pricing](https://azure.microsoft.com/pricing/details/container-registry/) +* Azure Cosmos DB: [Pricing](https://azure.microsoft.com/en-us/pricing/details/cosmos-db/autoscale-provisioned/) + + +⚠️ To avoid unnecessary costs, remember to take down your app if it's no longer in use, +either by deleting the resource group in the Portal or running `azd down`. + +### Security guidelines + +This template uses Azure Key Vault to store all connections to communicate between resources. + +This template also uses [Managed Identity](https://learn.microsoft.com/entra/identity/managed-identities-azure-resources/overview) for local development and deployment. + +To ensure continued best practices in your own repository, we recommend that anyone creating solutions based on our templates ensure that the [Github secret scanning](https://docs.github.com/code-security/secret-scanning/about-secret-scanning) setting is enabled. + +You may want to consider additional security measures, such as: + +* Enabling Microsoft Defender for Cloud to [secure your Azure resources](https://learn.microsoft.com/azure/security-center/defender-for-cloud). +* Protecting the Azure Container Apps instance with a [firewall](https://learn.microsoft.com/azure/container-apps/waf-app-gateway) and/or [Virtual Network](https://learn.microsoft.com/azure/container-apps/networking?tabs=workload-profiles-env%2Cazure-cli). + +### Additional Resources - [Python FastAPI documentation](https://fastapi.tiangolo.com/learn/) - [AutoGen Framework Documentation](https://microsoft.github.io/autogen/dev/user-guide/core-user-guide/index.html) - [Azure Container App documentation](https://learn.microsoft.com/en-us/azure/azure-functions/functions-how-to-custom-container?tabs=core-tools%2Cacr%2Cazure-cli2%2Cazure-cli&pivots=container-apps) - [Azure OpenAI Service - Documentation, quickstarts, API reference - Azure AI services | Microsoft Learn](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/use-your-data) - [Azure Cosmos DB documentation](https://learn.microsoft.com/en-us/azure/cosmos-db/) - -

-
-Customer truth -

-Customer stories coming soon. - -
-
-
- ---- ## Disclaimers @@ -250,3 +290,7 @@ You acknowledge that the Software and Microsoft Products and Services (1) are no You acknowledge the Software is not subject to SOC 1 and SOC 2 compliance audits. No Microsoft technology, nor any of its component technologies, including the Software, is intended or made available as a substitute for the professional advice, opinion, or judgement of a certified financial services professional. Do not use the Software to replace, substitute, or provide professional financial advice or judgment. BY ACCESSING OR USING THE SOFTWARE, YOU ACKNOWLEDGE THAT THE SOFTWARE IS NOT DESIGNED OR INTENDED TO SUPPORT ANY USE IN WHICH A SERVICE INTERRUPTION, DEFECT, ERROR, OR OTHER FAILURE OF THE SOFTWARE COULD RESULT IN THE DEATH OR SERIOUS BODILY INJURY OF ANY PERSON OR IN PHYSICAL OR ENVIRONMENTAL DAMAGE (COLLECTIVELY, “HIGH-RISK USE”), AND THAT YOU WILL ENSURE THAT, IN THE EVENT OF ANY INTERRUPTION, DEFECT, ERROR, OR OTHER FAILURE OF THE SOFTWARE, THE SAFETY OF PEOPLE, PROPERTY, AND THE ENVIRONMENT ARE NOT REDUCED BELOW A LEVEL THAT IS REASONABLY, APPROPRIATE, AND LEGAL, WHETHER IN GENERAL OR IN A SPECIFIC INDUSTRY. BY ACCESSING THE SOFTWARE, YOU FURTHER ACKNOWLEDGE THAT YOUR HIGH-RISK USE OF THE SOFTWARE IS AT YOUR OWN RISK. + +--- + + diff --git a/documentation/AzureGPTQuotaSettings.md b/documentation/AzureGPTQuotaSettings.md new file mode 100644 index 000000000..a8f7d6c5b --- /dev/null +++ b/documentation/AzureGPTQuotaSettings.md @@ -0,0 +1,10 @@ +## How to Check & Update Quota + +1. **Navigate** to the [Azure AI Foundry portal](https://ai.azure.com/). +2. **Select** the AI Project associated with this accelerator. +3. **Go to** the `Management Center` from the bottom-left navigation menu. +4. Select `Quota` + - Click on the `GlobalStandard` dropdown. + - Select the required **GPT model** (`GPT-4o`) + - Choose the **region** where the deployment is hosted. +5. Request More Quota or delete any unused model deployments as needed. diff --git a/documentation/CustomizingAzdParameters.md b/documentation/CustomizingAzdParameters.md new file mode 100644 index 000000000..0a842ab50 --- /dev/null +++ b/documentation/CustomizingAzdParameters.md @@ -0,0 +1,43 @@ +## [Optional]: Customizing resource names + +By default this template will use the environment name as the prefix to prevent naming collisions within Azure. The parameters below show the default values. You only need to run the statements below if you need to change the values. + + +> To override any of the parameters, run `azd env set ` before running `azd up`. On the first azd command, it will prompt you for the environment name. Be sure to choose 3-20 charaters alphanumeric unique name. + + +Change the Secondary Location (example: eastus2, westus2, etc.) + +```shell +azd env set AZURE_ENV_COSMOS_LOCATION eastus2 +``` + +Change the Model Deployment Type (allowed values: Standard, GlobalStandard) + +```shell +azd env set AZURE_ENV_MODEL_DEPLOYMENT_TYPE Standard +``` + +Set the Model Name (allowed values: gpt-4, gpt-4o) + +```shell +azd env set AZURE_ENV_MODEL_NAME gpt-4o +``` + +Change the Model Capacity (choose a number based on available GPT model capacity in your subscription) + +```shell +azd env set AZURE_ENV_MODEL_CAPACITY 30 +``` + +Change the Embedding Model + +```shell +azd env set AZURE_ENV_EMBEDDING_MODEL_NAME text-embedding-ada-002 +``` + +Change the Embedding Deployment Capacity (choose a number based on available embedding model capacity in your subscription) + +```shell +azd env set AZURE_ENV_EMBEDDING_MODEL_CAPACITY 80 +``` diff --git a/documentation/DeleteResourceGroup.md b/documentation/DeleteResourceGroup.md new file mode 100644 index 000000000..aebe0adb6 --- /dev/null +++ b/documentation/DeleteResourceGroup.md @@ -0,0 +1,53 @@ +# Deleting Resources After a Failed Deployment in Azure Portal + +If your deployment fails and you need to clean up the resources manually, follow these steps in the Azure Portal. + +--- + +## **1. Navigate to the Azure Portal** +1. Open [Azure Portal](https://portal.azure.com/). +2. Sign in with your Azure account. + +--- + +## **2. Find the Resource Group** +1. In the search bar at the top, type **"Resource groups"** and select it. +2. Locate the **resource group** associated with the failed deployment. + +![Resource Groups](images/resourcegroup.png) + +![Resource Groups](images/resource-groups.png) + +--- + +## **3. Delete the Resource Group** +1. Click on the **resource group name** to open it. +2. Click the **Delete resource group** button at the top. + +![Delete Resource Group](images/DeleteRG.png) + +3. Type the resource group name in the confirmation box and click **Delete**. + +📌 **Note:** Deleting a resource group will remove all resources inside it. + +--- + +## **4. Delete Individual Resources (If Needed)** +If you don’t want to delete the entire resource group, follow these steps: + +1. Open **Azure Portal** and go to the **Resource groups** section. +2. Click on the specific **resource group**. +3. Select the **resource** you want to delete (e.g., App Service, Storage Account). +4. Click **Delete** at the top. + +![Delete Individual Resource](images/deleteservices.png) + +--- + +## **5. Verify Deletion** +- After a few minutes, refresh the **Resource groups** page. +- Ensure the deleted resource or group no longer appears. + +📌 **Tip:** If a resource fails to delete, check if it's **locked** under the **Locks** section and remove the lock. + + diff --git a/documentation/ManualAzureDeployment.md b/documentation/ManualAzureDeployment.md new file mode 100644 index 000000000..d59b2d591 --- /dev/null +++ b/documentation/ManualAzureDeployment.md @@ -0,0 +1,109 @@ +# Manual Azure Deployment +Manual Deployment differs from the ‘Quick Deploy’ option in that it will install an Azure Container Registry (ACR) service, and relies on the installer to build and push the necessary containers to this ACR. This allows you to build and push your own code changes and provides a sample solution you can customize based on your requirements. + +## Prerequisites + +- Current Azure CLI installed + You can update to the latest version using ```az upgrade``` +- Azure account with appropriate permissions +- Docker installed + +## Deploy the Azure Services +All of the necessary Azure services can be deployed using the /deploy/macae.bicep script. This script will require the following parameters: + +``` +az login +az account set --subscription +az group create --name --location +``` +To deploy the script you can use the Azure CLI. +``` +az deployment group create \ + --resource-group \ + --template-file \ + --name +``` + +Note: if you are using windows with PowerShell, the continuation character (currently ‘\’) should change to the tick mark (‘`’). + +The template will require you fill in locations for Cosmos and OpenAI services. This is to avoid the possibility of regional quota errors for either of these resources. + +## Create the Containers +- Get admin credentials from ACR + +Retrieve the admin credentials for your Azure Container Registry (ACR): + +```sh +az acr credential show \ +--name \ +--resource-group +``` + +## Login to ACR + +Login to your Azure Container Registry: + +```sh +az acr login --name +``` + +## Build and push the image + +Build the frontend and backend Docker images and push them to your Azure Container Registry. Run the following from the src/backend and the src/frontend directory contexts: + +```sh +az acr build \ +--registry \ +--resource-group \ +--image . +``` + +## Add images to the Container APP and Web App services + +To add your newly created backend image: +- Navigate to the Container App Service in the Azure portal +- Click on Application/Containers in the left pane +- Click on the "Edit and deploy" button in the upper left of the containers pane +- In the "Create and deploy new revision" page, click on your container image 'backend'. This will give you the option of reconfiguring the container image, and also has an Environment variables tab +- Change the properties page to + - point to your Azure Container registry with a private image type and your image name (e.g. backendmacae:latest) + - under "Authentication type" select "Managed Identity" and choose the 'mace-containerapp-pull'... identity setup in the bicep template +- In the environment variables section add the following (each with a 'Manual entry' source): + + name: 'COSMOSDB_ENDPOINT' + value: \ + + name: 'COSMOSDB_DATABASE' + value: 'autogen' + Note: To change the default, you will need to create the database in Cosmos + + name: 'COSMOSDB_CONTAINER' + value: 'memory' + + name: 'AZURE_OPENAI_ENDPOINT' + value: + + name: 'AZURE_OPENAI_DEPLOYMENT_NAME' + value: 'gpt-4o' + + name: 'AZURE_OPENAI_API_VERSION' + value: '2024-08-01-preview' + Note: Version should be updated based on latest available + + name: 'FRONTEND_SITE_NAME' + value: 'https://.azurewebsites.net' + + name: 'APPLICATIONINSIGHTS_CONNECTION_STRING' + value: + +- Click 'Save' and deploy your new revision + +To add the new container to your website run the following: + +``` +az webapp config container set --resource-group \ +--name \ +--container-image-name \ +--container-registry-url +``` + diff --git a/documentation/SampleQuestions.md b/documentation/SampleQuestions.md new file mode 100644 index 000000000..770a994b7 --- /dev/null +++ b/documentation/SampleQuestions.md @@ -0,0 +1,25 @@ +# Sample Questions + +To help you get started, here are some **Sample Prompts** you can ask in the app: + +1. Run each of the following sample prompts and verify that a plan is generated: + - Launch a new marketing campaign + - Procure new office equipment + - Initiate a new product launch + +2. Run the **Onboard employee** prompt: + - Remove the employee name from the prompt to test how the solution handles missing information. + - The solution should ask for the missing detail before proceeding. + +3. Try running known **RAI test prompts** to confirm safeguard behavior: + - You should see a toast message indicating that a plan could not be generated due to policy restrictions. + + +**Home Page** +![HomePage](images/MACAE-GP1.png) + +**Task Page** +![GeneratedPlan](images/MACAE-GP2.png) + + +_This structured approach helps ensure the system handles prompts gracefully, verifies plan generation flows, and confirms RAI protections are working as intended._ diff --git a/documentation/TRANSPARENCY_FAQ.md b/documentation/TRANSPARENCY_FAQ.md new file mode 100644 index 000000000..ace333547 --- /dev/null +++ b/documentation/TRANSPARENCY_FAQ.md @@ -0,0 +1,17 @@ +## Document Generation Solution Accelerator: Responsible AI FAQ +- ### What is Build your own copilot - Generic Solution Accelerator? + This solution accelerator is an open-source GitHub Repository to help create AI assistants using Azure OpenAI Service and Azure AI Search. This can be used by anyone looking for reusable architecture and code snippets to build AI assistants with their own enterprise data. The repository showcases a generic scenario of a user who wants to generate a document template based on a sample set of data. + +- ### What can Document Generation Solution Accelerator do? + The sample solution included focuses on a generic use case - chat with your own data, generate a document template using your own data, and exporting the document in a docx format. The sample data is sourced from generic AI-generated promissory notes. The documents are intended for use as sample data only. The sample solution takes user input in text format and returns LLM responses in text format up to 800 tokens. It uses prompt flow to search data from AI search vector store, summarize the retrieved documents with Azure OpenAI. + +- ### What is/are Document Generation Solution Accelerator’s intended use(s)? + This repository is to be used only as a solution accelerator following the open-source license terms listed in the GitHub repository. The example scenario’s intended purpose is to help users generate a document template to perform their work more efficiently. + +- ### How was Document Generation Solution Accelerator evaluated? What metrics are used to measure performance? + We have used AI Foundry Prompt flow evaluation SDK to test for harmful content, groundedness, and potential security risks. + +- ### What are the limitations of Document Generation Solution Accelerator? How can users minimize the impact of Document Generation Solution Accelerator’s limitations when using the system? + This solution accelerator can only be used as a sample to accelerate the creation of AI assistants. The repository showcases a sample scenario of a user generating a document template. Users should review the system prompts provided and update as per their organizational guidance. Users should run their own evaluation flow either using the guidance provided in the GitHub repository or their choice of evaluation methods. AI-generated content may be inaccurate and should be manually reviewed. Currently, the sample repo is available in English only. +- ### What operational factors and settings allow for effective and responsible use of Document Generation Solution Accelerator? + Users can try different values for some parameters like system prompt, temperature, max tokens etc. shared as configurable environment variables while running run evaluations for AI assistants. Please note that these parameters are only provided as guidance to start the configuration but not as a complete available list to adjust the system behavior. Please always refer to the latest product documentation for these details or reach out to your Microsoft account team if you need assistance. diff --git a/documentation/images/MACAE-GP1.png b/documentation/images/MACAE-GP1.png new file mode 100644 index 000000000..4b2386f85 Binary files /dev/null and b/documentation/images/MACAE-GP1.png differ diff --git a/documentation/images/MACAE-GP2.png b/documentation/images/MACAE-GP2.png new file mode 100644 index 000000000..1e1a59a9c Binary files /dev/null and b/documentation/images/MACAE-GP2.png differ diff --git a/documentation/images/git_bash.png b/documentation/images/git_bash.png new file mode 100644 index 000000000..0e9f53a12 Binary files /dev/null and b/documentation/images/git_bash.png differ diff --git a/documentation/images/quota-check-output.png b/documentation/images/quota-check-output.png new file mode 100644 index 000000000..9c80e3298 Binary files /dev/null and b/documentation/images/quota-check-output.png differ diff --git a/documentation/quota_check.md b/documentation/quota_check.md new file mode 100644 index 000000000..a59edf231 --- /dev/null +++ b/documentation/quota_check.md @@ -0,0 +1,100 @@ +## Check Quota Availability Before Deployment + +Before deploying the accelerator, **ensure sufficient quota availability** for the required model. +> **For Global Standard | GPT-4o - the capacity to at least 50k tokens for optimal performance.** + +### Login if you have not done so already +``` +azd auth login +``` + + +### 📌 Default Models & Capacities: +``` +gpt-4o:50 +``` +### 📌 Default Regions: +``` +eastus, uksouth, eastus2, northcentralus, swedencentral, westus, westus2, southcentralus, canadacentral +``` +### Usage Scenarios: +- No parameters passed → Default models and capacities will be checked in default regions. +- Only model(s) provided → The script will check for those models in the default regions. +- Only region(s) provided → The script will check default models in the specified regions. +- Both models and regions provided → The script will check those models in the specified regions. +- `--verbose` passed → Enables detailed logging output for debugging and traceability. + +### **Input Formats** +> Use the --models, --regions, and --verbose options for parameter handling: + +✔️ Run without parameters to check default models & regions without verbose logging: + ``` + ./quota_check_params.sh + ``` +✔️ Enable verbose logging: + ``` + ./quota_check_params.sh --verbose + ``` +✔️ Check specific model(s) in default regions: + ``` + ./quota_check_params.sh --models gpt-4o:50 + ``` +✔️ Check default models in specific region(s): + ``` +./quota_check_params.sh --regions eastus,westus + ``` +✔️ Passing Both models and regions: + ``` + ./quota_check_params.sh --models gpt-4o:50 --regions eastus,westus2 + ``` +✔️ All parameters combined: + ``` + ./quota_check_params.sh --models gpt-4o:50 --regions eastus,westus --verbose + ``` + +### **Sample Output** +The final table lists regions with available quota. You can select any of these regions for deployment. + +![quota-check-ouput](images/quota-check-output.png) + +--- +### **If using Azure Portal and Cloud Shell** + +1. Navigate to the [Azure Portal](https://portal.azure.com). +2. Click on **Azure Cloud Shell** in the top right navigation menu. +3. Run the appropriate command based on your requirement: + + **To check quota for the deployment** + + ```sh + curl -L -o quota_check_params.sh "https://raw.githubusercontent.com/microsoft/document-generation-solution-accelerator/main/scripts/quota_check_params.sh" + chmod +x quota_check_params.sh + ./quota_check_params.sh + ``` + - Refer to [Input Formats](#input-formats) for detailed commands. + +### **If using VS Code or Codespaces** +1. Open the terminal in VS Code or Codespaces. +2. If you're using VS Code, click the dropdown on the right side of the terminal window, and select `Git Bash`. + ![git_bash](images/git_bash.png) +3. Navigate to the `scripts` folder where the script files are located and make the script as executable: + ```sh + cd scripts + chmod +x quota_check_params.sh + ``` +4. Run the appropriate script based on your requirement: + + **To check quota for the deployment** + + ```sh + ./quota_check_params.sh + ``` + - Refer to [Input Formats](#input-formats) for detailed commands. + +5. If you see the error `_bash: az: command not found_`, install Azure CLI: + + ```sh + curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash + az login + ``` +6. Rerun the script after installing Azure CLI. diff --git a/infra/scripts/checkquota.sh b/infra/scripts/checkquota.sh deleted file mode 100644 index afc340378..000000000 --- a/infra/scripts/checkquota.sh +++ /dev/null @@ -1,95 +0,0 @@ -#!/bin/bash - -# List of Azure regions to check for quota (update as needed) -IFS=', ' read -ra REGIONS <<< "$AZURE_REGIONS" - -SUBSCRIPTION_ID="${AZURE_SUBSCRIPTION_ID}" -GPT_MIN_CAPACITY="${GPT_MIN_CAPACITY}" -AZURE_CLIENT_ID="${AZURE_CLIENT_ID}" -AZURE_TENANT_ID="${AZURE_TENANT_ID}" -AZURE_CLIENT_SECRET="${AZURE_CLIENT_SECRET}" - -# Authenticate using Managed Identity -echo "Authentication using Managed Identity..." -if ! az login --service-principal -u "$AZURE_CLIENT_ID" -p "$AZURE_CLIENT_SECRET" --tenant "$AZURE_TENANT_ID"; then - echo "❌ Error: Failed to login using Managed Identity." - exit 1 -fi - -echo "🔄 Validating required environment variables..." -if [[ -z "$SUBSCRIPTION_ID" || -z "$GPT_MIN_CAPACITY" || -z "$REGIONS" ]]; then - echo "❌ ERROR: Missing required environment variables." - exit 1 -fi - -echo "🔄 Setting Azure subscription..." -if ! az account set --subscription "$SUBSCRIPTION_ID"; then - echo "❌ ERROR: Invalid subscription ID or insufficient permissions." - exit 1 -fi -echo "✅ Azure subscription set successfully." - -# Define models and their minimum required capacities -declare -A MIN_CAPACITY=( - ["OpenAI.Standard.gpt-4o"]=$GPT_MIN_CAPACITY -) - -VALID_REGION="" -for REGION in "${REGIONS[@]}"; do - echo "----------------------------------------" - echo "🔍 Checking region: $REGION" - - QUOTA_INFO=$(az cognitiveservices usage list --location "$REGION" --output json) - if [ -z "$QUOTA_INFO" ]; then - echo "⚠️ WARNING: Failed to retrieve quota for region $REGION. Skipping." - continue - fi - - INSUFFICIENT_QUOTA=false - for MODEL in "${!MIN_CAPACITY[@]}"; do - MODEL_INFO=$(echo "$QUOTA_INFO" | awk -v model="\"value\": \"$MODEL\"" ' - BEGIN { RS="},"; FS="," } - $0 ~ model { print $0 } - ') - - if [ -z "$MODEL_INFO" ]; then - echo "⚠️ WARNING: No quota information found for model: $MODEL in $REGION. Skipping." - continue - fi - - CURRENT_VALUE=$(echo "$MODEL_INFO" | awk -F': ' '/"currentValue"/ {print $2}' | tr -d ',' | tr -d ' ') - LIMIT=$(echo "$MODEL_INFO" | awk -F': ' '/"limit"/ {print $2}' | tr -d ',' | tr -d ' ') - - CURRENT_VALUE=${CURRENT_VALUE:-0} - LIMIT=${LIMIT:-0} - - CURRENT_VALUE=$(echo "$CURRENT_VALUE" | cut -d'.' -f1) - LIMIT=$(echo "$LIMIT" | cut -d'.' -f1) - - AVAILABLE=$((LIMIT - CURRENT_VALUE)) - - echo "✅ Model: $MODEL | Used: $CURRENT_VALUE | Limit: $LIMIT | Available: $AVAILABLE" - - if [ "$AVAILABLE" -lt "${MIN_CAPACITY[$MODEL]}" ]; then - echo "❌ ERROR: $MODEL in $REGION has insufficient quota." - INSUFFICIENT_QUOTA=true - break - fi - done - - if [ "$INSUFFICIENT_QUOTA" = false ]; then - VALID_REGION="$REGION" - break - fi - -done - -if [ -z "$VALID_REGION" ]; then - echo "❌ No region with sufficient quota found. Blocking deployment." - echo "QUOTA_FAILED=true" >> "$GITHUB_ENV" - exit 0 -else - echo "✅ Final Region: $VALID_REGION" - echo "VALID_REGION=$VALID_REGION" >> "$GITHUB_ENV" - exit 0 -fi diff --git a/infra/scripts/quota_check_params.sh b/infra/scripts/quota_check_params.sh new file mode 100644 index 000000000..add6ac475 --- /dev/null +++ b/infra/scripts/quota_check_params.sh @@ -0,0 +1,249 @@ +#!/bin/bash +# VERBOSE=false + +MODELS="" +REGIONS="" +VERBOSE=false + +while [[ $# -gt 0 ]]; do + case "$1" in + --models) + MODELS="$2" + shift 2 + ;; + --regions) + REGIONS="$2" + shift 2 + ;; + --verbose) + VERBOSE=true + shift + ;; + *) + echo "Unknown option: $1" + exit 1 + ;; + esac +done + +# Fallback to defaults if not provided +[[ -z "$MODELS" ]] +[[ -z "$REGIONS" ]] + +echo "Models: $MODELS" +echo "Regions: $REGIONS" +echo "Verbose: $VERBOSE" + +for arg in "$@"; do + if [ "$arg" = "--verbose" ]; then + VERBOSE=true + fi +done + +log_verbose() { + if [ "$VERBOSE" = true ]; then + echo "$1" + fi +} + +# Default Models and Capacities (Comma-separated in "model:capacity" format) +DEFAULT_MODEL_CAPACITY="gpt-4o:50" +# Convert the comma-separated string into an array +IFS=',' read -r -a MODEL_CAPACITY_PAIRS <<< "$DEFAULT_MODEL_CAPACITY" + +echo "🔄 Fetching available Azure subscriptions..." +SUBSCRIPTIONS=$(az account list --query "[?state=='Enabled'].{Name:name, ID:id}" --output tsv) +SUB_COUNT=$(echo "$SUBSCRIPTIONS" | wc -l) + +if [ "$SUB_COUNT" -eq 0 ]; then + echo "❌ ERROR: No active Azure subscriptions found. Please log in using 'az login' and ensure you have an active subscription." + exit 1 +elif [ "$SUB_COUNT" -eq 1 ]; then + # If only one subscription, automatically select it + AZURE_SUBSCRIPTION_ID=$(echo "$SUBSCRIPTIONS" | awk '{print $2}') + if [ -z "$AZURE_SUBSCRIPTION_ID" ]; then + echo "❌ ERROR: No active Azure subscriptions found. Please log in using 'az login' and ensure you have an active subscription." + exit 1 + fi + echo "✅ Using the only available subscription: $AZURE_SUBSCRIPTION_ID" +else + # If multiple subscriptions exist, prompt the user to choose one + echo "Multiple subscriptions found:" + echo "$SUBSCRIPTIONS" | awk '{print NR")", $1, "-", $2}' + + while true; do + echo "Enter the number of the subscription to use:" + read SUB_INDEX + + # Validate user input + if [[ "$SUB_INDEX" =~ ^[0-9]+$ ]] && [ "$SUB_INDEX" -ge 1 ] && [ "$SUB_INDEX" -le "$SUB_COUNT" ]; then + AZURE_SUBSCRIPTION_ID=$(echo "$SUBSCRIPTIONS" | awk -v idx="$SUB_INDEX" 'NR==idx {print $2}') + echo "✅ Selected Subscription: $AZURE_SUBSCRIPTION_ID" + break + else + echo "❌ Invalid selection. Please enter a valid number from the list." + fi + done +fi + + +# Set the selected subscription +az account set --subscription "$AZURE_SUBSCRIPTION_ID" +echo "🎯 Active Subscription: $(az account show --query '[name, id]' --output tsv)" + +# Default Regions to check (Comma-separated, now configurable) +DEFAULT_REGIONS="eastus,uksouth,eastus2,northcentralus,swedencentral,westus,westus2,southcentralus,canadacentral" +IFS=',' read -r -a DEFAULT_REGION_ARRAY <<< "$DEFAULT_REGIONS" + +# Read parameters (if any) +IFS=',' read -r -a USER_PROVIDED_PAIRS <<< "$MODELS" +USER_REGION="$REGIONS" + +IS_USER_PROVIDED_PAIRS=false + +if [ ${#USER_PROVIDED_PAIRS[@]} -lt 1 ]; then + echo "No parameters provided, using default model-capacity pairs: ${MODEL_CAPACITY_PAIRS[*]}" +else + echo "Using provided model and capacity pairs: ${USER_PROVIDED_PAIRS[*]}" + IS_USER_PROVIDED_PAIRS=true + MODEL_CAPACITY_PAIRS=("${USER_PROVIDED_PAIRS[@]}") +fi + +declare -a FINAL_MODEL_NAMES +declare -a FINAL_CAPACITIES +declare -a TABLE_ROWS + +for PAIR in "${MODEL_CAPACITY_PAIRS[@]}"; do + MODEL_NAME=$(echo "$PAIR" | cut -d':' -f1 | tr '[:upper:]' '[:lower:]') + CAPACITY=$(echo "$PAIR" | cut -d':' -f2) + + if [ -z "$MODEL_NAME" ] || [ -z "$CAPACITY" ]; then + echo "❌ ERROR: Invalid model and capacity pair '$PAIR'. Both model and capacity must be specified." + exit 1 + fi + + FINAL_MODEL_NAMES+=("$MODEL_NAME") + FINAL_CAPACITIES+=("$CAPACITY") + +done + +echo "🔄 Using Models: ${FINAL_MODEL_NAMES[*]} with respective Capacities: ${FINAL_CAPACITIES[*]}" +echo "----------------------------------------" + +# Check if the user provided a region, if not, use the default regions +if [ -n "$USER_REGION" ]; then + echo "🔍 User provided region: $USER_REGION" + IFS=',' read -r -a REGIONS <<< "$USER_REGION" +else + echo "No region specified, using default regions: ${DEFAULT_REGION_ARRAY[*]}" + REGIONS=("${DEFAULT_REGION_ARRAY[@]}") + APPLY_OR_CONDITION=true +fi + +echo "✅ Retrieved Azure regions. Checking availability..." +INDEX=1 + +VALID_REGIONS=() +for REGION in "${REGIONS[@]}"; do + log_verbose "----------------------------------------" + log_verbose "🔍 Checking region: $REGION" + + QUOTA_INFO=$(az cognitiveservices usage list --location "$REGION" --output json | tr '[:upper:]' '[:lower:]') + if [ -z "$QUOTA_INFO" ]; then + log_verbose "⚠️ WARNING: Failed to retrieve quota for region $REGION. Skipping." + continue + fi + + TEXT_EMBEDDING_AVAILABLE=false + AT_LEAST_ONE_MODEL_AVAILABLE=false + TEMP_TABLE_ROWS=() + + for index in "${!FINAL_MODEL_NAMES[@]}"; do + MODEL_NAME="${FINAL_MODEL_NAMES[$index]}" + REQUIRED_CAPACITY="${FINAL_CAPACITIES[$index]}" + FOUND=false + INSUFFICIENT_QUOTA=false + + if [ "$MODEL_NAME" = "text-embedding-ada-002" ]; then + MODEL_TYPES=("openai.standard.$MODEL_NAME") + else + MODEL_TYPES=("openai.standard.$MODEL_NAME" "openai.globalstandard.$MODEL_NAME") + fi + + for MODEL_TYPE in "${MODEL_TYPES[@]}"; do + FOUND=false + INSUFFICIENT_QUOTA=false + log_verbose "🔍 Checking model: $MODEL_NAME with required capacity: $REQUIRED_CAPACITY ($MODEL_TYPE)" + + MODEL_INFO=$(echo "$QUOTA_INFO" | awk -v model="\"value\": \"$MODEL_TYPE\"" ' + BEGIN { RS="},"; FS="," } + $0 ~ model { print $0 } + ') + + if [ -z "$MODEL_INFO" ]; then + FOUND=false + log_verbose "⚠️ WARNING: No quota information found for model: $MODEL_NAME in region: $REGION for model type: $MODEL_TYPE." + continue + fi + + if [ -n "$MODEL_INFO" ]; then + FOUND=true + CURRENT_VALUE=$(echo "$MODEL_INFO" | awk -F': ' '/"currentvalue"/ {print $2}' | tr -d ',' | tr -d ' ') + LIMIT=$(echo "$MODEL_INFO" | awk -F': ' '/"limit"/ {print $2}' | tr -d ',' | tr -d ' ') + + CURRENT_VALUE=${CURRENT_VALUE:-0} + LIMIT=${LIMIT:-0} + + CURRENT_VALUE=$(echo "$CURRENT_VALUE" | cut -d'.' -f1) + LIMIT=$(echo "$LIMIT" | cut -d'.' -f1) + + AVAILABLE=$((LIMIT - CURRENT_VALUE)) + log_verbose "✅ Model: $MODEL_TYPE | Used: $CURRENT_VALUE | Limit: $LIMIT | Available: $AVAILABLE" + + if [ "$AVAILABLE" -ge "$REQUIRED_CAPACITY" ]; then + FOUND=true + if [ "$MODEL_NAME" = "text-embedding-ada-002" ]; then + TEXT_EMBEDDING_AVAILABLE=true + fi + AT_LEAST_ONE_MODEL_AVAILABLE=true + TEMP_TABLE_ROWS+=("$(printf "| %-4s | %-20s | %-43s | %-10s | %-10s | %-10s |" "$INDEX" "$REGION" "$MODEL_TYPE" "$LIMIT" "$CURRENT_VALUE" "$AVAILABLE")") + else + INSUFFICIENT_QUOTA=true + fi + fi + + if [ "$FOUND" = false ]; then + log_verbose "❌ No models found for model: $MODEL_NAME in region: $REGION (${MODEL_TYPES[*]})" + + elif [ "$INSUFFICIENT_QUOTA" = true ]; then + log_verbose "⚠️ Model $MODEL_NAME in region: $REGION has insufficient quota (${MODEL_TYPES[*]})." + fi + done + done + +if { [ "$IS_USER_PROVIDED_PAIRS" = true ] && [ "$INSUFFICIENT_QUOTA" = false ] && [ "$FOUND" = true ]; } || { [ "$APPLY_OR_CONDITION" != true ] || [ "$AT_LEAST_ONE_MODEL_AVAILABLE" = true ]; }; then + VALID_REGIONS+=("$REGION") + TABLE_ROWS+=("${TEMP_TABLE_ROWS[@]}") + INDEX=$((INDEX + 1)) + elif [ ${#USER_PROVIDED_PAIRS[@]} -eq 0 ]; then + echo "🚫 Skipping $REGION as it does not meet quota requirements." + fi + +done + +if [ ${#TABLE_ROWS[@]} -eq 0 ]; then + echo "--------------------------------------------------------------------------------------------------------------------" + + echo "❌ No regions have sufficient quota for all required models. Please request a quota increase: https://aka.ms/oai/stuquotarequest" +else + echo "---------------------------------------------------------------------------------------------------------------------" + printf "| %-4s | %-20s | %-43s | %-10s | %-10s | %-10s |\n" "No." "Region" "Model Name" "Limit" "Used" "Available" + echo "---------------------------------------------------------------------------------------------------------------------" + for ROW in "${TABLE_ROWS[@]}"; do + echo "$ROW" + done + echo "---------------------------------------------------------------------------------------------------------------------" + echo "➡️ To request a quota increase, visit: https://aka.ms/oai/stuquotarequest" +fi + +echo "✅ Script completed."