From 98aa099451a80993dbbf152e9553e1daa49ddc61 Mon Sep 17 00:00:00 2001 From: "Dina Berry (MSFT)" Date: Thu, 28 Mar 2024 13:00:49 -0700 Subject: [PATCH 01/10] AI scale with APIM --- ...-chat-scaling-with-azure-api-management.md | 52 +++++++++++++++++++ 1 file changed, 52 insertions(+) create mode 100644 docs/ai/get-started-app-chat-scaling-with-azure-api-management.md diff --git a/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md b/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md new file mode 100644 index 0000000000000..75dd3565a688d --- /dev/null +++ b/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md @@ -0,0 +1,52 @@ +--- +title: Scale Azure OpenAI for .NET with Azure API Management +description: Learn how to add load balancing to your .NET application to extend the chat app beyond the Azure OpenAI token and model quota limits with Azure API Management. +ms.date: 03/28/2024 +ms.topic: get-started +ms.custom: devx-track-dotnet, devx-track-dotnet-ai +# CustomerIntent: As a .NET developer new to Azure OpenAI, I want to scale my Azure OpenAI capacity to avoid rate limit errors with Azure API Management. +--- + +# Scale Azure OpenAI for .NET chat using RAG with Azure API Management + +[!INCLUDE [aca-load-balancer-intro](~/azure-dev-docs-pr/articles/intro/includes/scaling-load-balancer-introduction-azure-api-management.md)] + +## Prerequisites + +* Azure subscription. [Create one for free](https://azure.microsoft.com/free/ai-services?azure-portal=true) +* Access granted to Azure OpenAI in the desired Azure subscription. + + Currently, access to this service is granted only by application. You can apply for access to Azure OpenAI by completing the form at https://aka.ms/oai/access. + +* [Dev containers](https://containers.dev/) are available for both samples, with all dependencies required to complete this article. You can run the dev containers in GitHub Codespaces (in a browser) or locally using Visual Studio Code. + + #### [Codespaces (recommended)](#tab/github-codespaces) + + * Only a [GitHub account](https://www.github.com/login) is required to use Codespaces + + #### [Visual Studio Code](#tab/visual-studio-code) + * [Azure Developer CLI](../azure-developer-cli/install-azd.md?tabs=winget-windows%2Cbrew-mac%2Cscript-linux&pivots=os-windows) + * [Docker Desktop](https://www.docker.com/products/docker-desktop/) - start Docker Desktop if it's not already running + * [Visual Studio Code](https://code.visualstudio.com/) with [Dev Container Extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) + + --- + +[!INCLUDE [scaling-load-balancer-aca-procedure.md](../intro/includes/scaling-load-balancer-procedure-azure-api-management.md)] + +[!INCLUDE [deployment-procedure](../intro/includes/redeploy-procedure-chat-azure-api-management.md)] + +[!INCLUDE [capacity.md](../intro/includes/scaling-load-balancer-capacity.md)] + +[!INCLUDE [py-apim-cleanup](../intro/includes/scaling-load-balancer-cleanup-azure-api-management.md)] + +## Sample code + +Samples used in this article include: + +* [.NET chat app with RAG](https://github.com/Azure-Samples/azure-search-openai-demo-csharp) +* [Load Balancer with Azure API Management](https://github.com/Azure-Samples/openai-apim-lb) + +## Next step + +* [View Azure API Management diagnostic data in Azure Monitor](/azure/api-management/api-management-howto-use-azure-monitor#view-diagnostic-data-in-azure-monitor) +* Use [Azure Load Testing](/azure/load-testing/) to load test your chat app with From 9b12d6bc5cc916c07ab13c4526b4d79b26396ce9 Mon Sep 17 00:00:00 2001 From: "Dina Berry (MSFT)" Date: Thu, 28 Mar 2024 13:04:44 -0700 Subject: [PATCH 02/10] fix links to other repo --- ...arted-app-chat-scaling-with-azure-api-management.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md b/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md index 75dd3565a688d..3b2aa0c86cba6 100644 --- a/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md +++ b/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md @@ -25,19 +25,19 @@ ms.custom: devx-track-dotnet, devx-track-dotnet-ai * Only a [GitHub account](https://www.github.com/login) is required to use Codespaces #### [Visual Studio Code](#tab/visual-studio-code) - * [Azure Developer CLI](../azure-developer-cli/install-azd.md?tabs=winget-windows%2Cbrew-mac%2Cscript-linux&pivots=os-windows) + * [Azure Developer CLI](~/azure-dev-docs-pr/articles/azure-developer-cli/install-azd.md?tabs=winget-windows%2Cbrew-mac%2Cscript-linux&pivots=os-windows) * [Docker Desktop](https://www.docker.com/products/docker-desktop/) - start Docker Desktop if it's not already running * [Visual Studio Code](https://code.visualstudio.com/) with [Dev Container Extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) --- -[!INCLUDE [scaling-load-balancer-aca-procedure.md](../intro/includes/scaling-load-balancer-procedure-azure-api-management.md)] +[!INCLUDE [scaling-load-balancer-aca-procedure.md](~/azure-dev-docs-pr/articles/intro/includes/scaling-load-balancer-procedure-azure-api-management.md)] -[!INCLUDE [deployment-procedure](../intro/includes/redeploy-procedure-chat-azure-api-management.md)] +[!INCLUDE [deployment-procedure](~/azure-dev-docs-pr/articles/intro/includes/redeploy-procedure-chat-azure-api-management.md)] -[!INCLUDE [capacity.md](../intro/includes/scaling-load-balancer-capacity.md)] +[!INCLUDE [capacity.md](~/azure-dev-docs-pr/articles/intro/includes/scaling-load-balancer-capacity.md)] -[!INCLUDE [py-apim-cleanup](../intro/includes/scaling-load-balancer-cleanup-azure-api-management.md)] +[!INCLUDE [py-apim-cleanup](~/azure-dev-docs-pr/articles/intro/includes/scaling-load-balancer-cleanup-azure-api-management.md)] ## Sample code From 8f86fe644de0cabafb62b2aa1fea097a58be38e0 Mon Sep 17 00:00:00 2001 From: "Dina Berry (MSFT)" Date: Thu, 28 Mar 2024 13:11:56 -0700 Subject: [PATCH 03/10] markdown lint fixes --- ...-chat-scaling-with-azure-api-management.md | 27 ++++++++++--------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md b/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md index 3b2aa0c86cba6..326fd99a90bc7 100644 --- a/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md +++ b/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md @@ -13,23 +13,24 @@ ms.custom: devx-track-dotnet, devx-track-dotnet-ai ## Prerequisites -* Azure subscription. [Create one for free](https://azure.microsoft.com/free/ai-services?azure-portal=true) +* Azure subscription. [Create one for free](https://azure.microsoft.com/free/ai-services?azure-portal=true) * Access granted to Azure OpenAI in the desired Azure subscription. - Currently, access to this service is granted only by application. You can apply for access to Azure OpenAI by completing the form at https://aka.ms/oai/access. + Currently, access to this service is granted only by application. You can apply for access to Azure OpenAI by completing the [form](https://aka.ms/oai/access). * [Dev containers](https://containers.dev/) are available for both samples, with all dependencies required to complete this article. You can run the dev containers in GitHub Codespaces (in a browser) or locally using Visual Studio Code. - #### [Codespaces (recommended)](#tab/github-codespaces) - - * Only a [GitHub account](https://www.github.com/login) is required to use Codespaces - - #### [Visual Studio Code](#tab/visual-studio-code) - * [Azure Developer CLI](~/azure-dev-docs-pr/articles/azure-developer-cli/install-azd.md?tabs=winget-windows%2Cbrew-mac%2Cscript-linux&pivots=os-windows) - * [Docker Desktop](https://www.docker.com/products/docker-desktop/) - start Docker Desktop if it's not already running - * [Visual Studio Code](https://code.visualstudio.com/) with [Dev Container Extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) - - --- +#### [Codespaces (recommended)](#tab/github-codespaces) + +* Only a [GitHub account](https://www.github.com/login) is required to use Codespaces + +#### [Visual Studio Code](#tab/visual-studio-code) + +* [Azure Developer CLI](~/azure-dev-docs-pr/articles/azure-developer-cli/install-azd.md?tabs=winget-windows%2Cbrew-mac%2Cscript-linux&pivots=os-windows) +* [Docker Desktop](https://www.docker.com/products/docker-desktop/) - start Docker Desktop if it's not already running +* [Visual Studio Code](https://code.visualstudio.com/) with [Dev Container Extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) + +--- [!INCLUDE [scaling-load-balancer-aca-procedure.md](~/azure-dev-docs-pr/articles/intro/includes/scaling-load-balancer-procedure-azure-api-management.md)] @@ -49,4 +50,4 @@ Samples used in this article include: ## Next step * [View Azure API Management diagnostic data in Azure Monitor](/azure/api-management/api-management-howto-use-azure-monitor#view-diagnostic-data-in-azure-monitor) -* Use [Azure Load Testing](/azure/load-testing/) to load test your chat app with +* Use [Azure Load Testing](/azure/load-testing/) to load test your chat app From fd5d0ca092df8642e536b74451e6aeda1cc4ac59 Mon Sep 17 00:00:00 2001 From: "Dina Berry (MSFT)" Date: Thu, 28 Mar 2024 13:14:53 -0700 Subject: [PATCH 04/10] Update docs/ai/get-started-app-chat-scaling-with-azure-api-management.md Co-authored-by: Genevieve Warren <24882762+gewarren@users.noreply.github.com> --- .../get-started-app-chat-scaling-with-azure-api-management.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md b/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md index 326fd99a90bc7..20ebccafd4fec 100644 --- a/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md +++ b/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md @@ -47,7 +47,7 @@ Samples used in this article include: * [.NET chat app with RAG](https://github.com/Azure-Samples/azure-search-openai-demo-csharp) * [Load Balancer with Azure API Management](https://github.com/Azure-Samples/openai-apim-lb) -## Next step +## Next steps * [View Azure API Management diagnostic data in Azure Monitor](/azure/api-management/api-management-howto-use-azure-monitor#view-diagnostic-data-in-azure-monitor) * Use [Azure Load Testing](/azure/load-testing/) to load test your chat app From 8b7ab6191464be60763d92f6f79ba6b6f16f0b79 Mon Sep 17 00:00:00 2001 From: "Dina Berry (MSFT)" Date: Thu, 28 Mar 2024 13:15:07 -0700 Subject: [PATCH 05/10] Update docs/ai/get-started-app-chat-scaling-with-azure-api-management.md Co-authored-by: Genevieve Warren <24882762+gewarren@users.noreply.github.com> --- .../get-started-app-chat-scaling-with-azure-api-management.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md b/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md index 20ebccafd4fec..db2e951d5f6ae 100644 --- a/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md +++ b/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md @@ -49,5 +49,5 @@ Samples used in this article include: ## Next steps -* [View Azure API Management diagnostic data in Azure Monitor](/azure/api-management/api-management-howto-use-azure-monitor#view-diagnostic-data-in-azure-monitor) +* [View Azure API Management diagnostic data in Azure Monitor.](/azure/api-management/api-management-howto-use-azure-monitor#view-diagnostic-data-in-azure-monitor) * Use [Azure Load Testing](/azure/load-testing/) to load test your chat app From 097e81fdd602ea22d539400753cecb6bdc2b4989 Mon Sep 17 00:00:00 2001 From: "Dina Berry (MSFT)" Date: Thu, 28 Mar 2024 13:15:59 -0700 Subject: [PATCH 06/10] fixes --- .../get-started-app-chat-scaling-with-azure-api-management.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md b/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md index 326fd99a90bc7..adfd57a607dc2 100644 --- a/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md +++ b/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md @@ -13,7 +13,7 @@ ms.custom: devx-track-dotnet, devx-track-dotnet-ai ## Prerequisites -* Azure subscription. [Create one for free](https://azure.microsoft.com/free/ai-services?azure-portal=true) +* An Azure subscription. [Create one for free.](https://azure.microsoft.com/free/ai-services?azure-portal=true) * Access granted to Azure OpenAI in the desired Azure subscription. Currently, access to this service is granted only by application. You can apply for access to Azure OpenAI by completing the [form](https://aka.ms/oai/access). @@ -50,4 +50,4 @@ Samples used in this article include: ## Next step * [View Azure API Management diagnostic data in Azure Monitor](/azure/api-management/api-management-howto-use-azure-monitor#view-diagnostic-data-in-azure-monitor) -* Use [Azure Load Testing](/azure/load-testing/) to load test your chat app +* Use [Azure Load Testing](/azure/load-testing/) to load test your chat app. From 4992fffe5138527704ca10b1923088614e6989ab Mon Sep 17 00:00:00 2001 From: "Dina Berry (MSFT)" Date: Thu, 28 Mar 2024 13:22:57 -0700 Subject: [PATCH 07/10] remove trailing space --- .../get-started-app-chat-scaling-with-azure-api-management.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md b/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md index 9b0decb72c47c..86afa20d31d37 100644 --- a/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md +++ b/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md @@ -42,7 +42,7 @@ ms.custom: devx-track-dotnet, devx-track-dotnet-ai ## Sample code -Samples used in this article include: +Samples used in this article include: * [.NET chat app with RAG](https://github.com/Azure-Samples/azure-search-openai-demo-csharp) * [Load Balancer with Azure API Management](https://github.com/Azure-Samples/openai-apim-lb) From 080b37faad97837a6dbc9ba3da477e4781a2364f Mon Sep 17 00:00:00 2001 From: "Dina Berry (MSFT)" Date: Thu, 28 Mar 2024 13:36:20 -0700 Subject: [PATCH 08/10] add doc to toc --- docs/azure/TOC.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/azure/TOC.yml b/docs/azure/TOC.yml index 52c66b5d70783..c171e3f584c4c 100644 --- a/docs/azure/TOC.yml +++ b/docs/azure/TOC.yml @@ -63,6 +63,8 @@ href: ../ai/get-started-app-chat-template.md - name: Scale Azure OpenAI with Azure Container apps href: ../ai/get-started-app-chat-scaling-with-azure-container-apps.md + - name: Scale Azure OpenAI with Azure API Management + href: ../ai/get-started-app-chat-scaling-with-azure-api-management.md - name: Azure SDK for .NET items: - name: What is the Azure SDK for .NET? From ea0e0fcc0c4259ad86bc19b493f56a8b3abad2a0 Mon Sep 17 00:00:00 2001 From: "Dina Berry (MSFT)" Date: Thu, 28 Mar 2024 14:07:38 -0700 Subject: [PATCH 09/10] edits --- .../get-started-app-chat-scaling-with-azure-api-management.md | 2 +- .../get-started-app-chat-scaling-with-azure-container-apps.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md b/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md index 86afa20d31d37..eccf15df18722 100644 --- a/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md +++ b/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md @@ -26,7 +26,7 @@ ms.custom: devx-track-dotnet, devx-track-dotnet-ai #### [Visual Studio Code](#tab/visual-studio-code) -* [Azure Developer CLI](~/azure-dev-docs-pr/articles/azure-developer-cli/install-azd.md?tabs=winget-windows%2Cbrew-mac%2Cscript-linux&pivots=os-windows) +* [Azure Developer CLI](/azure/developer/azure-developer-cli/install-azd) * [Docker Desktop](https://www.docker.com/products/docker-desktop/) - start Docker Desktop if it's not already running * [Visual Studio Code](https://code.visualstudio.com/) with [Dev Container Extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) diff --git a/docs/ai/get-started-app-chat-scaling-with-azure-container-apps.md b/docs/ai/get-started-app-chat-scaling-with-azure-container-apps.md index 2babbad8cd0d2..6c6a5b500424c 100644 --- a/docs/ai/get-started-app-chat-scaling-with-azure-container-apps.md +++ b/docs/ai/get-started-app-chat-scaling-with-azure-container-apps.md @@ -51,4 +51,4 @@ Samples used in this article include: ## Next step -* Use [Azure Load Testing](/azure/load-testing/) to load test your chat app +* Use [Azure Load Testing](/azure/load-testing/) to load test your chat app. From 73b093a378fdbf6bd61b89f10b4737f9d8f3aaf4 Mon Sep 17 00:00:00 2001 From: "Dina Berry (MSFT)" Date: Fri, 29 Mar 2024 16:11:00 -0700 Subject: [PATCH 10/10] touch --- .../get-started-app-chat-scaling-with-azure-api-management.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md b/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md index eccf15df18722..e0e91c23446ea 100644 --- a/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md +++ b/docs/ai/get-started-app-chat-scaling-with-azure-api-management.md @@ -1,7 +1,7 @@ --- title: Scale Azure OpenAI for .NET with Azure API Management description: Learn how to add load balancing to your .NET application to extend the chat app beyond the Azure OpenAI token and model quota limits with Azure API Management. -ms.date: 03/28/2024 +ms.date: 03/29/2024 ms.topic: get-started ms.custom: devx-track-dotnet, devx-track-dotnet-ai # CustomerIntent: As a .NET developer new to Azure OpenAI, I want to scale my Azure OpenAI capacity to avoid rate limit errors with Azure API Management.