Merge branch 'main' into 304-failover-network-documenation

Author: mattdot

.github/workflows/azure-dev-down.yml

@@ -85,3 +85,25 @@ jobs:
           azd package # trigger prepackage hook to setup terraform provider
           azd provision --preview  # https://github.com/Azure/azure-dev/issues/4317
           azd down --no-prompt --force --purge
+
+      - name: Purge Soft-Deleted Azure OpenAI Resources
+        shell: bash
+        run: |
+          # Get the OpenAI resource name and location from environment outputs
+          OPENAI_RESOURCE_NAME=$(azd env get-values --output json | jq -r '.openai_resource_name // empty')
+          AZURE_REGION=$(azd env get-values --output json | jq -r '.primary_azure_region // empty')
+          RESOURCE_GROUP=$(azd env get-values --output json | jq -r '.resource_group_name // empty')
+
+          # Only attempt to purge if we have the required information
+          if [[ -n "$OPENAI_RESOURCE_NAME" && -n "$AZURE_REGION" ]]; then
+            echo "Attempting to purge soft-deleted Azure OpenAI resource: $OPENAI_RESOURCE_NAME in $AZURE_REGION"
+            
+            # Purge the soft-deleted Cognitive Services account (continue on error if resource not found)
+            az cognitiveservices account purge \
+              --location "$AZURE_REGION" \
+              --resource-group "$RESOURCE_GROUP" \
+              --name "$OPENAI_RESOURCE_NAME" || echo "Resource may not be in soft-delete state or already purged"
+          else
+            echo "OpenAI resource information not found in environment outputs. Skipping purge."
+          fi
+

.github/workflows/azure-dev.yml

@@ -231,4 +231,26 @@ jobs:
           azd env set RESOURCE_TAGS "$RESOURCE_TAGS"
 
           azd env select "$AZURE_ENV_NAME"
-          azd down --no-prompt --force --purge
+          azd down --no-prompt --force --purge
+
+      - name: Purge Soft-Deleted Azure OpenAI Resources
+        if: ${{ github.event.inputs.run_azd_down == 'true' || github.event_name == 'pull_request' }}
+        shell: bash
+        run: |
+          # Get the OpenAI resource name and location from environment outputs
+          OPENAI_RESOURCE_NAME=$(azd env get-values --output json | jq -r '.openai_resource_name // empty')
+          AZURE_REGION=$(azd env get-values --output json | jq -r '.primary_azure_region // empty')
+          RESOURCE_GROUP=$(azd env get-values --output json | jq -r '.resource_group_name // empty')
+
+          # Only attempt to purge if we have the required information
+          if [[ -n "$OPENAI_RESOURCE_NAME" && -n "$AZURE_REGION" ]]; then
+            echo "Attempting to purge soft-deleted Azure OpenAI resource: $OPENAI_RESOURCE_NAME in $AZURE_REGION"
+            
+            # Purge the soft-deleted Cognitive Services account (continue on error if resource not found)
+            az cognitiveservices account purge \
+              --location "$AZURE_REGION" \
+              --resource-group "$RESOURCE_GROUP" \
+              --name "$OPENAI_RESOURCE_NAME" || echo "Resource may not be in soft-delete state or already purged"
+          else
+            echo "OpenAI resource information not found in environment outputs. Skipping purge."
+          fi

README.md

@@ -20,20 +20,13 @@ network security.
     - [Deploying](#deploying)
     - [Using the Bot](#using-the-bot)
     - [Clean Up](#clean-up)
-  - [Testing](#testing)
-    - [Copilot Studio Agent Test](#copilot-studio-agent-test)
-      - [Running Tests After Local Deployment Execution](#running-tests-after-local-deployment-execution)
-      - [Running Tests with Manual Environment Variable Configuration](#running-tests-with-manual-environment-variable-configuration)
-    - [AI Search Test (Optional)](#ai-search-test-optional)
-      - [Prerequisites for AI Search Tests](#prerequisites-for-ai-search-tests)
-      - [Running AI Search Tests Locally](#running-ai-search-tests-locally)
   - [Advanced Scenarios](#advanced-scenarios)
+    - [Security Considerations](#security-considerations)
+    - [Infrastructure Resilience Considerations](#infrastructure-resilience-considerations)
     - [GitHub Self-Hosted Runners](#github-self-hosted-runners)
+    - [Testing](#testing)
     - [Bring Your Own Networking](#bring-your-own-networking)
     - [Custom Resource Group](#custom-resource-group)
-  - [Additional Considerations](#additional-considerations)
-    - [Security Considerations](#security-considerations)
-    - [Production Readiness](#production-readiness)
   - [Resources](#resources)
   - [Data Collection](#data-collection)
   - [Responsible AI](#responsible-ai)
@@ -222,6 +215,13 @@ To clean up all the resources created by this sample:
 
 All the Azure and Power Platform resources will be deleted.
 
+> [!NOTE] 
+> Azure OpenAI resources are soft-deleted by default and remain in a "recently deleted" state for 48 hours. If you need to redeploy with the same resource names before the retention period expires, you can manually purge the soft-deleted resource using:
+> ```bash
+> az cognitiveservices account purge --location <region> --resource-group <rg-name> --name <openai-name>
+> ```
+> In CI/CD workflows, this purge step is automated to prevent deployment conflicts.
+
 ## Advanced Scenarios
 
 ### Security Considerations
@@ -288,3 +288,7 @@ systems to ensure ethical, safe, and inclusive AI practices. Learn more at <http
 
 This is a sample built to demonstrate the capabilities of modern Generative AI apps and how they can be built in Azure.
 For help with deploying this sample, please post in [GitHub Issues](/issues).
+
+
+
+

decision-log/006-azure-openai-soft-delete-purge.md

@@ -0,0 +1,105 @@
+# Decision Log 006: Azure OpenAI Soft-Delete Purge in CI/CD
+
+**Date:** 2025-10-16  
+**Status:** Approved
+
+## Context
+
+Azure OpenAI (Cognitive Services) resources implement a soft-delete feature for data protection. When resources are deleted via `azd down` or Terraform `destroy`, they enter a soft-deleted state and remain in the subscription's "recently deleted" list for a retention period (typically 48 hours). During this time, the resource name is reserved and cannot be reused.
+
+This creates a problem for CI/CD workflows that:
+1. Deploy infrastructure with deterministic resource names based on environment
+2. Run automated tests
+3. Clean up resources after testing
+4. Attempt to redeploy with the same resource names
+
+The second deployment fails because the soft-deleted resource with the same name still exists in the purge queue.
+
+## Decision
+
+We implement a post-destruction purge step in GitHub Actions workflows to immediately remove soft-deleted Azure OpenAI resources after `azd down` completes. This is implemented directly in workflow files rather than as an azd hook because:
+
+1. **azd Hook Limitations**: Azure Developer CLI (azd) does not currently support `postdown` or `predestroy` hooks
+2. **Workflow Control**: GitHub Actions workflows provide better visibility and control over the purge process
+3. **Error Handling**: Workflow steps can gracefully handle cases where resources are not in soft-delete state
+
+## Implementation
+
+### Terraform Output Addition
+
+Added `openai_resource_name` output to `infra/outputs.tf`:
+
+```hcl
+output "openai_resource_name" {
+  description = "The name of the Azure OpenAI resource (for purging soft-deleted resources)"
+  value       = module.azure_open_ai.resource.name
+}
+```
+
+This exposes the resource name to azd environment values for use in the purge command.
+
+### Workflow Step Addition
+
+Added "Purge Soft-Deleted Azure OpenAI Resources" step to both:
+- `.github/workflows/azure-dev.yml` (after "Destroy Infrastructure" step)
+- `.github/workflows/azure-dev-down.yml` (after "Azd down" step)
+
+The purge step:
+1. Retrieves resource metadata from `azd env get-values` output
+2. Executes `az cognitiveservices account purge` command
+3. Continues gracefully if resource is not found or already purged
+4. Only runs when infrastructure destruction occurs (PR builds or manual trigger with `run_azd_down`)
+
+### Command Used
+
+```bash
+az cognitiveservices account purge \
+  --location "$AZURE_REGION" \
+  --resource-group "$RESOURCE_GROUP" \
+  --name "$OPENAI_RESOURCE_NAME"
+```
+
+## Rationale
+
+1. **CI/CD Reliability**: Ensures subsequent deployments with the same resource names succeed without manual intervention
+2. **Cost Optimization**: Immediately releases resources rather than waiting for the retention period to expire
+3. **Clean State**: Prevents accumulation of soft-deleted resources in the subscription
+4. **Automation**: No manual Azure Portal interaction required to purge resources
+5. **Safety**: Error handling prevents workflow failure if resource is already purged or not in soft-delete state
+
+## Alternative Approaches Considered
+
+### 1. Dynamic Resource Naming
+**Rejected**: Would require changing naming strategy and complicate resource tracking across deployments
+
+### 2. Manual Purge Between Deployments
+**Rejected**: Defeats the purpose of automated CI/CD and introduces human error
+
+### 3. Terraform Custom Provisioner
+**Rejected**: Terraform provisioners are a last resort and azd handles destroy operations outside direct Terraform control
+
+### 4. Wait for Retention Period
+**Rejected**: 48-hour wait between deployments is unacceptable for CI/CD velocity
+
+## Consequences
+
+### Positive
+- Automated, reliable CI/CD deployments with consistent resource naming
+- No manual intervention required for resource cleanup
+- Clear audit trail of purge operations in workflow logs
+
+### Negative
+- Adds minor complexity to workflow files
+- Requires Azure CLI authentication in workflows (already present)
+- Purge operation is permanent and cannot be undone
+
+### Neutral
+- Purge step execution time is minimal (typically < 5 seconds)
+- Requires `openai_resource_name` output to be maintained in Terraform
+
+## References
+
+- [Azure Cognitive Services soft-delete documentation](https://learn.microsoft.com/azure/cognitive-services/manage-resources-deletion-recovery)
+- [Azure CLI cognitiveservices account purge command](https://learn.microsoft.com/cli/azure/cognitiveservices/account?view=azure-cli-latest#az-cognitiveservices-account-purge)
+- [Azure Developer CLI hooks documentation](https://learn.microsoft.com/azure/developer/azure-developer-cli/azd-extensibility)
+- GitHub Issue #309: Azure OpenAI Resources Soft-Deleted

docs/troubleshooting.md

@@ -8,6 +8,21 @@ If you see an InsufficientQuota error mentioning "Tokens Per Minute", the reques
 
 This occurs when Terraform tries to create the private endpoint before the Azure OpenAI (Cognitive Services) account leaves the `Accepted` state; wait until the resource shows `Succeeded` (portal or `az resource show`) and re-run the provisioning (`azd provision`).
 
+## Azure OpenAI resource name already exists error
+
+If you encounter an error like `Code="ResourceNameAlreadyExists" Message="The resource name 'oai-copilot-studio-xyz' is already in use"` during deployment, this typically indicates that an Azure OpenAI resource with the same name exists in a soft-deleted state.
+
+**Symptoms:**
+- Deployment fails with `ResourceNameAlreadyExists` error
+- The resource was previously deleted via `azd down` or Terraform destroy
+- Error occurs within 48 hours of the previous deletion
+
+**Cause:**
+Azure OpenAI (Cognitive Services) resources are soft-deleted by default and remain in a "recently deleted" state for 48 hours. During this retention period, the resource name is reserved and cannot be reused.
+
+**Solution:**
+In CI/CD workflows, soft-deleted resources are automatically purged after `azd down`. For local development, you can manually purge the resource by following the instructions in the [Clean Up section of the README](../README.md#clean-up).
+
 ## Use GitHub Copilot to help troubleshoot
 
 If you're unsure how to fix a deployment error, open the relevant files in VS Code and use GitHub Copilot for suggestions. Copilot can propose TFVARS overrides, sample values, terraform plan snippets, or concise support-request wording; always review and test generated suggestions before applying them.

infra/outputs.tf

@@ -43,6 +43,11 @@ output "openai_endpoint" {
   value       = module.azure_open_ai.endpoint
 }
 
+output "openai_resource_name" {
+  description = "The name of the Azure OpenAI resource"
+  value       = module.azure_open_ai.resource.name
+}
+
 output "primary_azure_region" {
   description = "The primary Azure region for deployment"
   value       = local.primary_azure_region