diff --git a/DEPLOYMENT.md b/DEPLOYMENT.md index 1f5da32..3b411e7 100644 --- a/DEPLOYMENT.md +++ b/DEPLOYMENT.md @@ -13,7 +13,7 @@ Healthcare AI Model Evaluator consists of: ## Getting Started -### Prerequisites +### Pre-requisites > [!IMPORTANT] > Follow the steps in order. Each step builds on the previous ones. @@ -22,7 +22,7 @@ Healthcare AI Model Evaluator consists of: - [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli) - [Azure Developer CLI (azd)](https://learn.microsoft.com/en-us/azure/developer/azure-developer-cli/install-azd) - [Docker Desktop](https://docs.docker.com/get-docker/) (for Functions deployment) -- DotNet v8.0.318 +- [DotNet v8.0.318](https://github.com/dotnet/core/blob/main/release-notes/8.0/8.0.21/8.0.318.md) **Azure Subscription Requirements:** - **Azure OpenAI**: Access to one of the supported models for Model-as-a-Judge with Metrics Azure Functions. @@ -67,6 +67,18 @@ Create a new environment with a short name: azd env new ``` +#### Pre-package Evaluator Addon + +By default, the [evaluator addon](./functions/addons/README.md) deployment is enabled. And for `azd up` to work properly, we must pre-package the addon, for that from the project root run: +```bash +./functions/addons/package_addon.sh evaluator +``` + +To disable the custom evaluator addon run: +```bash +azd env set ENABLE_EVALUATOR_ADDON false +``` + #### Azure OpenAI Configuration During deployment (`azd up`), you'll be prompted to select an Azure OpenAI model, capacity, and deployment type. You can review available models at the [Azure OpenAI Service models documentation](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/how-to/reasoning). @@ -89,6 +101,25 @@ azd env set EXISTING_AZURE_OPENAI_ENDPOINT "https://your-openai.openai.azure.com azd env set EXISTING_AZURE_OPENAI_KEY "your-api-key" ``` +#### Setup First Admin User + +You can bootstrap the first admin user automatically during deployment by setting these azd environment values: + +```bash +azd env set ROOT_ADMIN_EMAIL "admin@example.com" +azd env set ROOT_ADMIN_NAME "Admin User" +azd env set ROOT_ADMIN_PASSWORD "" +azd up +``` + +Once the application is running, you can: +1. Navigate to your application URL: `$(azd env get-value API_BASE_URL)` +2. Click "Sign in with Password" +3. Use the email/password you just created +4. Access the admin panel to create additional users + +> **Note**: This only needs to be set once. Additional users can be created through the web interface by admin users. + #### Optional Configuration **Regional Quota Flexibility**: If you have limited quota in your primary region, you can deploy specific resources to alternate regions: @@ -119,9 +150,6 @@ azd env set AZURE_LOCATION westus2 **Feature Flags**: Control optional components: ```sh -# Disable evaluator addon to reduce deployment time -azd env set ENABLE_EVALUATOR_ADDON false - # Disable Azure Communication Services azd env set ENABLE_ACS false ``` @@ -130,6 +158,10 @@ azd env set ENABLE_ACS false Now that your environment is configured, you can deploy all necessary resources and infrastructure for the Healthcare AI Model Evaluator. +```bash +azd up +``` + #### IP Filtering & Security Configuration <<<<<<< HEAD @@ -264,7 +296,7 @@ azd up > [!WARNING] > **Legacy migration (Mongo → SQL API)**: Older deployments used **Cosmos DB API for MongoDB**. This repository now provisions **Cosmos DB SQL API** instead, which means **existing Mongo data is not automatically migrated**. Plan for a one-time migration or accept data loss when moving environments. - +> > **Export (old Mongo API only — not for local dev)** > - Use `mongoexport` against your old Cosmos Mongo API account (or any Mongo-compatible endpoint): > ```sh @@ -334,9 +366,9 @@ azd up To start the deployment process, run: - ```bash - azd up - ``` +```bash +azd up +``` During deployment you will be prompted for any required variable not yet set, such as subscription, resource group and location. @@ -355,9 +387,9 @@ This command will: ### Step 4: Build and Push Metrics Function Docker Image - This step is necessary because `azd` does not automatically build and push Docker images for Azure Functions. +This step is necessary because `azd` does not automatically build and push Docker images for Azure Functions. - ```bash +```bash # From the root folder, get the registry name and endpoint AZURE_CONTAINER_REGISTRY_NAME=$(azd env get-value AZURE_CONTAINER_REGISTRY_NAME) AZURE_CONTAINER_REGISTRY_ENDPOINT=$(azd env get-value AZURE_CONTAINER_REGISTRY_ENDPOINT) @@ -369,14 +401,14 @@ cd functions az acr login --name $AZURE_CONTAINER_REGISTRY_NAME # Build the Docker image -docker compose build medbench-metrics +docker compose build haime-metrics # Tag the image for the registry -docker tag functions-medbench-metrics:latest $AZURE_CONTAINER_REGISTRY_ENDPOINT/medbench-metrics:latest +docker tag functions-haime-metrics:latest $AZURE_CONTAINER_REGISTRY_ENDPOINT/haime-metrics:latest # Push the image to Azure Container Registry -docker push $AZURE_CONTAINER_REGISTRY_ENDPOINT/medbench-metrics:latest - ``` +docker push $AZURE_CONTAINER_REGISTRY_ENDPOINT/haime-metrics:latest +``` > [!NOTE] > After pushing the image, the Azure Function will automatically pull and deploy it. This may take a few minutes. @@ -400,28 +432,6 @@ echo "Frontend: $(azd env get-value API_BASE_URL) echo "API: $(azd env get-value API_BASE_URL)/api" ``` -## Post-Deployment Setup - -### Create First Admin User - -You can bootstrap the first admin user automatically during deployment by setting these azd environment values: - -```bash -azd env set ROOT_ADMIN_EMAIL "admin@example.com" -azd env set ROOT_ADMIN_NAME "Admin User" -azd env set ROOT_ADMIN_PASSWORD "" -azd up -``` - -Once created, you can: -1. Navigate to your application URL: `$(azd env get-value API_BASE_URL)/webapp` -2. Click "Sign in with Password" -3. Use the email/password you just created -4. Access the admin panel to create additional users - -> **Note**: This only needs to be set once. Additional users can be created through the web interface by admin users. - - --- ## Architecture Overview @@ -551,7 +561,7 @@ For production healthcare environments, you should restrict access to your appli ### Integrate with Existing Azure Front Door -Most healthcare organizations already have Azure Front Door with WAF configured. You can integrate MedBench behind your existing Front Door. +Most healthcare organizations already have Azure Front Door with WAF configured. You can integrate HAIME behind your existing Front Door. #### Configure Container Apps for Front Door Integration @@ -608,18 +618,18 @@ az network front-door backend-pool backend add \ #### Update Front Door Routing Rules -Configure routing to send MedBench traffic to the new backend: +Configure routing to send HAIME traffic to the new backend: ```bash -# Create routing rule for MedBench +# Create routing rule for HAIME az network front-door routing-rule create \ --front-door-name "your-existing-frontdoor" \ --resource-group "your-frontdoor-rg" \ - --name "medbench-routing" \ + --name "haime-routing" \ --frontend-endpoints "your-frontend" \ --route-type Forward \ --backend-pool "your-backend-pool" \ - --patterns "/medbench/*" \ + --patterns "/haime/*" \ --accepted-protocols Https ``` diff --git a/README.md b/README.md index 6451fc8..63c2761 100644 --- a/README.md +++ b/README.md @@ -55,10 +55,12 @@ For a complete overview of the Healthcare AI Model Evaluator platform, refer to ## Deployment > [!IMPORTANT] -> See [DEPLOYMENT.md](./DEPLOYMENT.md) for complete deployment guide, configuration options, and troubleshooting. +> See [DEPLOYMENT.md](./DEPLOYMENT.md) for complete deployment guide, pre-requisites, configuration options, and troubleshooting. ### Quick start +Before starting with deployment, make sure you go through the [deployment pre-requisites](./DEPLOYMENT.md#pre-requisites). + Initialize and deploy the complete Healthcare AI Model Evaluator platform using Azure Developer CLI (azd). ```bash @@ -69,9 +71,14 @@ azd init -t microsoft/healthcare-ai-model-evaluator azd up ``` +To initialize the project targetting a specific branch, use the `--branch` flag: +```bash +azd init -t microsoft/healthcare-ai-model-evaluator --branch feature-branch +``` + ### First-time Setup -After deployment, bootstrap your first admin user by setting these azd environment values and re-running deployment: +To bootstrap your first admin user by setting these azd environment values and re-running deployment: ```bash azd env set ROOT_ADMIN_EMAIL "admin@example.com" @@ -94,9 +101,15 @@ npm run dev ### Backend Setup ```bash -export AZURE_STORAGE_ENDPOINT=[Your Storage Account blob endpoint] -export COSMOSDB_ENDPOINT=[Your Cosmos DB account endpoint] -export COSMOSDB_DATABASE=[Your Cosmos DB database name] +# For Linux/macOS +export AZURE_STORAGE_CONNECTION_STRING="Your Storage Account connection string" +export COSMOSDB_CONNECTION_STRING="Your mongodb connection string" +export COSMOSDB_ENDPOINT="Your mongodb endpoint" + +# For PowerShell +$env:AZURE_STORAGE_CONNECTION_STRING="Your Storage Account connection string" +$env:COSMOSDB_CONNECTION_STRING="Your mongodb connection string" +$env:COSMOSDB_ENDPOINT="Your mongodb endpoint" cd backend dotnet restore diff --git a/functions/Dockerfile b/functions/Dockerfile index 2df7cba..e5f5b14 100644 --- a/functions/Dockerfile +++ b/functions/Dockerfile @@ -10,7 +10,7 @@ ENV CONTAINER_VERSION="v8.1_fixed_verify" # RUN apt-get update && apt-get install -y git build-essential && rm -rf /var/lib/apt/lists/* RUN apt-get update && apt-get install -y --no-install-recommends git build-essential && rm -rf /var/lib/apt/lists/* && \ # Create a non-root user for security - groupadd -r appuser && useradd -r -g appuser -d /home/site appuser && \ + groupadd -r appuser && useradd -m -r -g appuser -d /home/site appuser && \ # Create a virtual environment to isolate dependencies python -m venv /home/site/venv && \ # Set ownership of the home directory diff --git a/functions/Dockerfile.addon-dev b/functions/Dockerfile.addon-dev index 444d115..661ef98 100644 --- a/functions/Dockerfile.addon-dev +++ b/functions/Dockerfile.addon-dev @@ -16,7 +16,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \ gcc \ g++ \ && rm -rf /var/lib/apt/lists/* \ - && groupadd -r appuser && useradd -r -g appuser -d /home/site appuser \ + && groupadd -r appuser && useradd -m -r -g appuser -d /home/site appuser \ && chown -R appuser:appuser /home/site # Copy requirements folder and main requirements diff --git a/infra/main.parameters.json b/infra/main.parameters.json index ac9f7c5..2f2a5a8 100644 --- a/infra/main.parameters.json +++ b/infra/main.parameters.json @@ -125,6 +125,15 @@ "allowedWebIp": { "value": "${ALLOWED_WEB_IP=}" }, + "rootAdminEmail": { + "value": "${ROOT_ADMIN_EMAIL=}" + }, + "rootAdminName": { + "value": "${ROOT_ADMIN_NAME=}" + }, + "rootAdminPassword": { + "value": "${ROOT_ADMIN_PASSWORD=}" + }, "deploymentNetworking": { "value": "${DEPLOYMENT_NETWORKING=private}" }, diff --git a/infra/scripts/postprovision.ps1 b/infra/scripts/postprovision.ps1 index a98dc63..ab4cfea 100644 --- a/infra/scripts/postprovision.ps1 +++ b/infra/scripts/postprovision.ps1 @@ -65,19 +65,25 @@ if ($clientId -and $clientId -ne "placeholder-will-be-updated-by-script" -and $c if ($principalId -and $principalId -ne "null") { Write-Host "Container App managed identity principal ID: $principalId" - # Assign Cosmos DB role + # Assign Cosmos DB SQL data-plane role (Built-in Data Contributor) + # This is required for reading/writing data via managed identity + # Built-in role IDs: 00000000-0000-0000-0000-000000000001 (Reader), 00000000-0000-0000-0000-000000000002 (Contributor) $cosmosAccountName = $env:COSMOS_ACCOUNT_NAME if ($cosmosAccountName) { - Write-Host "Assigning Cosmos DB role to Container App managed identity..." - $cosmosRoleId = "5bd9cd88-fe45-4216-938b-f97437e15450" # Cosmos DB Account Reader Writer - az cosmosdb sql role assignment create ` - --account-name $cosmosAccountName ` - --resource-group $resourceGroupName ` - --principal-id $principalId ` - --role-definition-id $cosmosRoleId ` - --scope "/subscriptions/$($env:AZURE_SUBSCRIPTION_ID)/resourceGroups/$resourceGroupName/providers/Microsoft.DocumentDB/databaseAccounts/$cosmosAccountName" ` - --output none 2>$null || Write-Host "Role assignment may already exist" - Write-Host "✅ Cosmos DB role assignment completed" + Write-Host "Assigning Cosmos DB SQL data-plane role to Container App managed identity..." + $cosmosAccountId = az cosmosdb show --name $cosmosAccountName --resource-group $resourceGroupName --query "id" -o tsv 2>$null + if ($cosmosAccountId) { + az cosmosdb sql role assignment create ` + --account-name $cosmosAccountName ` + --resource-group $resourceGroupName ` + --principal-id $principalId ` + --role-definition-id "$cosmosAccountId/sqlRoleDefinitions/00000000-0000-0000-0000-000000000002" ` + --scope $cosmosAccountId ` + --output none 2>$null || Write-Host "SQL role assignment may already exist" + Write-Host "✅ Cosmos DB SQL data-plane role assignment completed" + } else { + Write-Host "⚠️ Warning: Could not get Cosmos DB account ID" + } } # Assign Storage role @@ -248,19 +254,25 @@ try { if ($principalId -and $principalId -ne "null") { Write-Host "Container App managed identity principal ID: $principalId" - # Assign Cosmos DB role + # Assign Cosmos DB SQL data-plane role (Built-in Data Contributor) + # This is required for reading/writing data via managed identity + # Built-in role IDs: 00000000-0000-0000-0000-000000000001 (Reader), 00000000-0000-0000-0000-000000000002 (Contributor) $cosmosAccountName = $env:COSMOS_ACCOUNT_NAME if ($cosmosAccountName) { - Write-Host "Assigning Cosmos DB role to Container App managed identity..." - $cosmosRoleId = "5bd9cd88-fe45-4216-938b-f97437e15450" # Cosmos DB Account Reader Writer - az cosmosdb sql role assignment create ` - --account-name $cosmosAccountName ` - --resource-group $resourceGroupName ` - --principal-id $principalId ` - --role-definition-id $cosmosRoleId ` - --scope "/subscriptions/$($env:AZURE_SUBSCRIPTION_ID)/resourceGroups/$resourceGroupName/providers/Microsoft.DocumentDB/databaseAccounts/$cosmosAccountName" ` - --output none 2>$null || Write-Host "Role assignment may already exist" - Write-Host "✅ Cosmos DB role assignment completed" + Write-Host "Assigning Cosmos DB SQL data-plane role to Container App managed identity..." + $cosmosAccountId = az cosmosdb show --name $cosmosAccountName --resource-group $resourceGroupName --query "id" -o tsv 2>$null + if ($cosmosAccountId) { + az cosmosdb sql role assignment create ` + --account-name $cosmosAccountName ` + --resource-group $resourceGroupName ` + --principal-id $principalId ` + --role-definition-id "$cosmosAccountId/sqlRoleDefinitions/00000000-0000-0000-0000-000000000002" ` + --scope $cosmosAccountId ` + --output none 2>$null || Write-Host "SQL role assignment may already exist" + Write-Host "✅ Cosmos DB SQL data-plane role assignment completed" + } else { + Write-Host "⚠️ Warning: Could not get Cosmos DB account ID" + } } # Assign Storage role diff --git a/infra/scripts/postprovision.sh b/infra/scripts/postprovision.sh index 62cd362..528f26e 100755 --- a/infra/scripts/postprovision.sh +++ b/infra/scripts/postprovision.sh @@ -203,17 +203,24 @@ PRINCIPAL_ID=$(az containerapp show --name "$CONTAINER_APP_NAME" --resource-grou if [ -n "$PRINCIPAL_ID" ] && [ "$PRINCIPAL_ID" != "null" ]; then echo "Container App managed identity principal ID: $PRINCIPAL_ID" - # Assign Cosmos DB role + # Assign Cosmos DB SQL data-plane role (Built-in Data Contributor) + # This is required for reading/writing data via managed identity + # Built-in role IDs: 00000000-0000-0000-0000-000000000001 (Reader), 00000000-0000-0000-0000-000000000002 (Contributor) if [ -n "$COSMOS_ACCOUNT_NAME" ]; then - echo "Assigning Cosmos DB role to Container App managed identity..." - az cosmosdb sql role assignment create \ - --account-name "$COSMOS_ACCOUNT_NAME" \ - --resource-group "$RESOURCE_GROUP_NAME" \ - --principal-id "$PRINCIPAL_ID" \ - --role-definition-id "5bd9cd88-fe45-4216-938b-f97437e15450" \ - --scope "/subscriptions/$AZURE_SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP_NAME/providers/Microsoft.DocumentDB/databaseAccounts/$COSMOS_ACCOUNT_NAME" \ - --output none 2>/dev/null || echo "Role assignment may already exist" - echo "✅ Cosmos DB role assignment completed" + echo "Assigning Cosmos DB SQL data-plane role to Container App managed identity..." + COSMOS_ACCOUNT_ID=$(az cosmosdb show --name "$COSMOS_ACCOUNT_NAME" --resource-group "$RESOURCE_GROUP_NAME" --query "id" -o tsv 2>/dev/null || echo "") + if [ -n "$COSMOS_ACCOUNT_ID" ]; then + az cosmosdb sql role assignment create \ + --account-name "$COSMOS_ACCOUNT_NAME" \ + --resource-group "$RESOURCE_GROUP_NAME" \ + --principal-id "$PRINCIPAL_ID" \ + --role-definition-id "${COSMOS_ACCOUNT_ID}/sqlRoleDefinitions/00000000-0000-0000-0000-000000000002" \ + --scope "$COSMOS_ACCOUNT_ID" \ + --output none 2>/dev/null || echo "SQL role assignment may already exist" + echo "✅ Cosmos DB SQL data-plane role assignment completed" + else + echo "⚠️ Warning: Could not get Cosmos DB account ID" + fi fi # Assign Storage role