Fix/postprovision

DEPLOYMENT.md

@@ -13,7 +13,7 @@ Healthcare AI Model Evaluator consists of:
 
 ## Getting Started
 
-### Prerequisites
+### Pre-requisites
 
 > [!IMPORTANT]
 > Follow the steps in order. Each step builds on the previous ones.
@@ -22,7 +22,7 @@ Healthcare AI Model Evaluator consists of:
 - [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli)
 - [Azure Developer CLI (azd)](https://learn.microsoft.com/en-us/azure/developer/azure-developer-cli/install-azd)
 - [Docker Desktop](https://docs.docker.com/get-docker/) (for Functions deployment)
-- DotNet v8.0.318
+- [DotNet v8.0.318](https://github.com/dotnet/core/blob/main/release-notes/8.0/8.0.21/8.0.318.md)
 
 **Azure Subscription Requirements:**
 - **Azure OpenAI**: Access to one of the supported models for Model-as-a-Judge with Metrics Azure Functions.
@@ -67,6 +67,18 @@ Create a new environment with a short name:
 azd env new <envName>
 ```
 
+#### Pre-package Evaluator Addon
+
+By default, the [evaluator addon](./functions/addons/README.md) deployment is enabled. And for `azd up` to work properly, we must pre-package the addon, for that from the project root run:
+```bash
+./functions/addons/package_addon.sh evaluator
+```
+
+To disable the custom evaluator addon run:
+```bash
+azd env set ENABLE_EVALUATOR_ADDON false
+```
+
 #### Azure OpenAI Configuration
 
 During deployment (`azd up`), you'll be prompted to select an Azure OpenAI model, capacity, and deployment type. You can review available models at the [Azure OpenAI Service models documentation](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/how-to/reasoning).
@@ -89,6 +101,25 @@ azd env set EXISTING_AZURE_OPENAI_ENDPOINT "https://your-openai.openai.azure.com
 azd env set EXISTING_AZURE_OPENAI_KEY "your-api-key"
 ```
 
+#### Setup First Admin User
+
+You can bootstrap the first admin user automatically during deployment by setting these azd environment values:
+
+```bash
+azd env set ROOT_ADMIN_EMAIL "admin@example.com"
+azd env set ROOT_ADMIN_NAME "Admin User"
+azd env set ROOT_ADMIN_PASSWORD "<strong-password>"
+azd up
+```
+
+Once the application is running, you can:
+1. Navigate to your application URL: `$(azd env get-value API_BASE_URL)`
+2. Click "Sign in with Password" 
+3. Use the email/password you just created
+4. Access the admin panel to create additional users
+
+> **Note**: This only needs to be set once. Additional users can be created through the web interface by admin users.
+
 #### Optional Configuration
 
 **Regional Quota Flexibility**: If you have limited quota in your primary region, you can deploy specific resources to alternate regions:
@@ -119,9 +150,6 @@ azd env set AZURE_LOCATION westus2
 **Feature Flags**: Control optional components:
 
 ```sh
-# Disable evaluator addon to reduce deployment time
-azd env set ENABLE_EVALUATOR_ADDON false
-
 # Disable Azure Communication Services
 azd env set ENABLE_ACS false
 ```
@@ -130,6 +158,10 @@ azd env set ENABLE_ACS false
 
 Now that your environment is configured, you can deploy all necessary resources and infrastructure for the Healthcare AI Model Evaluator.
 
+```bash
+azd up
+```
+
 #### IP Filtering & Security Configuration
 
 <<<<<<< HEAD
@@ -264,7 +296,7 @@ azd up
 
 > [!WARNING]
 > **Legacy migration (Mongo → SQL API)**: Older deployments used **Cosmos DB API for MongoDB**. This repository now provisions **Cosmos DB SQL API** instead, which means **existing Mongo data is not automatically migrated**. Plan for a one-time migration or accept data loss when moving environments.
-
+>
 > **Export (old Mongo API only — not for local dev)**
 > - Use `mongoexport` against your old Cosmos Mongo API account (or any Mongo-compatible endpoint):
 >   ```sh
@@ -334,9 +366,9 @@ azd up
 
 To start the deployment process, run:
 
-   ```bash
-   azd up
-   ```
+```bash
+azd up
+```
 
 During deployment you will be prompted for any required variable not yet set, such as subscription, resource group and location.
 
@@ -355,9 +387,9 @@ This command will:
 
 ### Step 4: Build and Push Metrics Function Docker Image
 
-   This step is necessary because `azd` does not automatically build and push Docker images for Azure Functions.
+This step is necessary because `azd` does not automatically build and push Docker images for Azure Functions.
 
-   ```bash
+```bash
 # From the root folder, get the registry name and endpoint
 AZURE_CONTAINER_REGISTRY_NAME=$(azd env get-value AZURE_CONTAINER_REGISTRY_NAME)
 AZURE_CONTAINER_REGISTRY_ENDPOINT=$(azd env get-value AZURE_CONTAINER_REGISTRY_ENDPOINT)
@@ -369,14 +401,14 @@ cd functions
 az acr login --name $AZURE_CONTAINER_REGISTRY_NAME
 
 # Build the Docker image
-docker compose build medbench-metrics
+docker compose build haime-metrics
 
 # Tag the image for the registry
-docker tag functions-medbench-metrics:latest $AZURE_CONTAINER_REGISTRY_ENDPOINT/medbench-metrics:latest
+docker tag functions-haime-metrics:latest $AZURE_CONTAINER_REGISTRY_ENDPOINT/haime-metrics:latest
 
 # Push the image to Azure Container Registry
-docker push $AZURE_CONTAINER_REGISTRY_ENDPOINT/medbench-metrics:latest
-   ```
+docker push $AZURE_CONTAINER_REGISTRY_ENDPOINT/haime-metrics:latest
+```
 
 > [!NOTE]
 > After pushing the image, the Azure Function will automatically pull and deploy it. This may take a few minutes.
@@ -400,28 +432,6 @@ echo "Frontend: $(azd env get-value API_BASE_URL)
 echo "API: $(azd env get-value API_BASE_URL)/api"
 ```
 
-## Post-Deployment Setup
-
-### Create First Admin User
-
-You can bootstrap the first admin user automatically during deployment by setting these azd environment values:
-
-```bash
-azd env set ROOT_ADMIN_EMAIL "admin@example.com"
-azd env set ROOT_ADMIN_NAME "Admin User"
-azd env set ROOT_ADMIN_PASSWORD "<strong-password>"
-azd up
-```
-
-Once created, you can:
-1. Navigate to your application URL: `$(azd env get-value API_BASE_URL)/webapp`
-2. Click "Sign in with Password" 
-3. Use the email/password you just created
-4. Access the admin panel to create additional users
-
-> **Note**: This only needs to be set once. Additional users can be created through the web interface by admin users.
-
-
 ---
 
 ## Architecture Overview
@@ -551,7 +561,7 @@ For production healthcare environments, you should restrict access to your appli
 
 ### Integrate with Existing Azure Front Door
 
-Most healthcare organizations already have Azure Front Door with WAF configured. You can integrate MedBench behind your existing Front Door.
+Most healthcare organizations already have Azure Front Door with WAF configured. You can integrate HAIME behind your existing Front Door.
 
 #### Configure Container Apps for Front Door Integration
 
@@ -608,18 +618,18 @@ az network front-door backend-pool backend add \
 
 #### Update Front Door Routing Rules
 
-Configure routing to send MedBench traffic to the new backend:
+Configure routing to send HAIME traffic to the new backend:
 
 ```bash
-# Create routing rule for MedBench
+# Create routing rule for HAIME
 az network front-door routing-rule create \
   --front-door-name "your-existing-frontdoor" \
   --resource-group "your-frontdoor-rg" \
-  --name "medbench-routing" \
+  --name "haime-routing" \
   --frontend-endpoints "your-frontend" \
   --route-type Forward \
   --backend-pool "your-backend-pool" \
-  --patterns "/medbench/*" \
+  --patterns "/haime/*" \
   --accepted-protocols Https
 ```
 

README.md

@@ -55,10 +55,12 @@ For a complete overview of the Healthcare AI Model Evaluator platform, refer to
 ## Deployment
 
 > [!IMPORTANT]
-> See [DEPLOYMENT.md](./DEPLOYMENT.md) for complete deployment guide, configuration options, and troubleshooting.
+> See [DEPLOYMENT.md](./DEPLOYMENT.md) for complete deployment guide, pre-requisites, configuration options, and troubleshooting.
 
 ### Quick start
 
+Before starting with deployment, make sure you go through the [deployment pre-requisites](./DEPLOYMENT.md#pre-requisites).
+
 Initialize and deploy the complete Healthcare AI Model Evaluator platform using Azure Developer CLI (azd).
 
 ```bash
@@ -69,9 +71,14 @@ azd init -t microsoft/healthcare-ai-model-evaluator
 azd up
 ```
 
+To initialize the project targetting a specific branch, use the `--branch` flag:
+```bash
+azd init -t microsoft/healthcare-ai-model-evaluator --branch feature-branch
+```
+
 ### First-time Setup
 
-After deployment, bootstrap your first admin user by setting these azd environment values and re-running deployment:
+To bootstrap your first admin user by setting these azd environment values and re-running deployment:
 
 ```bash
 azd env set ROOT_ADMIN_EMAIL "admin@example.com"
@@ -94,9 +101,15 @@ npm run dev
 
 ### Backend Setup
 ```bash
-export AZURE_STORAGE_ENDPOINT=[Your Storage Account blob endpoint]
-export COSMOSDB_ENDPOINT=[Your Cosmos DB account endpoint]
-export COSMOSDB_DATABASE=[Your Cosmos DB database name]
+# For Linux/macOS
+export AZURE_STORAGE_CONNECTION_STRING="Your Storage Account connection string"
+export COSMOSDB_CONNECTION_STRING="Your mongodb connection string"
+export COSMOSDB_ENDPOINT="Your mongodb endpoint"
+
+# For PowerShell
+$env:AZURE_STORAGE_CONNECTION_STRING="Your Storage Account connection string"
+$env:COSMOSDB_CONNECTION_STRING="Your mongodb connection string"
+$env:COSMOSDB_ENDPOINT="Your mongodb endpoint"
 
 cd backend
 dotnet restore

functions/Dockerfile

@@ -10,7 +10,7 @@ ENV CONTAINER_VERSION="v8.1_fixed_verify"
 # RUN apt-get update && apt-get install -y git build-essential && rm -rf /var/lib/apt/lists/*
 RUN apt-get update && apt-get install -y --no-install-recommends git build-essential && rm -rf /var/lib/apt/lists/* && \
     # Create a non-root user for security
-    groupadd -r appuser && useradd -r -g appuser -d /home/site appuser && \
+    groupadd -r appuser && useradd -m -r -g appuser -d /home/site appuser && \
     # Create a virtual environment to isolate dependencies
     python -m venv /home/site/venv && \
     # Set ownership of the home directory

functions/Dockerfile.addon-dev

@@ -16,7 +16,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
     gcc \
     g++ \
     && rm -rf /var/lib/apt/lists/* \
-    && groupadd -r appuser && useradd -r -g appuser -d /home/site appuser \
+    && groupadd -r appuser && useradd -m -r -g appuser -d /home/site appuser \
     && chown -R appuser:appuser /home/site
 
 # Copy requirements folder and main requirements

infra/main.parameters.json

@@ -125,6 +125,15 @@
     "allowedWebIp": {
       "value": "${ALLOWED_WEB_IP=}"
     },
+    "rootAdminEmail": {
+      "value": "${ROOT_ADMIN_EMAIL=}"
+    },
+    "rootAdminName": {
+      "value": "${ROOT_ADMIN_NAME=}"
+    },
+    "rootAdminPassword": {
+      "value": "${ROOT_ADMIN_PASSWORD=}"
+    },
     "deploymentNetworking": {
       "value": "${DEPLOYMENT_NETWORKING=private}"
     },

infra/scripts/postprovision.ps1

@@ -65,19 +65,25 @@ if ($clientId -and $clientId -ne "placeholder-will-be-updated-by-script" -and $c
         if ($principalId -and $principalId -ne "null") {
             Write-Host "Container App managed identity principal ID: $principalId"
 
-            # Assign Cosmos DB role
+            # Assign Cosmos DB SQL data-plane role (Built-in Data Contributor)
+            # This is required for reading/writing data via managed identity
+            # Built-in role IDs: 00000000-0000-0000-0000-000000000001 (Reader), 00000000-0000-0000-0000-000000000002 (Contributor)
             $cosmosAccountName = $env:COSMOS_ACCOUNT_NAME
             if ($cosmosAccountName) {
-                Write-Host "Assigning Cosmos DB role to Container App managed identity..."
-                $cosmosRoleId = "5bd9cd88-fe45-4216-938b-f97437e15450" # Cosmos DB Account Reader Writer
-                az cosmosdb sql role assignment create `
-                    --account-name $cosmosAccountName `
-                    --resource-group $resourceGroupName `
-                    --principal-id $principalId `
-                    --role-definition-id $cosmosRoleId `
-                    --scope "/subscriptions/$($env:AZURE_SUBSCRIPTION_ID)/resourceGroups/$resourceGroupName/providers/Microsoft.DocumentDB/databaseAccounts/$cosmosAccountName" `
-                    --output none 2>$null || Write-Host "Role assignment may already exist"
-                Write-Host "✅ Cosmos DB role assignment completed"
+                Write-Host "Assigning Cosmos DB SQL data-plane role to Container App managed identity..."
+                $cosmosAccountId = az cosmosdb show --name $cosmosAccountName --resource-group $resourceGroupName --query "id" -o tsv 2>$null
+                if ($cosmosAccountId) {
+                    az cosmosdb sql role assignment create `
+                        --account-name $cosmosAccountName `
+                        --resource-group $resourceGroupName `
+                        --principal-id $principalId `
+                        --role-definition-id "$cosmosAccountId/sqlRoleDefinitions/00000000-0000-0000-0000-000000000002" `
+                        --scope $cosmosAccountId `
+                        --output none 2>$null || Write-Host "SQL role assignment may already exist"
+                    Write-Host "✅ Cosmos DB SQL data-plane role assignment completed"
+                } else {
+                    Write-Host "⚠️  Warning: Could not get Cosmos DB account ID"
+                }
             }
 
             # Assign Storage role
@@ -248,19 +254,25 @@ try {
     if ($principalId -and $principalId -ne "null") {
         Write-Host "Container App managed identity principal ID: $principalId"
 
-        # Assign Cosmos DB role
+        # Assign Cosmos DB SQL data-plane role (Built-in Data Contributor)
+        # This is required for reading/writing data via managed identity
+        # Built-in role IDs: 00000000-0000-0000-0000-000000000001 (Reader), 00000000-0000-0000-0000-000000000002 (Contributor)
         $cosmosAccountName = $env:COSMOS_ACCOUNT_NAME
         if ($cosmosAccountName) {
-            Write-Host "Assigning Cosmos DB role to Container App managed identity..."
-            $cosmosRoleId = "5bd9cd88-fe45-4216-938b-f97437e15450" # Cosmos DB Account Reader Writer
-            az cosmosdb sql role assignment create `
-                --account-name $cosmosAccountName `
-                --resource-group $resourceGroupName `
-                --principal-id $principalId `
-                --role-definition-id $cosmosRoleId `
-                --scope "/subscriptions/$($env:AZURE_SUBSCRIPTION_ID)/resourceGroups/$resourceGroupName/providers/Microsoft.DocumentDB/databaseAccounts/$cosmosAccountName" `
-                --output none 2>$null || Write-Host "Role assignment may already exist"
-            Write-Host "✅ Cosmos DB role assignment completed"
+            Write-Host "Assigning Cosmos DB SQL data-plane role to Container App managed identity..."
+            $cosmosAccountId = az cosmosdb show --name $cosmosAccountName --resource-group $resourceGroupName --query "id" -o tsv 2>$null
+            if ($cosmosAccountId) {
+                az cosmosdb sql role assignment create `
+                    --account-name $cosmosAccountName `
+                    --resource-group $resourceGroupName `
+                    --principal-id $principalId `
+                    --role-definition-id "$cosmosAccountId/sqlRoleDefinitions/00000000-0000-0000-0000-000000000002" `
+                    --scope $cosmosAccountId `
+                    --output none 2>$null || Write-Host "SQL role assignment may already exist"
+                Write-Host "✅ Cosmos DB SQL data-plane role assignment completed"
+            } else {
+                Write-Host "⚠️  Warning: Could not get Cosmos DB account ID"
+            }
         }
 
         # Assign Storage role

infra/scripts/postprovision.sh

@@ -203,17 +203,24 @@ PRINCIPAL_ID=$(az containerapp show --name "$CONTAINER_APP_NAME" --resource-grou
 if [ -n "$PRINCIPAL_ID" ] && [ "$PRINCIPAL_ID" != "null" ]; then
     echo "Container App managed identity principal ID: $PRINCIPAL_ID"
 
-    # Assign Cosmos DB role
+    # Assign Cosmos DB SQL data-plane role (Built-in Data Contributor)
+    # This is required for reading/writing data via managed identity
+    # Built-in role IDs: 00000000-0000-0000-0000-000000000001 (Reader), 00000000-0000-0000-0000-000000000002 (Contributor)
     if [ -n "$COSMOS_ACCOUNT_NAME" ]; then
-        echo "Assigning Cosmos DB role to Container App managed identity..."
-        az cosmosdb sql role assignment create \
-            --account-name "$COSMOS_ACCOUNT_NAME" \
-            --resource-group "$RESOURCE_GROUP_NAME" \
-            --principal-id "$PRINCIPAL_ID" \
-            --role-definition-id "5bd9cd88-fe45-4216-938b-f97437e15450" \
-            --scope "/subscriptions/$AZURE_SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP_NAME/providers/Microsoft.DocumentDB/databaseAccounts/$COSMOS_ACCOUNT_NAME" \
-            --output none 2>/dev/null || echo "Role assignment may already exist"
-        echo "✅ Cosmos DB role assignment completed"
+        echo "Assigning Cosmos DB SQL data-plane role to Container App managed identity..."
+        COSMOS_ACCOUNT_ID=$(az cosmosdb show --name "$COSMOS_ACCOUNT_NAME" --resource-group "$RESOURCE_GROUP_NAME" --query "id" -o tsv 2>/dev/null || echo "")
+        if [ -n "$COSMOS_ACCOUNT_ID" ]; then
+            az cosmosdb sql role assignment create \
+                --account-name "$COSMOS_ACCOUNT_NAME" \
+                --resource-group "$RESOURCE_GROUP_NAME" \
+                --principal-id "$PRINCIPAL_ID" \
+                --role-definition-id "${COSMOS_ACCOUNT_ID}/sqlRoleDefinitions/00000000-0000-0000-0000-000000000002" \
+                --scope "$COSMOS_ACCOUNT_ID" \
+                --output none 2>/dev/null || echo "SQL role assignment may already exist"
+            echo "✅ Cosmos DB SQL data-plane role assignment completed"
+        else
+            echo "⚠️  Warning: Could not get Cosmos DB account ID"
+        fi
     fi
 
     # Assign Storage role