Merge pull request #50 from Azure-Samples/scenario_improvements

Refactor scenarios and modules to improve readability

Author: jamesmcroft

.devcontainer/README.md

@@ -13,8 +13,6 @@ The following tools are included in the devcontainer:
 - **PowerShell Core** - Used for running deployment scripts for the necessary infrastructure.
 - **Azure CLI** - Used to managed the Azure resources.
 - **Azure Developer CLI** - Used to manage the Azure resources.
-- **.NET 8.0 SDK** - Used for .NET samples.
-- **Node 22.x** - Used for JavaScript samples.
 - **Python 3.12** - Used for Python samples.
 - **GitHub CLI** - Used to interact with the GitHub repository.
 - **Docker-in-Docker** - Used to run Docker containers from within the devcontainer.
@@ -30,9 +28,7 @@ The following VS Code extensions are included in the devcontainer:
 - **Bicep** - Bicep language support.
 - **Azure Tools** - Azure resource management support.
 - **PowerShell** - PowerShell language support.
-- **C# Dev Kit** - C# language support.
 - **GitHub Pull Requests** - GitHub pull request support.
-- **Polyglot Notebooks** - Jupyter notebook support for C# and PowerShell.
 
 ## Python Dependencies
 
@@ -44,7 +40,6 @@ The following Python dependencies are included in the devcontainer:
 - **azure-storage-blob** - To interact with the Azure Blob Storage service.
 - **ipycanvas** - To create interactive canvases in Jupyter notebooks.
 - **ipykernel** - To create Jupyter kernels.
-- **marker-pdf** - To extract text from PDF files as Markdown.
 - **matplotlib** - To create plots in Jupyter notebooks.
 - **notebook** - To create Jupyter notebooks.
 - **numpy** - To work with numerical data.
@@ -57,6 +52,5 @@ The following Python dependencies are included in the devcontainer:
 - **pytesseract** - To extract text from images.
 - **python-dotenv** - To load environment variables from a `.env` file.
 - **seaborn** - To create plots in Jupyter notebooks.
-- **surya-ocr** - To extract text from images.
+- **tabulate** - To create tables in Jupyter notebooks.
 - **tiktoken** - To calculate confidence scores for structured outputs using OpenAI's logprobs.
-- **transformers** - To interact with the Hugging Face Transformers library.

README.md

@@ -42,7 +42,6 @@ The techniques demonstrated take advance of various capabilities from each servi
 | --------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
 | [Data Extraction - Azure AI Document Intelligence + Azure OpenAI GPT-4o](./samples/extraction/text-based/document-intelligence-openai.ipynb)              | Demonstrates how to use Azure AI Document Intelligence pre-built layout and Azure OpenAI GPT models to extract structured data from documents.                                                            | Predominantly text-based documents such as invoices, receipts, and forms.                                                             |
 | [Data Extraction - Azure AI Document Intelligence + Phi-3.5 MoE](./samples/extraction/text-based/document-intelligence-phi.ipynb)                         | Demonstrates how to use Azure AI Document Intelligence pre-built layout and Microsoft's Phi-3 models to extract structured data from documents.                                                           | Predominantly text-based documents such as invoices, receipts, and forms.                                                             |
-| [Data Extraction - Marker/Surya + Azure OpenAI GPT-4o](./samples/extraction/text-based/marker-surya-openai.ipynb)                                         | Demonstrates how to use Marker/Surya and Azure OpenAI GPT models to extract structured data from documents.                                                                                               | Predominantly text-based documents such as invoices, receipts, and forms.                                                             |
 | [Data Extraction - Azure OpenAI GPT-4o with Vision](./samples/extraction/vision-based/openai.ipynb)                                                       | Demonstrates how to use Azure OpenAI GPT-4o and GPT-4o-mini models to extract structured data from documents using their built-in vision capabilities.                                                    | Complex documents with a mix of text and images, including diagrams, signatures, selection marks, etc. such as reports and contracts. |
 | [Data Extraction - Comprehensive Azure AI Document Intelligence + Azure OpenAI GPT-4o with Vision](./samples/extraction/vision-based/comprehensive.ipynb) | Demonstrates how to improve the accuracy and confidence in extracting structured data from documents by combining Azure AI Document Intelligence and Azure OpenAI GPT-4o models with vision capabilities. | Any structured or unstructured document type.                                                                                         |
 | [Classification - Azure OpenAI GPT-4o with Vision](./samples/classification/openai.ipynb)                                                                 | Demonstrates how to use Azure OpenAI GPT-4o and GPT-4o-mini models to classify documents using their built-in vision capabilities.                                                                        | Processing multiple documents types or documents with varying purposes, such as contracts, legal documents, and emails.               |

Setup-Environment.ps1

@@ -1,26 +1,10 @@
-<#
-.SYNOPSIS
-    Deploys the infrastructure and applications required to run the solution.
-.PARAMETER DeploymentName
-	The name of the deployment.
-.PARAMETER Location
-    The location of the deployment.
-.PARAMETER SkipInfrastructure
-    Whether to skip the infrastructure deployment. Requires InfrastructureOutputs.json to exist in the infra directory.
-.EXAMPLE
-    .\Setup-Environment.ps1 -DeploymentName 'ai-document-data-extraction' -Location 'eastus' -SkipInfrastructure $false
-.NOTES
-    Author: James Croft
-#>
-
 param
 (
     [Parameter(Mandatory = $true)]
     [string]$DeploymentName,
     [Parameter(Mandatory = $true)]
     [string]$Location,
-    [Parameter(Mandatory = $true)]
-    [string]$SkipInfrastructure
+    [switch]$WhatIf
 )
 
 function Set-ConfigurationFileVariable($configurationFile, $variableName, $variableValue) {
@@ -36,19 +20,14 @@ function Set-ConfigurationFileVariable($configurationFile, $variableName, $varia
 
 Write-Host "Starting environment setup..."
 
-if ($SkipInfrastructure -eq '$false' -or -not (Test-Path -Path './infra/InfrastructureOutputs.json')) {
-    Write-Host "Deploying infrastructure..."
-    $InfrastructureOutputs = (./infra/Deploy-Infrastructure.ps1 `
-            -DeploymentName $DeploymentName `
-            -Location $Location)
-}
-else {
-    Write-Host "Skipping infrastructure deployment. Using existing outputs..."
-    $InfrastructureOutputs = Get-Content -Path './infra/InfrastructureOutputs.json' -Raw | ConvertFrom-Json
-}
+Write-Host "Deploying infrastructure..."
+$InfrastructureOutputs = (./infra/Deploy-Infrastructure.ps1 `
+        -DeploymentName $DeploymentName `
+        -Location $Location `
+        -WhatIf:$WhatIf)
 
 if (-not $InfrastructureOutputs) {
-    Write-Error "Infrastructure deployment outputs are not available. Exiting..."
+    Write-Error "Failed to deploy infrastructure."
     exit 1
 }
 

infra/Deploy-Infrastructure.ps1

@@ -1,43 +1,65 @@
-<#
-.SYNOPSIS
-    Deploys the core infrastructure for the AI Document Data Extraction demos to an Azure subscription.
-.DESCRIPTION
-    This script initiates the deployment of the main.bicep template to the current default Azure subscription,
-    determined by the Azure CLI. The deployment name and location are required parameters.
-.PARAMETER DeploymentName
-    The name of the deployment to create in an Azure subscription.
-.PARAMETER Location
-    The location to deploy the Azure resources to.
-.EXAMPLE
-    .\Deploy-Infrastructure.ps1 -DeploymentName "ai-document-data-extraction" -Location "eastus"
-.NOTES
-    Author: James Croft
-#>
-
 param
 (
     [Parameter(Mandatory = $true)]
     [string]$DeploymentName,
     [Parameter(Mandatory = $true)]
-    [string]$Location
+    [string]$Location,
+    [switch]$WhatIf
 )
 
 Write-Host "Starting infrastructure deployment..."
 
 Push-Location -Path $PSScriptRoot
 
-az --version
-
-$userPrincipalId = ((az rest --method GET --uri "https://graph.microsoft.com/v1.0/me") | ConvertFrom-Json).id
-
-$DeploymentOutputs = (az deployment sub create --name $DeploymentName --location $Location --template-file './main.bicep' `
+$principalId = (az ad signed-in-user show --query id -o tsv)
+$identity = @{ 
+    "principalId"   = "$principalId"
+    "principalType" = "User" 
+} 
+$identityArray = ConvertTo-Json @($identity) -Depth 5 -Compress
+     
+if ($whatIf) {
+    Write-Host "Previewing Azure infrastructure deployment. No changes will be made."
+    
+    $result = (az deployment sub what-if `
+            --name $deploymentName `
+            --location $location `
+            --template-file './main.bicep' `
+            --parameters './main.parameters.json' `
+            --parameters workloadName=$deploymentName `
+            --parameters location=$location `
+            --parameters identities=$identityArray `
+            --no-pretty-print) | ConvertFrom-Json
+    
+    if (-not $result) {
+        Write-Error "Azure infrastructure deployment preview failed."
+        exit 1
+    }
+    
+    Write-Host "Azure infrastructure deployment preview succeeded."
+    $result.changes | Format-List
+    exit
+}
+    
+$deploymentOutputs = (az deployment sub create `
+        --name $deploymentName `
+        --location $location `
+        --template-file './main.bicep' `
         --parameters './main.parameters.json' `
-        --parameters workloadName=$DeploymentName `
-        --parameters location=$Location `
-        --parameters userPrincipalId=$userPrincipalId `
+        --parameters workloadName=$deploymentName `
+        --parameters location=$location `
+        --parameters identities=$identityArray `
         --query properties.outputs -o json) | ConvertFrom-Json
-$DeploymentOutputs | ConvertTo-Json | Out-File -FilePath './InfrastructureOutputs.json' -Encoding utf8
+    
+if (-not $deploymentOutputs) {
+    Write-Error "Azure infrastructure deployment failed."
+    exit 1
+}
+    
+Write-Host "Azure infrastructure deployment succeeded."
+$deploymentOutputs | Format-List
+$deploymentOutputs | ConvertTo-Json | Out-File -FilePath './InfrastructureOutputs.json' -Encoding utf8
 
 Pop-Location
-
-return $DeploymentOutputs
+    
+return $deploymentOutputs

infra/ai_ml/ai-hub.bicep

@@ -39,7 +39,7 @@ param keyVaultId string
 @description('ID for the Application Insights associated with the AI Hub.')
 param applicationInsightsId string
 @description('ID for the Container Registry associated with the AI Hub.')
-param containerRegistryId string
+param containerRegistryId string?
 @description('ID for the Managed Identity associated with the AI Hub. Defaults to the system-assigned identity.')
 param identityId string?
 @description('Name for the AI Services resource to connect to.')