Merge remote-tracking branch 'upstream/main'

Author: Xiting Zhang

eng/common/instructions/azsdk-tools/create-sdk-locally.instructions.md

@@ -0,0 +1,119 @@
+---
+description: "Guide the user to generate and build SDKs locally for a TypeSpec based API spec"
+---
+
+# Goal
+
+Help the user generate and build SDKs locally from TypeSpec API specifications using the `azure-sdk-mcp` tools.
+
+---
+
+## Part A: Generate SDK Locally
+
+### Step 1: Outline workflow
+
+**Goal**: Ensure the user understands the overall SDK generation and build process before starting.  
+**Actions**:
+
+- Present the high-level steps involved in generating and building SDK locally:
+  1. Select target language
+  2. Verify SDK repository
+  3. Validate repository path
+  4. Identify path to configuration file
+  5. Generate SDK using `azsdk_package_generate_code` MCP tool
+  6. Identify SDK project path
+  7. Build/Compile SDK using `azsdk_package_build_code` MCP tool
+- Ask the user to confirm readiness to proceed.
+
+---
+
+### Step 2: Select language
+
+**Goal**: Confirm the target language for SDK generation.  
+**Actions**:
+
+- Prompt user to choose one of the supported languages:
+  - .NET
+  - Java
+  - JavaScript
+  - Python
+  - Go
+- Validate input against the allowed list.
+
+---
+
+### Step 3: Verify SDK repository
+
+**Goal**: Ensure the correct Azure SDK language repository is available locally.
+**Actions**:
+
+- Prompt user to provide the path to their **locally cloned repository** for the selected language.
+- Note: The **local folder name can be arbitrary**, but the repository must have originated from one of the official Azure SDK repositories:
+  - `azure-sdk-for-net` (.NET)
+  - `azure-sdk-for-java` (Java)
+  - `azure-sdk-for-js` (JavaScript)
+  - `azure-sdk-for-python` (Python)
+  - `azure-sdk-for-go` (Go)
+- If the repository is not cloned → instruct user to clone the appropriate remote repository from GitHub.
+- MCP tool will automatically validate the remote origin and repository structure.
+
+---
+
+### Step 4: Validate repository path
+
+**Actions**:
+
+- Check if the provided repository path exists and matches the selected SDK language repository.
+- If invalid → prompt user to re-enter a valid path.
+
+---
+
+### Step 5: Identify path to configuration file
+
+**Goal**: Determine the correct path to the TypeSpec configuration file based on the working context.  
+**Actions**:
+
+- **Scenario A: Working in a repository cloned from `azure-rest-api-specs`**
+  - Identify the path to `tspconfig.yaml` (local path or HTTPS URL).
+  - The local folder name can be arbitrary; the MCP tool will validate that the remote origin URL points to the official `azure-rest-api-specs` repository.
+  - Example paths (pointing directly to tspconfig.yaml):
+    - `/home/usr/azure-rest-api-specs/specification/contosowidgetmanager/Contoso.Management/tspconfig.yaml`
+    - `https://github.com/Azure/azure-rest-api-specs/blob/4af373fc5826cf5a2365a20dde01c4b2efde48f0/specification/contosowidgetmanager/Contoso.Management/tspconfig.yaml`
+
+- **Scenario B: Working in one of the official Azure SDK language repositories**  
+  (i.e., originally cloned from `azure-sdk-for-net`, `azure-sdk-for-java`, `azure-sdk-for-js`, `azure-sdk-for-python`, `azure-sdk-for-go`)
+  - Identify the path to `tsp-location.yaml`.
+  - The local folder name can be arbitrary; MCP tool will validate the remote origin URL.
+  - Example path:
+    `/home/usr/azure-sdk-for-net/sdk/contoso/Azure.ResourceManager.Contoso/tsp-location.yaml`
+
+---
+
+### Step 6: Generate SDK
+
+**Actions**:
+
+- Run `azsdk_package_generate_code` MCP tool to generate the SDK locally.
+
+---
+
+## Part B: Build / Compile SDK Locally
+
+### Step 1: Identify SDK project path
+
+**Goal**: Locate the generated SDK project directory for building/compiling.
+**Actions**:
+
+- Find the project directory inside the selected Azure SDK language repository.
+- Typical structure:  
+  `sdk/{service-name}/{package-name}/`
+- Example:  
+  `/path/to/azure-sdk-for-net/contoso/Azure.ResourceManager.Contoso/`
+
+---
+
+### Step 2: Build/Compile the SDK
+
+**Actions**:
+
+- Run `azsdk_package_build_code` MCP tool to compile the SDK in the identified project directory.

eng/common/instructions/azsdk-tools/local-sdk-workflow.instructions.md

@@ -5,13 +5,16 @@ description: 'Identify languages configured in the TypeSpec project and add it t
 **Goal**: Identify languages configured in the TypeSpec project and generate the json object with language and package name.
 1. Identify the language emitter configuration in the `tspconfig.yaml` file in the TypeSpec project root.
 2. Identify the package name or namespace for each language emitter.
+   - For Java and Python, use `emitter-output-dir` for package name if it exists. Otherwise use `package-dir` to get the package name as fallback approach.
+   - For .NET, use namespace property to get package name.
+   - For JavaScript, use `packagedetails:name` property to get package name.
+   - For Go, use module name and remove `github.com/Azure/azure-sdk-for-go/` to get package name
 3. Map the language name in emitter to one of the following in Pascal case(except .NET):
    - .NET
    - Java
    - Python
    - JavaScript
    - Go
-4. Remove `github.com/Azure/azure-sdk-for-go/` from Go package name.
 4. Create a JSON array object with the following structure:
    ```json
    [

eng/common/instructions/azsdk-tools/sdk-details-in-release-plan.instructions.md

@@ -57,10 +57,12 @@ Wait for the user to respond with a confirmation before proceeding to Step 1. Us
 **Goal**: Determine how to generate SDKs
 **Actions**:
 1. Present options: "How would you like to generate SDKs?"
-    - Option A: "Generate SDK locally". This is currently supported only for Python. Do not recommend this for other languages.
+    - Option A: "Generate SDK locally".
     - Option B: "Use SDK generation pipeline"
 2. Based on selection:
-    - If Option A: Refer to #file:create-sdk-locally.instructions.md and then proceed to Step 6
+    - If Option A: 
+        - Follow #file:./local-sdk-workflow.instructions.md to generate and compile the SDK.
+        - After SDK has been generated, to continue the SDK release, users can create the SDK pull request manually then proceed to Step 9.
     - If Option B: Continue to Step 6
 **Success Criteria**: SDK generation method selected
 

eng/common/instructions/azsdk-tools/typespec-to-sdk.instructions.md

@@ -38,6 +38,32 @@ function Get-PurgeableGroupResources {
     $purgeableResources += $deletedKeyVaults
   }
 
+  Write-Verbose "Retrieving AI resources from resource group $ResourceGroupName"
+
+  # Get AI resources that will go into soft-deleted state when the resource group is deleted
+  $subscriptionId = (Get-AzContext).Subscription.Id
+  $aiResources = @()
+
+  # Get active Cognitive Services accounts from the resource group
+  $response = Invoke-AzRestMethod -Method GET -Path "/subscriptions/$subscriptionId/resourceGroups/$ResourceGroupName/providers/Microsoft.CognitiveServices/accounts?api-version=2024-10-01" -ErrorAction Ignore
+  if ($response.StatusCode -ge 200 -and $response.StatusCode -lt 300 -and $response.Content) {
+    $content = $response.Content | ConvertFrom-Json
+    
+    foreach ($r in $content.value) {
+      $aiResources += [pscustomobject] @{
+        AzsdkResourceType = "Cognitive Services ($($r.kind))"
+        AzsdkName         = $r.name
+        Name              = $r.name
+        Id                = $r.id
+      }
+    }
+  }
+
+  if ($aiResources) {
+    Write-Verbose "Found $($aiResources.Count) AI resources to potentially purge after resource group deletion."
+    $purgeableResources += $aiResources
+  }
+
   return $purgeableResources
 }
 
@@ -94,6 +120,29 @@ function Get-PurgeableResources {
   }
   catch { }
 
+  Write-Verbose "Retrieving deleted Cognitive Services accounts from subscription $subscriptionId"
+
+  # Get deleted Cognitive Services accounts for the current subscription.
+  $response = Invoke-AzRestMethod -Method GET -Path "/subscriptions/$subscriptionId/providers/Microsoft.CognitiveServices/deletedAccounts?api-version=2024-10-01" -ErrorAction Ignore
+  if ($response.StatusCode -ge 200 -and $response.StatusCode -lt 300 -and $response.Content) {
+    $content = $response.Content | ConvertFrom-Json
+
+    $deletedCognitiveServices = @()
+    foreach ($r in $content.value) {
+      $deletedCognitiveServices += [pscustomobject] @{
+        AzsdkResourceType = "Cognitive Services ($($r.kind))"
+        AzsdkName         = $r.name
+        Name              = $r.name
+        Id                = $r.id
+      }
+    }
+
+    if ($deletedCognitiveServices) {
+      Write-Verbose "Found $($deletedCognitiveServices.Count) deleted Cognitive Services accounts to potentially purge."
+      $purgeableResources += $deletedCognitiveServices
+    }
+  }
+
   return $purgeableResources
 }
 
@@ -117,33 +166,37 @@ filter Remove-PurgeableResources {
   }
 
   $subscriptionId = (Get-AzContext).Subscription.Id
+  $verboseFlag = $VerbosePreference -eq 'Continue'
 
   foreach ($r in $Resource) {
-    Log "Attempting to purge $($r.AzsdkResourceType) '$($r.AzsdkName)'"
     switch ($r.AzsdkResourceType) {
       'Key Vault' {
         if ($r.EnablePurgeProtection) {
-          Write-Warning "Key Vault '$($r.VaultName)' has purge protection enabled and may not be purged until $($r.ScheduledPurgeDate)"
+          Write-Verbose "Key Vault '$($r.VaultName)' has purge protection enabled and may not be purged until $($r.ScheduledPurgeDate)" -Verbose:$verboseFlag
           continue
         }
 
+        Log "Attempting to purge $($r.AzsdkResourceType) '$($r.AzsdkName)'"
+
         # Use `-AsJob` to start a lightweight, cancellable job and pass to `Wait-PurgeableResoruceJob` for consistent behavior.
         Remove-AzKeyVault -VaultName $r.VaultName -Location $r.Location -InRemovedState -Force -ErrorAction Continue -AsJob `
         | Wait-PurgeableResourceJob -Resource $r -Timeout $Timeout -PassThru:$PassThru
       }
 
       'Managed HSM' {
         if ($r.EnablePurgeProtection) {
-          Write-Warning "Managed HSM '$($r.Name)' has purge protection enabled and may not be purged until $($r.ScheduledPurgeDate)"
+          Write-Verbose "Managed HSM '$($r.Name)' has purge protection enabled and may not be purged until $($r.ScheduledPurgeDate)" -Verbose:$verboseFlag
           continue
         }
 
+        Log "Attempting to purge $($r.AzsdkResourceType) '$($r.AzsdkName)'"
+
         # Use `GetNewClosure()` on the `-Action` ScriptBlock to make sure variables are captured.
         Invoke-AzRestMethod -Method POST -Path "/subscriptions/$subscriptionId/providers/Microsoft.KeyVault/locations/$($r.Location)/deletedManagedHSMs/$($r.Name)/purge?api-version=2023-02-01" -ErrorAction Ignore -AsJob `
         | Wait-PurgeableResourceJob -Resource $r -Timeout $Timeout -PassThru:$PassThru -Action {
           param ( $response )
           if ($response.StatusCode -ge 200 -and $response.StatusCode -lt 300) {
-            Write-Warning "Successfully requested that Managed HSM '$($r.Name)' be purged, but may take a few minutes before it is actually purged."
+            Write-Verbose "Successfully requested that Managed HSM '$($r.Name)' be purged, but may take a few minutes before it is actually purged." -Verbose:$verboseFlag
           }
           elseif ($response.Content) {
             $content = $response.Content | ConvertFrom-Json
@@ -155,6 +208,22 @@ filter Remove-PurgeableResources {
         }.GetNewClosure()
       }
 
+      { $_.StartsWith('Cognitive Services') }
+      {
+        Log "Attempting to purge $($r.AzsdkResourceType) '$($r.AzsdkName)'"
+        # Use `GetNewClosure()` on the `-Action` ScriptBlock to make sure variables are captured.
+        Invoke-AzRestMethod -Method DELETE -Path "$($r.id)?api-version=2024-10-01" -ErrorAction Ignore -AsJob `
+        | Wait-PurgeableResourceJob -Resource $r -Timeout $Timeout -PassThru:$PassThru -Action {
+          param ( $response )
+
+          if ($response.StatusCode -eq 200 -or $response.StatusCode -eq 202 -or $response.StatusCode -eq 204) {
+            Write-Verbose "Successfully purged $($r.AzsdkResourceType) '$($r.Name)'." -Verbose:$verboseFlag
+          } else {
+            Write-Warning "Failed purging $($r.AzsdkResourceType) '$($r.Name)' with status code $($response.StatusCode)."
+          }
+        }.GetNewClosure()
+      }
+
       default {
         Write-Warning "Cannot purge $($r.AzsdkResourceType) '$($r.AzsdkName)'. Add support to https://github.com/Azure/azure-sdk-tools/blob/main/eng/common/scripts/Helpers/Resource-Helpers.ps1."
       }

eng/common/scripts/Helpers/Resource-Helpers.ps1

@@ -0,0 +1,80 @@
+# TypeSpec Client Generator CLI
+
+This directory contains npm package definitions for `@azure-tools/typespec-client-generator-cli` (tsp-client) with pinned versions to ensure reproducible builds across environments.
+
+## Files
+
+- **`package.json`** - npm package definition with pinned tsp-client version
+- **`package-lock.json`** - Lock file ensuring exact dependency versions
+
+## Prerequisites
+
+- **Node.js** (with npm) - Required to install and run tsp-client
+
+## Installation
+
+### Install dependencies
+
+```bash
+# Navigate to this directory
+cd eng/common/tsp-client
+
+# Install dependencies
+npm ci
+```
+
+## Usage
+
+After installation, you can run tsp-client by navigating to the directory and using npm exec:
+
+```bash
+cd eng/common/tsp-client
+
+# Get help
+npm exec --no -- tsp-client --help
+
+# Check version
+npm exec --no -- tsp-client version
+
+# Generate client code
+npm exec --no -- tsp-client generate --output-dir ./generated
+
+# Initialize a new project
+npm exec --no -- tsp-client init --tsp-config ./tspconfig.yaml
+```
+
+## CI/CD Best Practices
+
+```bash
+cd eng/common/tsp-client
+npm ci
+npm exec --no -- tsp-client init --update-if-exists --tsp-config https://github.com/Azure/azure-rest-api-specs/blob/dee71463cbde1d416c47cf544e34f7966a94ddcb/specification/contosowidgetmanager/Contoso.WidgetManager/tspconfig.yaml
+```
+
+## Package Management
+
+### Automatic Updates via Dependabot
+
+Dependabot is configured to automatically check for updates to `@azure-tools/typespec-client-generator-cli` daily and create pull requests with updated `package.json` and `package-lock.json` files. This ensures the package stays current with the latest versions while maintaining security through the PR review process.
+
+### Manual Version Updates
+
+If you need to manually update the tsp-client version:
+
+1. Edit `package.json` to update the version:
+
+   ```json
+   {
+     "dependencies": {
+       "@azure-tools/typespec-client-generator-cli": "0.28.1"
+     }
+   }
+   ```
+
+2. Update the lock file:
+
+   ```bash
+   npm install
+   ```
+
+3. Commit both `package.json` and `package-lock.json`