|
| 1 | +--- |
| 2 | +title: Manage HDInsight on AKS clusters using PowerShell (Preview) |
| 3 | +description: Manage HDInsight on AKS clusters using PowerShell. |
| 4 | +ms.service: hdinsight-aks |
| 5 | +ms.topic: how-to |
| 6 | +ms.date: 12/11/2023 |
| 7 | +--- |
| 8 | +# Manage HDInsight on AKS clusters using PowerShell |
| 9 | + |
| 10 | +Azure PowerShell is a powerful scripting environment that you can use to control and automate the deployment and management of your workloads in Microsoft Azure. This document provides information about how to create a HDInsight on AKS cluster by using Azure PowerShell. It also includes an example script. |
| 11 | + |
| 12 | + |
| 13 | +## Prerequisites |
| 14 | + |
| 15 | +To create an HDInsight on AKS cluster by using Azure PowerShell, you must complete the following procedures: |
| 16 | +- [Install Azure PowerShell](/powershell/azure/install-azure-powershell) |
| 17 | +- Create an [Azure resource group](/azure/azure-resource-manager/management/manage-resource-groups-portal#create-resource-groups) |
| 18 | +- Create an [Azure Data Lase Store Gen2](/azure/storage/blobs/create-data-lake-storage-account) account |
| 19 | +- Create an [Azure Managed Identity](/entra/identity/managed-identities-azure-resources/qs-configure-portal-windows-vm) |
| 20 | + |
| 21 | + |
| 22 | + ## Set up Azure Environment with PowerShell |
| 23 | + |
| 24 | +The following script demonstrates how to set up an Azure environment with PowerShell: |
| 25 | + |
| 26 | +1. Open PowerShell |
| 27 | +1. Copy the following code |
| 28 | + 1. Install the module Az.HdInsightOnAks |
| 29 | + Install-Module -Name Az.HdInsightOnAks |
| 30 | + |
| 31 | + :::image type="content" source="./media/powershell-cluster-create/powershell.png" alt-text="Screenshot shows install the module HDInsight on AKS." lightbox="./media/powershell-cluster-create/powershell.png"::: |
| 32 | + 1. Log in to Azure account and set the default subscription ID |
| 33 | + Connect-AzAccount |
| 34 | + Set-AzContext -Subscription {your subscription ID} |
| 35 | + |
| 36 | + |
| 37 | +## Variables required in script |
| 38 | + |
| 39 | +- Cluster Name |
| 40 | +- Cluster Pool Name |
| 41 | +- Subscription ID |
| 42 | +- Resource Group Name |
| 43 | +- Region Name |
| 44 | +- Cluster Type |
| 45 | +- SKU |
| 46 | +- Worker Node count |
| 47 | +- MSI resource ID: |
| 48 | + ``` |
| 49 | + /subscriptions/<subscription ID>/resourcegroups/<resource group name>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/<Managed identity name>" |
| 50 | + ``` |
| 51 | +- MSI client ID |
| 52 | +- MSI object ID |
| 53 | + |
| 54 | + :::image type="content" source="./media/powershell-cluster-create/overview.png" alt-text="Screenshot shows MSI object ID." lightbox="./media/powershell-cluster-create/overview.png"::: |
| 55 | +- Microsoft Entra user ID: [Find tenant ID, domain name, user object ID - Partner Center | Microsoft Learn](/partner-center/find-ids-and-domain-names) |
| 56 | +- [HDInsight on AKS VM list](/azure/hdinsight-aks/virtual-machine-recommendation-capacity-planning) |
| 57 | + |
| 58 | +### Create HDInsight On AKS cluster pool |
| 59 | + |
| 60 | +Copy the following code to PowerShell |
| 61 | +``` |
| 62 | +$clusterPoolName="<your cluster pool name>"; |
| 63 | +
|
| 64 | +$resourceGroupName="<your resource group name>"; |
| 65 | +
|
| 66 | +$location="West US 2"; |
| 67 | +
|
| 68 | +$vmSize="Standard_E4s_v3" |
| 69 | +``` |
| 70 | +**Get the available cluster pool version** |
| 71 | +``` |
| 72 | +$clusterPoolVersion=Get-AzHdInsightOnAksAvailableClusterPoolVersion -Location $location |
| 73 | +
|
| 74 | +Write-Output "Start to create cluster pool..." |
| 75 | +
|
| 76 | +$clusterPoolResult=New-AzHdInsightOnAksClusterPool -Name $clusterPoolName -ResourceGroupName $resourceGroupName -Location $location -VmSize $vmSize -ClusterPoolVersion $clusterPoolVersion.ClusterPoolVersionValue |
| 77 | +``` |
| 78 | + |
| 79 | +Write-Output "Created cluster pool with name $($clusterPoolResult.Name) successfully" |
| 80 | + |
| 81 | +### Create HDInsight On AKS cluster under existing cluster pool |
| 82 | + |
| 83 | +Here, we are going to create cluster under the cluster pool created in the previous step |
| 84 | + |
| 85 | +Run the following code in PowerShell: |
| 86 | + |
| 87 | +**Create Trino Cluster** |
| 88 | +``` |
| 89 | +
|
| 90 | +
|
| 91 | +$clusterPoolName=$“{Cluster Pool Name}”; |
| 92 | +
|
| 93 | +$resourceGroupName=$“{Resource Group Name}”; |
| 94 | +
|
| 95 | +$location=“{Region Name}”; |
| 96 | +
|
| 97 | +$clusterType="{Trino}"; |
| 98 | +
|
| 99 | +Get available cluster version based the command Get-AzHdInsightOnAksAvailableClusterVersion |
| 100 | +
|
| 101 | +$clusterVersion= (Get-AzHdInsightOnAksAvailableClusterVersion -Location $location | Where-Object {$_.ClusterType -eq $clusterType})[0] |
| 102 | +$msiResourceId="<your user msi resource id>"; |
| 103 | +$msiClientId="<your user msi client id>"; |
| 104 | +$msiObjectId="<your msi object id>"; |
| 105 | +$userId="<your Microsoft Entra user id>"; |
| 106 | +``` |
| 107 | +**Create node profile** |
| 108 | +``` |
| 109 | +$vmSize="Standard_D8d_v5"; // {Mention the SKU name} |
| 110 | +$workerCount=5; // {Mention the SKU count} |
| 111 | +$nodeProfile = New-AzHdInsightOnAksNodeProfileObject -Type Worker -Count $workerCount -VMSize $vmSize |
| 112 | +$clusterName="<your cluster name>"; |
| 113 | +
|
| 114 | +Write-Output "Start to create cluster..." |
| 115 | +
|
| 116 | +$clusterResult=New-AzHdInsightOnAksCluster -Name $clusterName ` |
| 117 | +
|
| 118 | + -PoolName $clusterPoolName ` |
| 119 | +
|
| 120 | + -ResourceGroupName $resourceGroupName ` |
| 121 | +
|
| 122 | + -Location $location ` |
| 123 | +
|
| 124 | + -ClusterType $clusterType ` |
| 125 | +
|
| 126 | + -ClusterVersion $clusterVersion.ClusterVersionValue ` |
| 127 | +
|
| 128 | + -OssVersion $clusterVersion.OssVersion ` |
| 129 | +
|
| 130 | + -AssignedIdentityResourceId $msiResourceId ` |
| 131 | +
|
| 132 | + -AssignedIdentityClientId $msiClientId ` |
| 133 | +
|
| 134 | + -AssignedIdentityObjectId $msiObjectId ` |
| 135 | +
|
| 136 | + -ComputeProfileNode $nodeProfile ` |
| 137 | +
|
| 138 | + -AuthorizationUserId $userId |
| 139 | +
|
| 140 | + ``` |
| 141 | +Write-Output "Created cluster with name $($clusterResult.Name) successfully" |
| 142 | + |
| 143 | +**Get the cluster with cluster name:** |
| 144 | + |
| 145 | +``` |
| 146 | +Get-AzHdInsightOnAksCluster -ResourceGroupName $resourceGroupName -PoolName $clusterPoolName -Name $clusterName |
| 147 | +``` |
| 148 | + |
| 149 | +**List the clusters under the cluster pool** |
| 150 | + |
| 151 | +``` |
| 152 | +Get-AzHdInsightOnAksCluster -ResourceGroupName $resourceGroupName -PoolName $clusterPoolName |
| 153 | + ``` |
| 154 | + |
| 155 | +**Delete the cluster with cluster name** |
| 156 | + |
| 157 | +``` |
| 158 | +Remove-AzHdInsightOnAksCluster -Name $clusterName -PoolName $clusterpoolName -ResourceGroupName $resourceGroupName |
| 159 | +``` |
| 160 | + |
| 161 | +## Next steps |
| 162 | + |
| 163 | +Now you created an HDInsight on AKS cluster, use the following resources to learn how to work with your cluster. |
| 164 | + |
| 165 | +To customize and manage your HDInsight on AKS cluster, refer the following documentation: |
| 166 | +- Az.HdInsightOnAks module is available in PowerShell gallery: [https://www.powershellgallery.com/packages/Az.HdInsightOnAks/0.1.0](https://www.powershellgallery.com/packages/Az.HdInsightOnAks/0.1.0 ) |
| 167 | +- Publicly available [HDInsight On AKS PowerShell module](/powershell/module/az.hdinsightonaks/#hdinsightonaks) doc |
| 168 | + |
| 169 | + |
| 170 | + |
| 171 | + |
0 commit comments