You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
title: Manage Azure HDInsight clusters using Azure CLI
3
-
description: Learn how to use the Azure CLI to manage Azure HDInsight clusters. Cluster types include Apache Hadoop, Spark, HBase, Kafka, Interactive Query.
3
+
description: Learn how to use the Azure CLI to manage Azure HDInsight clusters. Cluster types include Apache Hadoop, Spark, HBase, Kafka, and Interactive Query.
4
4
ms.service: azure-hdinsight
5
5
ms.topic: conceptual
6
6
ms.custom: hdinsightactive, devx-track-azurecli
7
7
ms.date: 12/02/2024
8
8
---
9
9
10
-
# Manage Azure HDInsight clusters using Azure CLI
10
+
# Manage Azure HDInsight clusters by using the Azure CLI
Learn how to use [Azure CLI](/cli/azure/) to manage Azure HDInsight clusters. The Azure CLI is Microsoft's cross-platform command-line experience for managing Azure resources.
14
+
Learn how to use the [Azure CLI](/cli/azure/) to manage Azure HDInsight clusters. The Azure CLI is the Microsoftcommand-line interface. You can use this cross-platform experience to manage Azure resources.
15
15
16
16
If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/?WT.mc_id=A261C142F) before you begin.
17
17
18
18
## Prerequisites
19
19
20
20
* Azure CLI. If you haven't installed the Azure CLI, see [Install the Azure CLI](/cli/azure/install-azure-cli) for steps.
21
-
22
-
* An Apache Hadoop cluster on HDInsight. See [Get Started with HDInsight on Linux](hadoop/apache-hadoop-linux-tutorial-get-started.md).
21
+
* An Apache Hadoop cluster on HDInsight. See [Get started with HDInsight on Linux](hadoop/apache-hadoop-linux-tutorial-get-started.md).
23
22
24
23
## Connect to Azure
25
24
26
-
Sign in to your Azure subscription. If you plan to use Azure Cloud Shell, then select **Try it** in the upper-right corner of the code block. Else, enter the following command:
25
+
Sign in to your Azure subscription. If you plan to use Azure Cloud Shell, select **Try it** in the upper-right corner of the code block. Otherwise, enter the following command:
27
26
28
27
```azurecli-interactive
29
28
az login
@@ -34,7 +33,7 @@ az login
34
33
35
34
## List clusters
36
35
37
-
Use [az HDInsight list](/cli/azure/hdinsight#az-hdinsight-list) to list clusters. Edit the following commands by replacing `RESOURCE_GROUP_NAME` with the name of your resource group, then enter the commands:
36
+
Use [az HDInsight list](/cli/azure/hdinsight#az-hdinsight-list) to list clusters. Edit the following commands by replacing `RESOURCE_GROUP_NAME` with the name of your resource group, and then enter the commands:
38
37
39
38
```azurecli-interactive
40
39
# List all clusters in the current subscription
@@ -52,39 +51,39 @@ az hdinsight list --resource-group RESOURCE_GROUP_NAME --query "[].{clusterName:
52
51
53
52
## Show cluster
54
53
55
-
Use [az HDInsight show](/cli/azure/hdinsight#az-hdinsight-show) to show information for a specified cluster. Edit the following command by replacing `RESOURCE_GROUP_NAME`, and `CLUSTER_NAME` with the relevant information, then enter the command:
54
+
Use [az HDInsight show](/cli/azure/hdinsight#az-hdinsight-show) to show information for a specified cluster. Edit the following command by replacing `RESOURCE_GROUP_NAME` and `CLUSTER_NAME` with the relevant information. Then enter the command:
56
55
57
56
```azurecli-interactive
58
57
az hdinsight show --resource-group RESOURCE_GROUP_NAME --name CLUSTER_NAME
59
58
```
60
59
61
60
## Delete clusters
62
61
63
-
Use [az HDInsight delete](/cli/azure/hdinsight#az-hdinsight-delete) to delete a specified cluster. Edit the following command by replacing `RESOURCE_GROUP_NAME`, and `CLUSTER_NAME` with the relevant information, then enter the command:
62
+
Use [az HDInsight delete](/cli/azure/hdinsight#az-hdinsight-delete) to delete a specified cluster. Edit the following command by replacing `RESOURCE_GROUP_NAME` and `CLUSTER_NAME` with the relevant information. Then enter the command:
64
63
65
64
```azurecli-interactive
66
65
az hdinsight delete --resource-group RESOURCE_GROUP_NAME --name CLUSTER_NAME
67
66
```
68
67
69
-
You can also delete a cluster by deleting the resource group that contains the cluster. Note, this deletes all the resources in the group including the default storage account.
68
+
You can also delete a cluster by deleting the resource group that contains the cluster. This action deletes all the resources in the group, including the default storage account.
70
69
71
70
```azurecli-interactive
72
71
az group delete --name RESOURCE_GROUP_NAME
73
72
```
74
73
75
74
## Scale clusters
76
75
77
-
Use [az HDInsight resize](/cli/azure/hdinsight#az-hdinsight-resize) to resize the specified HDInsight cluster to the specified size. Edit the following command by replacing `RESOURCE_GROUP_NAME`, and `CLUSTER_NAME` with the relevant information. Replace `WORKERNODE_COUNT` with the desired number of worker nodes for your cluster. For more information about scaling clusters, see [Scale HDInsight clusters](./hdinsight-scaling-best-practices.md). Enter the command:
76
+
Use [az HDInsight resize](/cli/azure/hdinsight#az-hdinsight-resize) to resize the specified HDInsight cluster to the specified size. Edit the following command by replacing `RESOURCE_GROUP_NAME` and `CLUSTER_NAME` with the relevant information. Replace `WORKERNODE_COUNT` with the number of worker nodes that you want for your cluster. For more information about scaling clusters, see [Scale HDInsight clusters](./hdinsight-scaling-best-practices.md). Enter the command:
78
77
79
78
```azurecli-interactive
80
79
az hdinsight resize --resource-group RESOURCE_GROUP_NAME --name CLUSTER_NAME --workernode-count WORKERNODE_COUNT
81
80
```
82
81
83
-
## Next steps
82
+
## Related content
84
83
85
-
In this article, you've learned how to perform different HDInsight cluster administrative tasks. To learn more, see the following articles:
84
+
In this article, you learned how to perform different HDInsight cluster administrative tasks. To learn more, see the following articles:
86
85
87
86
*[Manage Apache Hadoop clusters in HDInsight by using the Azure portal](hdinsight-administer-use-portal-linux.md)
88
87
*[Administer HDInsight by using Azure PowerShell](hdinsight-administer-use-powershell.md)
89
88
*[Get started with Azure HDInsight](hadoop/apache-hadoop-linux-tutorial-get-started.md)
90
-
*[Get started with Azure CLI](/cli/azure/get-started-with-azure-cli)
89
+
*[Get started with the Azure CLI](/cli/azure/get-started-with-azure-cli)
Learn how to manage HDInsight clusters using [HDInsight.NET SDK](/dotnet/api/overview/azure/hdinsight).
14
+
Learn how to manage Azure HDInsight clusters by using the[HDInsight.NET SDK](/dotnet/api/overview/azure/hdinsight).
15
15
16
-
**Prerequisites**
16
+
## Prerequisites
17
17
18
-
Before you begin this article, you must have the following:
18
+
Before you begin this article, you must have:
19
19
20
-
***An Azure subscription**. See [Get Azure free trial](https://azure.microsoft.com/documentation/videos/get-azure-free-trial-for-testing-hadoop-in-hdinsight/).
20
+
* An Azure account with an active subscription. If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/?WT.mc_id=A261C142F) before you begin.
// This is the GUID for the PowerShell client. Used for interactive logins in this example.
@@ -98,8 +98,7 @@ namespace HDInsightManagement
98
98
}
99
99
```
100
100
101
-
You shall see a prompt when you run this program. If you don't want to see the prompt, see [Create non-interactive authentication .NET HDInsight applications](hdinsight-create-non-interactive-authentication-dotnet-applications.md).
102
-
101
+
You see a prompt when you run this program. If you don't want to see the prompt, see [Create noninteractive authentication .NET HDInsight applications](hdinsight-create-non-interactive-authentication-dotnet-applications.md).
103
102
104
103
## List clusters
105
104
@@ -117,7 +116,7 @@ foreach (var name in results.Clusters) {
117
116
118
117
## Delete clusters
119
118
120
-
Use the following code snippet to delete a cluster synchronously or asynchronously:
119
+
Use the following code snippet to delete a cluster synchronously or asynchronously:
121
120
122
121
```csharp
123
122
_hdiManagementClient.Clusters.Delete("<Resource Group Name>", "<Cluster Name>");
@@ -126,21 +125,21 @@ _hdiManagementClient.Clusters.DeleteAsync("<Resource Group Name>", "<Cluster Nam
126
125
127
126
## Scale clusters
128
127
129
-
The cluster scaling feature allows you to change the number of worker nodes used by a cluster that is running in Azure HDInsight without having to re-create the cluster.
128
+
Use the cluster scaling feature to change the number of worker nodes used by a cluster that's running in HDInsight without having to re-create the cluster.
130
129
131
130
> [!NOTE]
132
-
> Only clusters with HDInsight version 3.1.3 or higher are supported. If you are unsure of the version of your cluster, you can check the Properties page. See[List and show clusters](hdinsight-administer-use-portal-linux.md#showClusters).
131
+
> Only clusters with HDInsight version 3.1.3 or higher are supported. If you're unsure of the version of your cluster, check the **Properties** page. For more information, see[List and show clusters](hdinsight-administer-use-portal-linux.md#showClusters).
133
132
134
-
The impact of changing the number of data nodes for each type of cluster supported by HDInsight:
133
+
The effect of changing the number of data nodes for each type of cluster supported by HDInsight:
135
134
136
135
* Apache Hadoop
137
136
138
-
You can seamlessly increase the number of worker nodes in a Hadoop cluster that is running without impacting any pending or running jobs. New jobs can also be submitted while the operation is in progress. Failures in a scaling operation are gracefully handled so that the cluster is always left in a functional state.
137
+
You can seamlessly increase the number of worker nodes in a Hadoop cluster that's running without affecting any pending or running jobs. You can also submit new jobs while the operation is in progress. Failures in a scaling operation are gracefully handled so that the cluster is always left in a functional state.
139
138
140
-
When a Hadoop cluster is scaled down by reducing the number of data nodes, some of the services in the cluster are restarted. This causes all running and pending jobs to fail at the completion of the scaling operation. You can, however, resubmit the jobs once the operation is complete.
139
+
When a Hadoop cluster is scaled down by reducing the number of data nodes, some of the services in the cluster are restarted. All running and pending jobs fail at the completion of the scaling operation. After the operation is finished, you can resubmit the jobs.
141
140
* Apache HBase
142
141
143
-
You can seamlessly add or remove nodes to your HBase cluster while it is running. Regional Servers are automatically balanced within a few minutes of completing the scaling operation. However, you can also manually balance the regional servers by logging into the headnode of cluster and running the following commands from a command prompt window:
142
+
You can seamlessly add or remove nodes to your HBase cluster while it's running. Regional servers are automatically balanced within a few minutes of completing the scaling operation. You can also manually balance the regional servers. Sign in to the head node of a cluster and run the following commands from a command prompt window:
144
143
145
144
146
145
```bash
@@ -151,11 +150,11 @@ The impact of changing the number of data nodes for each type of cluster support
151
150
152
151
## Update HTTP user credentials
153
152
154
-
It is the same procedure as Grant/revoke HTTP access. If the cluster has been granted the HTTP access, you must first revoke it. And then grant the access with new HTTP user credentials.
153
+
The update procedure is the same as the one you use to grant or revoke HTTP access. If the cluster was granted HTTP access, you must first revoke it. Then you can grant access with new HTTP user credentials.
155
154
156
155
## Find the default storage account
157
156
158
-
The following code snippet demonstrates how to get the default storage account name and the default storage account key for a cluster.
157
+
The following code snippet demonstrates how to get the default storage account name and key for a cluster.
159
158
160
159
```csharp
161
160
var results = _hdiManagementClient.Clusters.GetClusterConfigurations(<Resource Group Name>, <Cluster Name>, "core-site");
@@ -167,31 +166,22 @@ foreach (var key in results.Configuration.Keys)
167
166
168
167
## Submit jobs
169
168
170
-
**To submit MapReduce jobs**
171
-
172
-
See [Run MapReduce samples in HDInsight](hadoop/apache-hadoop-run-samples-linux.md).
173
-
174
-
**To submit Apache Hive jobs**
175
-
176
-
See [Run Apache Hive queries using .NET SDK](hadoop/apache-hadoop-use-hive-dotnet-sdk.md).
177
-
178
-
**To submit Apache Sqoop jobs**
179
-
180
-
See [Use Apache Sqoop with HDInsight](hadoop/apache-hadoop-use-sqoop-dotnet-sdk.md).
181
-
182
-
**To submit Apache Oozie jobs**
169
+
Learn how to submit jobsfor the following products:
183
170
184
-
See [Use Apache Oozie with Hadoop to define and run a workflow in HDInsight](hdinsight-use-oozie-linux-mac.md).
171
+
- **MapReduce**: [Run MapReduce samples in HDInsight](hadoop/apache-hadoop-run-samples-linux.md)
172
+
- **Apache Hive**: [Run Apache Hive queries by using the .NET SDK](hadoop/apache-hadoop-use-hive-dotnet-sdk.md)
173
+
- **Apache Sqoop**: [Use Apache Sqoop with HDInsight](hadoop/apache-hadoop-use-sqoop-dotnet-sdk.md)
174
+
- **Apache Oozie**: [Use Apache Oozie with Hadoop to define and run a workflow in HDInsight](hdinsight-use-oozie-linux-mac.md)
185
175
186
-
## Upload data to Azure Blob storage
176
+
## Upload data to Azure Blob Storage
187
177
188
-
See [Upload data to HDInsight][hdinsight-upload-data].
178
+
To upload data, see [Upload data to HDInsight][hdinsight-upload-data].
0 commit comments