Skip to content

Commit 14b0042

Browse files
authored
Merge pull request #107665 from dagiro/hadoop1
hadoop1
2 parents 375e4c9 + 90428e4 commit 14b0042

File tree

2 files changed

+41
-27
lines changed

2 files changed

+41
-27
lines changed

articles/hdinsight/TOC.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -418,8 +418,8 @@
418418
items:
419419
- name: Create Apache Hadoop cluster - Portal
420420
href: ./hadoop/apache-hadoop-linux-create-cluster-get-started-portal.md
421-
- name: Create Apache Hadoop cluster - Template
422-
displayName: resource manager template, arm template, resource manager group
421+
- name: Create Apache Hadoop cluster - ARM Template
422+
displayName: Resource Manager
423423
href: ./hadoop/apache-hadoop-linux-tutorial-get-started.md
424424
- name: Tutorials
425425
items:
Lines changed: 39 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -1,43 +1,59 @@
11
---
2-
title: 'Quickstart: Apache Hadoop & Resource Manager - Azure HDInsight'
2+
title: 'Quickstart: Create Apache Hadoop cluster in Azure HDInsight using Resource Manager template'
33
description: In this quickstart, you create Apache Hadoop cluster in Azure HDInsight using Resource Manager template
44
author: hrasheed-msft
55
ms.author: hrasheed
66
ms.reviewer: jasonh
77
ms.service: hdinsight
88
ms.topic: quickstart
9-
ms.custom: hdinsightactive,hdiseo17may2017,mvc,seodec18
10-
ms.date: 03/11/2020
9+
ms.custom: subject-armqs
10+
ms.date: 03/13/2020
1111
#Customer intent: As a data analyst, I need to create a Hadoop cluster in Azure HDInsight using Resource Manager template
1212
---
1313

1414
# Quickstart: Create Apache Hadoop cluster in Azure HDInsight using Resource Manager template
1515

16-
In this quickstart, you learn how to create an Apache Hadoop cluster in Azure HDInsight using a Resource Manager template.
17-
18-
Similar templates can be viewed at [Azure quickstart templates](https://azure.microsoft.com/resources/templates/?resourceType=Microsoft.Hdinsight&pageNumber=1&sort=Popular). The template reference can be found [here](https://docs.microsoft.com/azure/templates/microsoft.hdinsight/allversions). You can also create a cluster using the [Azure portal](apache-hadoop-linux-create-cluster-get-started-portal.md).
16+
In this quickstart, you use an Azure Resource Manager template to create an [Apache Hadoop](./apache-hadoop-introduction.md) cluster in Azure HDInsight. Hadoop was the original open-source framework for distributed processing and analysis of big data sets on clusters. The Hadoop ecosystem includes related software and utilities, including Apache Hive, Apache HBase, Spark, Kafka, and many others.
1917

18+
[!INCLUDE [About Azure Resource Manager](../../../includes/resource-manager-quickstart-introduction.md)]
19+
2020
Currently HDInsight comes with [seven different cluster types](../hdinsight-overview.md#cluster-types-in-hdinsight). Each cluster type supports a different set of components. All cluster types support Hive. For a list of supported components in HDInsight, see [What's new in the Hadoop cluster versions provided by HDInsight?](../hdinsight-component-versioning.md)
2121

2222
If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/?WT.mc_id=A261C142F) before you begin.
2323

24-
## Create a Hadoop cluster
24+
## Create an Apache Hadoop cluster
25+
26+
### Review the template
27+
28+
The template used in this quickstart is from [Azure Quickstart templates](https://github.com/Azure/azure-quickstart-templates/tree/master/101-hdinsight-linux-ssh-password).
29+
30+
:::code language="json" source="~/quickstart-templates/101-hdinsight-linux-ssh-password/azuredeploy.json" range="1-148":::
31+
32+
33+
Two Azure resources are defined in the template:
2534

26-
1. Select the **Deploy to Azure** button below to sign in to Azure and open the Resource Manager template in the Azure portal.
35+
* [Microsoft.Storage/storageAccounts](https://docs.microsoft.com/azure/templates/microsoft.storage/storageaccounts): create an Azure Storage Account.
36+
* [Microsoft.HDInsight/cluster](https://docs.microsoft.com/azure/templates/microsoft.hdinsight/clusters): create an HDInsight cluster.
37+
38+
### Deploy the template
39+
40+
1. Select the **Deploy to Azure** button below to sign in to Azure and open the Resource Manager template.
2741

2842
<a href="https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure%2Fazure-quickstart-templates%2Fmaster%2F101-hdinsight-linux-ssh-password%2Fazuredeploy.json" target="_blank"><img src="./media/apache-hadoop-linux-tutorial-get-started/hdi-deploy-to-azure1.png" alt="Deploy to Azure button for new cluster"></a>
2943

30-
2. Enter or select the following values:
44+
1. Enter or select the following values:
3145

3246
|Property |Description |
3347
|---------|---------|
34-
|Subscription| Select your Azure subscription. |
35-
|Resource group | Create a resource group or select an existing resource group. A resource group is a container of Azure components. In this case, the resource group contains the HDInsight cluster and the dependent Azure Storage account. |
36-
|Location | Select an Azure location where you want to create your cluster. Choose a location closer to you for better performance. |
37-
|Cluster Name | Enter a name for the Hadoop cluster. Because all clusters in HDInsight share the same DNS namespace this name needs to be unique. The name may only contain lowercase letters, numbers, and hyphens, and must begin with a letter. Each hyphen must be preceded and followed by a non-hyphen character. The name must also be between 3 and 59 characters long. |
48+
|Subscription|From the drop-down list, select the Azure subscription that's used for the cluster.|
49+
|Resource group|From the drop-down list, select your existing resource group, or select **Create new**.|
50+
|Location|The value will autopopulate with the location used for the resource group.|
51+
|Cluster Name|Enter a globally unique name. For this template, use only lowercase letters, and numbers.|
3852
|Cluster Type | Select **hadoop**. |
39-
|Cluster login name and password | The default login name is **admin**. The password must be at least 10 characters in length and must contain at least one digit, one uppercase, and one lower case letter, one non-alphanumeric character (except characters ' " ` \). Make sure you **do not provide** common passwords such as "Pass@word1".|
40-
|SSH username and password| The default username is **sshuser**. You can rename the SSH username. The SSH user password has the same requirements as the cluster login password.|
53+
|Cluster Login User Name|Provide the username, default is **admin**.|
54+
|Cluster Login Password|Provide a password. The password must be at least 10 characters in length and must contain at least one digit, one uppercase, and one lower case letter, one non-alphanumeric character (except characters ' " ` ). |
55+
|Ssh User Name|Provide the username, default is **sshuser**|
56+
|Ssh Password|Provide the password.|
4157

4258
Some properties have been hardcoded in the template. You can configure these values from the template. For more explanation of these properties, see [Create Apache Hadoop clusters in HDInsight](../hdinsight-hadoop-provision-linux-clusters.md).
4359

@@ -46,9 +62,11 @@ If you don't have an Azure subscription, create a [free account](https://azure.m
4662
4763
![HDInsight Linux gets started Resource Manager template on portal](./media/apache-hadoop-linux-tutorial-get-started/hdinsight-linux-get-started-arm-template-on-portal.png "Deploy Hadoop cluster in HDInsight using the Azure portal and a resource group manager template")
4864

49-
3. Select **I agree to the terms and conditions stated above**, then select **Purchase**. You'll receive a notification that your deployment is in progress. It takes about 20 minutes to create a cluster.
65+
1. Review the **TERMS AND CONDITIONS**. Then select **I agree to the terms and conditions stated above**, then **Purchase**. You'll receive a notification that your deployment is in progress. It takes about 20 minutes to create a cluster.
66+
67+
## Review deployed resources
5068

51-
4. Once the cluster is created, you'll receive a **Deployment succeeded** notification with a **Go to resource group** link. Your **Resource group** page will list your new HDInsight cluster and the default storage associated with the cluster. Each cluster has an [Azure Storage account](../hdinsight-hadoop-use-blob-storage.md) or an [Azure Data Lake Storage account](../hdinsight-hadoop-use-data-lake-store.md) dependency. It's referred as the default storage account. The HDInsight cluster and its default storage account must be colocated in the same Azure region. Deleting clusters doesn't delete the storage account.
69+
Once the cluster is created, you'll receive a **Deployment succeeded** notification with a **Go to resource** link. Your Resource group page will list your new HDInsight cluster and the default storage associated with the cluster. Each cluster has an [Azure Storage](../hdinsight-hadoop-use-blob-storage.md) account or an [Azure Data Lake Storage account](../hdinsight-hadoop-use-data-lake-store.md) dependency. It's referred as the default storage account. The HDInsight cluster and its default storage account must be colocated in the same Azure region. Deleting clusters doesn't delete the storage account.
5270

5371
> [!NOTE]
5472
> For other cluster creation methods and understanding the properties used in this quickstart, see [Create HDInsight clusters](../hdinsight-hadoop-provision-linux-clusters.md).
@@ -60,19 +78,15 @@ After you complete the quickstart, you may want to delete the cluster. With HDIn
6078
> [!NOTE]
6179
> If you are *immediately* proceeding to the next tutorial to learn how to run ETL operations using Hadoop on HDInsight, you may want to keep the cluster running. This is because in the tutorial you have to create a Hadoop cluster again. However, if you are not going through the next tutorial right away, you must delete the cluster now.
6280
63-
To delete the cluster and/or the default storage account:
64-
65-
1. Go back to the browser tab where you have the Azure portal. You shall be on the cluster overview page. If you only want to delete the cluster but retain the default storage account, select **Delete**.
66-
67-
![HDInsight delete cluster from portal](./media/apache-hadoop-linux-tutorial-get-started/hdinsight-delete-cluster.png "HDInsight delete cluster from portal")
81+
From the Azure portal, navigate to your cluster, and select **Delete**.
6882

69-
2. If you want to delete the cluster as well as the default storage account, select the resource group name (highlighted in the previous screenshot) to open the resource group page.
83+
![HDInsight delete cluster from portal](./media/apache-hadoop-linux-tutorial-get-started/hdinsight-delete-cluster.png "HDInsight delete cluster from portal")
7084

71-
3. Select **Delete resource group** to delete the resource group, which contains the cluster and the default storage account. Note deleting the resource group deletes the storage account. If you want to keep the storage account, choose to delete the cluster only.
85+
You can also select the resource group name to open the resource group page, and then select **Delete resource group**. By deleting the resource group, you delete both the HDInsight cluster, and the default storage account.
7286

7387
## Next steps
7488

7589
In this quickstart, you learned how to create an Apache Hadoop cluster in HDInsight using a Resource Manager template. In the next article, you learn how to perform an extract, transform, and load (ETL) operation using Hadoop on HDInsight.
7690

7791
> [!div class="nextstepaction"]
78-
> [Extract, transform, and load data using Interactive Query on HDInsight](../interactive-query/interactive-query-tutorial-analyze-flight-data.md)
92+
> [Extract, transform, and load data using Interactive Query on HDInsight](../interactive-query/interactive-query-tutorial-analyze-flight-data.md)

0 commit comments

Comments
 (0)