Skip to content

Commit eb3da84

Browse files
committed
Creating new Bicep quickstart - HDInsight Hadoop
1 parent 3388b1a commit eb3da84

File tree

3 files changed

+108
-0
lines changed

3 files changed

+108
-0
lines changed

articles/azure-resource-manager/bicep/toc.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,8 @@
5151
href: ../../analysis-services/analysis-services-create-bicep-file.md?toc=/azure/azure-resource-manager/bicep/toc.json
5252
- name: Event Hubs
5353
href: ../../event-hubs/event-hubs-bicep-namespace-event-hub.md?toc=/azure/azure-resource-manager/bicep/toc.json
54+
- name: HDInsight - Hadoop
55+
href: ../../hdinsight/hadoop/apache-hadoop-linux-tutorial-get-started-bicep.md?toc=/azure/azure-resource-manager/bicep/toc.json
5456
- name: Compute
5557
items:
5658
- name: Batch

articles/hdinsight/TOC.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -510,6 +510,9 @@ items:
510510
items:
511511
- name: Create Apache Hadoop cluster - Portal
512512
href: ./hadoop/apache-hadoop-linux-create-cluster-get-started-portal.md
513+
- name: Create Apache Hadoop cluster - Bicep
514+
displayName: ARM, Resource Manager, Template
515+
href: ./hadoop/apache-hadoop-linux-tutorial-get-started-bicep.md
513516
- name: Create Apache Hadoop cluster - ARM Template
514517
displayName: Resource Manager
515518
href: ./hadoop/apache-hadoop-linux-tutorial-get-started.md
Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
---
2+
title: 'Quickstart: Create Apache Hadoop cluster in Azure HDInsight using Bicep'
3+
description: In this quickstart, you create Apache Hadoop cluster in Azure HDInsight using Bicep
4+
author: schaffererin
5+
ms.author: v-eschaffer
6+
ms.service: hdinsight
7+
ms.topic: quickstart
8+
ms.custom: subject-armqs, mode-arm
9+
ms.date: 04/08/2022
10+
#Customer intent: As a data analyst, I need to create a Hadoop cluster in Azure HDInsight using Bicep
11+
---
12+
13+
# Quickstart: Create Apache Hadoop cluster in Azure HDInsight using Bicep
14+
15+
In this quickstart, you use Bicep to create an [Apache Hadoop](./apache-hadoop-introduction.md) cluster in Azure HDInsight. Hadoop was the original open-source framework for distributed processing and analysis of big data sets on clusters. The Hadoop ecosystem includes related software and utilities, including Apache Hive, Apache HBase, Spark, Kafka, and many others.
16+
17+
[!INCLUDE [About Bicep](../../../includes/resource-manager-quickstart-bicep-introduction.md)]
18+
19+
Currently HDInsight comes with [seven different cluster types](../hdinsight-overview.md#cluster-types-in-hdinsight). Each cluster type supports a different set of components. All cluster types support Hive. For a list of supported components in HDInsight, see [What's new in the Hadoop cluster versions provided by HDInsight?](../hdinsight-component-versioning.md)
20+
21+
## Prerequisites
22+
23+
If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/?WT.mc_id=A261C142F) before you begin.
24+
25+
## Review the Bicep file
26+
27+
The Bicep file used in this quickstart is from [Azure Quickstart Templates](https://azure.microsoft.com/resources/templates/hdinsight-linux-ssh-password/).
28+
29+
:::code language="bicep" source="~/quickstart-templates/quickstarts/microsoft.hdinsight/hdinsight-linux-ssh-password/main.bicep":::
30+
31+
Two Azure resources are defined in the Bicep file:
32+
33+
* [Microsoft.Storage/storageAccounts](/azure/templates/microsoft.storage/storageaccounts): create an Azure Storage Account.
34+
* [Microsoft.HDInsight/cluster](/azure/templates/microsoft.hdinsight/clusters): create an HDInsight cluster.
35+
36+
## Deploy the Bicep file
37+
38+
1. Save the Bicep file as **main.bicep** to your local computer.
39+
1. Deploy the Bicep file using either Azure CLI or Azure PowerShell.
40+
41+
# [CLI](#tab/CLI)
42+
43+
```azurecli
44+
az group create --name exampleRG --location eastus
45+
az deployment group create --resource-group exampleRG --template-file main.bicep --parameters clusterName=<cluster-name> clusterType=<cluster-type> clusterLoginUserName=<cluster-username> sshUserName=<ssh-username>
46+
```
47+
48+
# [PowerShell](#tab/PowerShell)
49+
50+
```azurepowershell
51+
New-AzResourceGroup -Name exampleRG -Location eastus
52+
New-AzResourceGroupDeployment -ResourceGroupName exampleRG -TemplateFile ./main.bicep -clusterName "<cluster-name>" -clusterType "<cluster-type>" -clusterLoginUserName "<cluster-username>" -sshUserName "<ssh-username>"
53+
```
54+
55+
---
56+
57+
> [!NOTE]
58+
> Replace **\<cluster-name\>** with the name of the HDInsight cluster to create. Replace **\<cluster-type\>** with the type of the HDInsight cluster to create. Allowed strings include: `hadoop`, `interactivehive`, `hbase`, `storm`, and `spark`. Replace **\<cluster-username\>** with the credentials used to submit jobs to the cluster and to log in to cluster dashboards. Replace **\<ssh-username\>** with the credentials used to remotely access the cluster. The username cannot be admin. You'll be prompted to enter **clusterLoginPassword**, which must be at least 10 characters long and contain one digit, one uppercase letter, one lowercase letter, and one non-alphanumeric character except single quotes, double quotes, backslashes, right brackets, or full stops. It also must not contain three consecutive characters from the cluster username or SSH username. You'll also be prompted to enter **sshPassword**, which must be 6-72 characters long and must contain at least one digit, one uppercase letter, and one lowercase letter. It must not contain any three consecutive characters from the cluster login name.
59+
60+
When the deployment finishes, you should see a message indicating the deployment succeeded.
61+
62+
## Review deployed resources
63+
64+
Use the Azure portal, Azure CLI, or Azure PowerShell to list the deployed resources in the resource group.
65+
66+
# [CLI](#tab/CLI)
67+
68+
```azurecli-interactive
69+
az resource list --resource-group exampleRG
70+
```
71+
72+
# [PowerShell](#tab/PowerShell)
73+
74+
```azurepowershell-interactive
75+
Get-AzResource -ResourceGroupName exampleRG
76+
```
77+
78+
---
79+
80+
## Clean up resources
81+
82+
When no longer needed, use the Azure portal, Azure CLI, or Azure PowerShell to delete the resource group and its resources.
83+
84+
# [CLI](#tab/CLI)
85+
86+
```azurecli-interactive
87+
az group delete --name exampleRG
88+
```
89+
90+
# [PowerShell](#tab/PowerShell)
91+
92+
```azurepowershell-interactive
93+
Remove-AzResourceGroup -Name exampleRG
94+
```
95+
96+
---
97+
98+
## Next steps
99+
100+
In this quickstart, you learned how to create an Apache Hadoop cluster in HDInsight using Bicep. In the next article, you learn how to perform an extract, transform, and load (ETL) operation using Hadoop on HDInsight.
101+
102+
> [!div class="nextstepaction"]
103+
> [Extract, transform, and load data using Interactive Query on HDInsight](../interactive-query/interactive-query-tutorial-analyze-flight-data.md)

0 commit comments

Comments
 (0)