Skip to content

Commit 4f2b6b7

Browse files
authored
Merge pull request #104803 from dagiro/freshness192
freshness192
2 parents b7c6c92 + a0aab24 commit 4f2b6b7

File tree

1 file changed

+15
-46
lines changed

1 file changed

+15
-46
lines changed

articles/hdinsight/hdinsight-hadoop-create-linux-clusters-with-secure-transfer-storage.md

Lines changed: 15 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -7,47 +7,35 @@ ms.reviewer: jasonh
77
ms.service: hdinsight
88
ms.topic: conceptual
99
ms.custom: hdinsightactive
10-
ms.date: 01/22/2020
10+
ms.date: 02/18/2020
1111
---
1212

13-
# Create Apache Hadoop cluster with secure transfer storage accounts in Azure HDInsight
13+
# Apache Hadoop clusters with secure transfer storage accounts in Azure HDInsight
1414

1515
The [Secure transfer required](../storage/common/storage-require-secure-transfer.md) feature enhances the security of your Azure Storage account by enforcing all requests to your account through a secure connection. This feature and the wasbs scheme are only supported by HDInsight cluster version 3.6 or newer.
1616

17-
**Enabling secure storage transfer after creating a cluster can result in errors using your storage account and is not recommended. It is better to create a new cluster with the property enabled.**
17+
> [!IMPORTANT]
18+
> Enabling secure storage transfer after creating a cluster can result in errors using your storage account and is not recommended. It is better to create a new cluster using a storage account with secure transfer already enabled.
1819
19-
## Prerequisites
20+
## Storage accounts
2021

21-
Before you begin this article, you must have:
22+
### Azure portal
2223

23-
* Azure subscription: To create a free one-month trial account, browse to [azure.microsoft.com/free](https://azure.microsoft.com/free).
24-
* An Azure Storage account with secure transfer enabled. For the instructions, see [Create a storage account](../storage/common/storage-account-create.md) and [Require secure transfer](../storage/common/storage-require-secure-transfer.md).
25-
* A Blob container on the storage account.
24+
By default, the secure transfer required property is enabled when you create a storage account in Azure portal.
2625

27-
## Create cluster
26+
To update an existing storage account with Azure portal, see [Require secure transfer with Azure portal](../storage/common/storage-require-secure-transfer.md#require-secure-transfer-for-an-existing-storage-account).
2827

29-
[!INCLUDE [delete-cluster-warning](../../includes/hdinsight-delete-cluster-warning.md)]
28+
### PowerShell
3029

31-
In this section, you create a Hadoop cluster in HDInsight using an [Azure Resource Manager template](../azure-resource-manager/templates/deploy-powershell.md). The template is located in [GitHub](https://azure.microsoft.com/resources/templates/101-hdinsight-linux-with-existing-default-storage-account/). Resource Manager template experience isn't required for following this article. For other cluster creation methods and understanding the properties used in this article, see [Create HDInsight clusters](hdinsight-hadoop-provision-linux-clusters.md).
30+
For the PowerShell cmdlet [New-AzStorageAccount](https://docs.microsoft.com/powershell/module/az.storage/new-azstorageaccount), ensure parameter `-EnableHttpsTrafficOnly` is set to `1`.
3231

33-
1. Click the following image to sign in to Azure and open the Resource Manager template in the Azure portal.
32+
To update an existing storage account with PowerShell, see [Require secure transfer with PowerShell](../storage/common/storage-require-secure-transfer.md#require-secure-transfer-with-powershell).
3433

35-
<a href="https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure%2Fazure-quickstart-templates%2Fmaster%2F101-hdinsight-linux-with-existing-default-storage-account%2Fazuredeploy.json" target="_blank"><img src="./media/hdinsight-hadoop-create-linux-clusters-with-secure-transfer-storage/hdi-deploy-to-azure1.png" alt="Deploy to Azure button for new cluster"></a>
34+
### Azure CLI
3635

37-
2. Follow the instructions to create the cluster with the following specifications:
36+
For the Azure CLI command [az storage account create](https://docs.microsoft.com/cli/azure/storage/account?view=azure-cli-latest#az-storage-account-create), ensure parameter `--https-only` is set to `true`.
3837

39-
* Specify HDInsight version 3.6. Version 3.6 or newer is required.
40-
* Specify a secure transfer enabled storage account.
41-
* Use short name for the storage account.
42-
* Both the storage account and the blob container must be created beforehand.
43-
44-
For the instructions, see [Create cluster](hadoop/apache-hadoop-linux-tutorial-get-started.md#create-cluster).
45-
46-
If you use script action to provide your own configuration files, you must use wasbs in the following settings:
47-
48-
* fs.defaultFS (core-site)
49-
* spark.eventLog.dir
50-
* spark.history.fs.logDirectory
38+
To update an existing storage account with Azure CLI, see [Require secure transfer with Azure CLI](../storage/common/storage-require-secure-transfer.md#require-secure-transfer-with-azure-cli).
5139

5240
## Add additional storage accounts
5341

@@ -59,25 +47,6 @@ There are several options to add additional secure transfer enabled storage acco
5947

6048
## Next steps
6149

62-
In this article, you've learned how to create an HDInsight cluster, and enable secure transfer to the storage accounts.
63-
64-
To learn more about analyzing data with HDInsight, see the following articles:
65-
66-
* To learn more about using [Apache Hive](https://hive.apache.org/) with HDInsight, including how to perform Hive queries from Visual Studio, see [Use Apache Hive with HDInsight](hadoop/hdinsight-use-hive.md).
67-
* To learn about [Apache Hadoop MapReduce](https://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html), a way to write programs that process data on Hadoop, see [Use Apache Hadoop MapReduce with HDInsight](hadoop/hdinsight-use-mapreduce.md).
68-
* To learn about using the HDInsight Tools for Visual Studio to analyze data on HDInsight, see [Get started using Visual Studio Apache Hadoop tools for HDInsight](hadoop/apache-hadoop-visual-studio-tools-get-started.md).
69-
70-
To learn more about how HDInsight stores data or how to get data into HDInsight, see the following articles:
71-
50+
* The use of Azure Storage (WASB) instead of [Apache Hadoop HDFS](https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsUserGuide.html) as the default data store
7251
* For information on how HDInsight uses Azure Storage, see [Use Azure Storage with HDInsight](hdinsight-hadoop-use-blob-storage.md).
7352
* For information on how to upload data to HDInsight, see [Upload data to HDInsight](hdinsight-upload-data.md).
74-
75-
To learn more about creating or managing an HDInsight cluster, see the following articles:
76-
77-
* To learn about managing your Linux-based HDInsight cluster, see [Manage HDInsight clusters using Apache Ambari](hdinsight-hadoop-manage-ambari.md).
78-
* To learn more about the options you can select when creating an HDInsight cluster, see [Creating HDInsight on Linux using custom options](hdinsight-hadoop-provision-linux-clusters.md).
79-
* If you're familiar with Linux, and Apache Hadoop, but want to know specifics about Hadoop on the HDInsight, see [Working with HDInsight on Linux](hdinsight-hadoop-linux-information.md). This article provides information such as:
80-
81-
* URLs for services hosted on the cluster, such as [Apache Ambari](https://ambari.apache.org/) and [WebHCat](https://cwiki.apache.org/confluence/display/Hive/WebHCat)
82-
* The location of [Apache Hadoop](https://hadoop.apache.org/) files and examples on the local file system
83-
* The use of Azure Storage (WASB) instead of [Apache Hadoop HDFS](https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsUserGuide.html) as the default data store

0 commit comments

Comments
 (0)