You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/hdinsight/hdinsight-component-versioning.md
+36-38Lines changed: 36 additions & 38 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,6 @@
1
1
---
2
2
title: Apache Hadoop components and versions - Azure HDInsight
3
-
description: Learn the Apache Hadoop components and versions in Azure HDInsight.
3
+
description: Learn about the Apache Hadoop components and versions in Azure HDInsight.
4
4
author: hrasheed-msft
5
5
ms.author: hrasheed
6
6
ms.reviewer: jasonh
@@ -12,18 +12,18 @@ ms.date: 04/09/2020
12
12
13
13
# What are the Apache Hadoop components and versions available with HDInsight?
14
14
15
-
Learn about the [Apache Hadoop](https://hadoop.apache.org/) environment components and versions in Microsoft Azure HDInsight, and the Enterprise Security Package. Also, learn how to check Hadoop component versions in HDInsight.
15
+
In this article, you learn about the [Apache Hadoop](https://hadoop.apache.org/) environment components and versions in Azure HDInsight and the Enterprise Security Package. You also learn how to check the Hadoop component versions in HDInsight.
16
16
17
17
## Apache Hadoop components available with different HDInsight versions
18
18
19
-
Azure HDInsight supports multiple Hadoop cluster versions that can be deployed at any time. On April 4, 2017, the default cluster version used by Azure HDInsight is 3.6.
19
+
Azure HDInsight supports multiple Hadoop cluster versions that can be deployed at any time. On April 4, 2017, the default cluster version used by Azure HDInsight was 3.6.
20
20
21
-
The component versions associated with HDInsight cluster versions are listed in the following table:
21
+
The component versions associated with HDInsight cluster versions are listed in the following table.
22
22
23
-
> [!NOTE]
24
-
> The default version for the HDInsight service might change without notice. If you have a version dependency, specify the HDInsight version when you create your clusters with the .NET SDK with Azure PowerShell and Azure Classic CLI.
23
+
> [!NOTE]
24
+
> The default version for the HDInsight service might change without notice. If you have a version dependency, specify the HDInsight version when you create your clusters with the .NET SDK with Azure PowerShell and the Azure classic CLI.
| Apache Kafka | 1.1.1, 2.1 | 1.1, 1.0* (See the note.) |
44
44
| Apache Ambari | 2.7.0 | 2.6.0 |
45
45
| Apache Zeppelin | 0.8.0 | 0.7.3 |
46
46
| Mono | 4.2.1 | 4.2.1 |
47
47
48
48
> [!NOTE]
49
-
> Due to system performance considerations, support for Kafka version 0.10 was expired in March 2019.
49
+
> Support for Kafka version 0.10 expired in March 2019 because of system performance considerations.
50
50
51
51
## Check for current Hadoop component version information
52
52
53
-
The Hadoop environment component versions associated with HDInsight cluster versions can change with updates to HDInsight. To check the Hadoop components and to verify which versions are being used for a cluster, use the Ambari REST API. The **GetComponentInformation** command retrieves information about service components. For details, see the [Apache Ambari documentation](https://github.com/apache/ambari/blob/trunk/ambari-server/docs/api/v1/index.md).
53
+
The Hadoop environment component versions associated with HDInsight cluster versions can change with updates to HDInsight. To check the Hadoop components and to verify which versions are being used for a cluster, use the Ambari REST API. The **GetComponentInformation** command retrieves information about service components. For more information, see the [Apache Ambari documentation](https://github.com/apache/ambari/blob/trunk/ambari-server/docs/api/v1/index.md).
54
54
55
55
### Release notes
56
56
57
-
See [HDInsight release notes](hdinsight-release-notes.md) for additional release notes on the latest versions of HDInsight.
57
+
For additional release notes on the latest versions of HDInsight, see [HDInsight release notes](hdinsight-release-notes.md).
58
58
59
59
## Supported HDInsight versions
60
60
61
61
### Support expiration and retirement for HDInsight versions
62
62
63
-
**Support expiration** means Microsoft will no longer provide support for the specified HDInsight version. And it will no longer be available through the Azure portal for cluster creation. However, these versions can still be created using the Azure CLI or the various SDKs.
63
+
**Support expiration** means that Microsoft no longer provides support for the specific HDInsight version. And it's no longer available through the Azure portal for cluster creation. These versions can still be created by using the Azure CLI or the various SDKs.
64
64
65
-
**Retirement**of an HDInsight version means that existing clusters will continue to run as-is. However, new clusters of this version can't be created through any means (including CLI and SDKs). Other control plane features (such as manual scaling and Autoscaling) may also not work after version retirement. Support isn't available for retired versions.
65
+
**Retirement** means that existing clusters of an HDInsight version continue to run asis. New clusters of this version can't be created through any means, which includes the CLI and SDKs. Other control plane features, such as manual scaling and autoscaling, also might not work after version retirement. Support isn't available for retired versions.
66
66
67
-
The following tables list the versions of HDInsight. The support expiration and retirement dates are also provided, when they're known.
67
+
The following tables list the versions of HDInsight. The support expiration and retirement dates are also provided when they're known.
68
68
69
69
### Available versions
70
70
71
-
The following table lists the versions of HDInsight that are available in the Azure portal and other deployment methods like PowerShell and .NET SDK.
71
+
This table lists the versions of HDInsight that are available in the Azure portal and via other deployment methods like PowerShell and the .NET SDK.
72
72
73
73
| HDInsight version | VM OS | Release date | Support expiration date | Retirement date | High availability | Availability in the Azure portal |
Spark 2.1, 2.2 & Kafka 1.0 support will expire on June 30, 2020.
78
+
Spark 2.1, 2.2, and Kafka 1.0 support expires on June 30, 2020.
79
79
80
-
> [!NOTE]
81
-
> After support for a version has expired, it might not be available through the Microsoft Azure portal. However, cluster versions continue to be available using the `Version` parameter in the Windows PowerShell [New-AzHDInsightCluster](https://docs.microsoft.com/powershell/module/az.hdinsight/new-azhdinsightcluster) command and the .NET SDK until the version retirement date.
80
+
> [!NOTE]
81
+
> After support for a version expires, it might not be available through the Azure portal. Cluster versions continue to be available by using the **Version** parameter in the Windows PowerShell [New-AzHDInsightCluster](https://docs.microsoft.com/powershell/module/az.hdinsight/new-azhdinsightcluster) command and the .NET SDK until the version retirement date.
82
82
83
83
### Retired versions
84
84
85
-
The following table lists the versions of HDInsight that **aren't** available in the Azure portal.
85
+
This table lists the versions of HDInsight that aren't available in the Azure portal.
86
86
87
-
| HDInsight version | HDP version | VM OS | Release date | Support expiration date | Retirement date | High availability | Availability on the Azure portal |
87
+
| HDInsight version | HDP version | VM OS | Release date | Support expiration date | Retirement date | High availability | Availability in the Azure portal |
> Highly available clusters with two head nodes are deployed by default for HDInsight version 2.1 and later. They are not available for HDInsight version 1.6 clusters.
99
+
> [!NOTE]
100
+
> Highly available clusters with two head nodes are deployed by default for HDInsight version 2.1 and later. They aren't available for HDInsight version 1.6 clusters.
101
101
102
102
## Enterprise Security Package for HDInsight
103
103
104
-
Enterprise Security is an optional package that you can add on your HDInsight cluster as part of create cluster workflow. The Enterprise Security Package supports:
104
+
The Enterprise Security Package is an optional package that you can add on to your HDInsight cluster as part of creating a cluster workflow. The Enterprise Security Package supports:
105
105
106
106
- Integration with Active Directory for authentication.
107
107
108
-
In the past, you created HDInsight clusters with local admin user and local SSH user. The local admin user can access all the files, folders, tables, and columns. With Enterprise Security Package, you enable role-based access control by integrating HDInsight with your Active Directory. Which includes on-premises Active Directory, Azure Active Directory Domain Services. Or Active Directory on IaaS virtual machine. Domain administrator on the cluster can grant users to use their own corporate (domain) user-name and password.
108
+
In the past, you created HDInsight clusters with a local admin user and a local SSH user. The local admin user can access all the files, folders, tables, and columns. With the Enterprise Security Package, you enable role-based access control by integrating HDInsight with your Active Directory instance, which includes on-premises Active Directory, Azure Active Directory Domain Services, or Active Directory on infrastructure as a service (IaaS) virtual machines. The domain administrator on the cluster can grant users the right to use their own corporate (domain) username and password.
109
109
110
110
For more information, see:
111
111
112
112
-[An introduction to Apache Hadoop security with domain-joined HDInsight clusters](./domain-joined/hdinsight-security-overview.md)
113
113
-[Plan Azure domain-joined Apache Hadoop clusters in HDInsight](./domain-joined/apache-domain-joined-architecture.md)
-[Configure Domain-joined HDInsight clusters using Azure Active Directory Domain Services](./domain-joined/apache-domain-joined-configure-using-azure-adds.md)
114
+
-[Configure a domain-joined sandbox environment](./domain-joined/apache-domain-joined-configure.md)
115
+
-[Configure domain-joined HDInsight clusters by using Azure Active Directory Domain Services](./domain-joined/apache-domain-joined-configure-using-azure-adds.md)
116
116
117
-
- Authorization for data
117
+
- Authorization for data.
118
118
119
119
- Integration with Apache Ranger for authorization for Hive, Spark SQL, and Yarn Queues.
120
120
- You can set access control on files and folders.
121
121
122
-
For more information, see:
123
-
124
-
-[Configure Apache Hive policies in Domain-joined HDInsight](./domain-joined/apache-domain-joined-run-hive.md)
122
+
For more information, see [Configure Apache Hive policies in domain-joined HDInsight](./domain-joined/apache-domain-joined-run-hive.md).
125
123
126
-
-View the audit logs to monitor accesses and the configured policies.
124
+
-Viewing the audit logs to monitor accesses and the configured policies.
127
125
128
126
### Supported cluster types
129
127
@@ -139,18 +137,18 @@ Currently, only the following cluster types support the Enterprise Security Pack
139
137
140
138
The Enterprise Security Package supports using Azure Data Lake Storage as both the primary storage and the add-on storage.
141
139
142
-
### Pricing and servicelevel agreement (SLA)
140
+
### Pricing and service-level agreement
143
141
144
-
For information on pricing and SLA for the Enterprise Security Package, see [HDInsight pricing](https://azure.microsoft.com/pricing/details/hdinsight/).
142
+
For information on pricing and the service-level agreement (SLA) for the Enterprise Security Package, see [HDInsight pricing](https://azure.microsoft.com/pricing/details/hdinsight/).
145
143
146
-
## Servicelevel agreement for HDInsight cluster versions
144
+
## Service-level agreement for HDInsight cluster versions
147
145
148
-
The service level agreement (SLA) is defined as a _support window_. Support window is the time period an HDInsight version is supported by `Microsoft Customer Service and Support`. If the version has a passed _support expiration date_, the HDInsight cluster is outside the support window. Support expiration for HDInsight version X (after a newer X+1 version is available) is the later:
146
+
The SLA is defined as a _support window_. A support window is the time period that an HDInsight version is supported by Microsoft Customer Service and Support. If the version has passed the _support expiration date_, the HDInsight cluster is outside the support window. Support expiration for HDInsight version X (after a newer X+1 version is available) is the later of:
149
147
150
-
- Formula 1: Add 180 days to the date when the HDInsight cluster version X was released.
151
-
- Formula 2: Add 90 days to the date when the HDInsight cluster version X+1 is made available in Azure portal.
148
+
-**Formula 1:** Add 180 days to the date when the HDInsight cluster version X was released.
149
+
-**Formula 2:** Add 90 days to the date when the HDInsight cluster version X+1 is made available in the Azure portal.
152
150
153
-
The _retirement date_ is the date after which the cluster version can't be created on HDInsight. Starting July 31, 2017, you can't resize an HDInsight cluster after its retirement date.
151
+
The _retirement date_ is the date after which the cluster version can't be created on HDInsight. As of July 31, 2017, you can't resize an HDInsight cluster after its retirement date.
154
152
155
153
## Default node configuration and virtual machine sizes for clusters
0 commit comments