You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/hdinsight/hdinsight-component-versioning.md
+11-41Lines changed: 11 additions & 41 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -12,11 +12,11 @@ ms.date: 02/26/2020
12
12
13
13
# What are the Apache Hadoop components and versions available with HDInsight?
14
14
15
-
Learn about the [Apache Hadoop](https://hadoop.apache.org/)ecosystem components and versions in Microsoft Azure HDInsight, as well as the Enterprise Security Package. Also, learn how to check Hadoop component versions in HDInsight.
15
+
Learn about the [Apache Hadoop](https://hadoop.apache.org/)environment components and versions in Microsoft Azure HDInsight, and the Enterprise Security Package. Also, learn how to check Hadoop component versions in HDInsight.
16
16
17
17
## Apache Hadoop components available with different HDInsight versions
18
18
19
-
Azure HDInsight supports multiple Hadoop cluster versions that can be deployed at any time. As of April 4, 2017, the default cluster version used by Azure HDInsight is 3.6.
19
+
Azure HDInsight supports multiple Hadoop cluster versions that can be deployed at any time. On April 4, 2017, the default cluster version used by Azure HDInsight is 3.6.
20
20
21
21
The component versions associated with HDInsight cluster versions are listed in the following table:
22
22
@@ -50,7 +50,7 @@ The component versions associated with HDInsight cluster versions are listed in
50
50
51
51
## Check for current Hadoop component version information
52
52
53
-
The Hadoop ecosystem component versions associated with HDInsight cluster versions can change with updates to HDInsight. To check the Hadoop components and to verify which versions are being used for a cluster, use the Ambari REST API. The **GetComponentInformation** command retrieves information about service components. For details, see the [Apache Ambari documentation](https://github.com/apache/ambari/blob/trunk/ambari-server/docs/api/v1/index.md).
53
+
The Hadoop environment component versions associated with HDInsight cluster versions can change with updates to HDInsight. To check the Hadoop components and to verify which versions are being used for a cluster, use the Ambari REST API. The **GetComponentInformation** command retrieves information about service components. For details, see the [Apache Ambari documentation](https://github.com/apache/ambari/blob/trunk/ambari-server/docs/api/v1/index.md).
54
54
55
55
### Release notes
56
56
@@ -60,15 +60,15 @@ See [HDInsight release notes](hdinsight-release-notes.md) for additional release
60
60
61
61
### Support expiration and retirement for HDInsight versions
62
62
63
-
**Support expiration** means that Microsoft will no longer provide support for the specified HDInsight version and it will no longer be avaiable through the Azure portal for cluster creation. However, these versions can still be created using the Azure CLI or the various SDKs.
63
+
**Support expiration** means Microsoft will no longer provide support for the specified HDInsight version. And it will no longer be available through the Azure portal for cluster creation. However, these versions can still be created using the Azure CLI or the various SDKs.
64
64
65
-
**Retirement** of an HDInsight version means that existing clusters will continue to run as-is. However, new clusters of this version cannot be created through any means (including CLI and SDKs). Other control plane features (such as manual scaling and Autoscaling) may also not work after version retirement. Support is not available for retired versions.
65
+
**Retirement** of an HDInsight version means that existing clusters will continue to run as-is. However, new clusters of this version can't be created through any means (including CLI and SDKs). Other control plane features (such as manual scaling and Autoscaling) may also not work after version retirement. Support isn't available for retired versions.
66
66
67
67
The following tables list the versions of HDInsight. The support expiration and retirement dates are also provided, when they're known.
68
68
69
69
### Available versions
70
70
71
-
The following table lists the versions of HDInsight that are available in the Azure portal as well as other deployment methods like PowerShell and .NET SDK.
71
+
The following table lists the versions of HDInsight that are available in the Azure portal and other deployment methods like PowerShell and .NET SDK.
72
72
73
73
| HDInsight version | VM OS | Release date | Support expiration date | Retirement date | High availability | Availability in the Azure portal |
74
74
| --- | --- | --- | --- | --- | --- | --- |
@@ -82,7 +82,7 @@ Spark 2.1, 2.2 & Kafka 1.0 support will expire on June 30, 2020.
82
82
83
83
### Retired versions
84
84
85
-
The following table lists the versions of HDInsight that are **not** available in the Azure portal.
85
+
The following table lists the versions of HDInsight that **aren't** available in the Azure portal.
86
86
87
87
| HDInsight version | HDP version | VM OS | Release date | Support expiration date | Retirement date | High availability | Availability on the Azure portal |
88
88
| --- | --- | --- | --- | --- | --- | --- | --- |
@@ -105,7 +105,7 @@ Enterprise Security is an optional package that you can add on your HDInsight cl
105
105
106
106
- Integration with Active Directory for authentication.
107
107
108
-
In the past, you can only create HDInsight clusters with a local admin user and a local SSH user. The local admin user can access all the files, folders, tables, and columns. With the Enterprise Security Package, you can enable role-based access control by integrating HDInsight clusters with your own Active Directory, which include on-premises Active Directory, Azure Active Directory Domain Services, or Active Directory on IaaS virtual machine. Domain administrator on the cluster can grant users to use their own corporate (domain) user-name and password to access the cluster.
108
+
In the past, you created HDInsight clusters with local admin user and local SSH user. The local admin user can access all the files, folders, tables, and columns. With Enterprise Security Package, you enable role-based access control by integrating HDInsight with your Active Directory. Which includes on-premises Active Directory, Azure Active Directory Domain Services. Or Active Directory on IaaS virtual machine. Domain administrator on the cluster can grant users to use their own corporate (domain) user-name and password.
109
109
110
110
For more information, see:
111
111
@@ -139,38 +139,19 @@ Currently, only the following cluster types support the Enterprise Security Pack
139
139
140
140
The Enterprise Security Package supports using Azure Data Lake Storage as both the primary storage and the add-on storage.
141
141
142
-
### Pricing and service level agreement
142
+
### Pricing and service level agreement (SLA)
143
143
144
144
For information on pricing and SLA for the Enterprise Security Package, see [HDInsight pricing](https://azure.microsoft.com/pricing/details/hdinsight/).
145
145
146
146
## Service level agreement for HDInsight cluster versions
147
147
148
-
The service level agreement (SLA) is defined in terms of a _support window_. The support window is the period of time that an HDInsight cluster version is supported by Microsoft Customer Service and Support. If the version has a _support expiration date_ that has passed, the HDInsight cluster is outside the support window. The support expiration date for a specified HDInsight version X (after a newer X+1 version is available) is calculated as the later of:
148
+
The service level agreement (SLA) is defined as a _support window_. Support window is the time period an HDInsight version is supported by `Microsoft Customer Service and Support`. If the version has a passed _support expiration date_, the HDInsight cluster is outside the support window. Support expiration for HDInsight version X (after a newer X+1 version is available) is the later:
149
149
150
150
- Formula 1: Add 180 days to the date when the HDInsight cluster version X was released.
151
151
- Formula 2: Add 90 days to the date when the HDInsight cluster version X+1 is made available in Azure portal.
152
152
153
153
The _retirement date_ is the date after which the cluster version can't be created on HDInsight. Starting July 31, 2017, you can't resize an HDInsight cluster after its retirement date.
154
154
155
-
## Hortonworks release notes associated with HDInsight versions
156
-
157
-
The section provides links to release notes for the Hortonworks Data Platform distributions and Apache components that are used with HDInsight.
158
-
* HDInsight cluster version 4.0 uses a Hadoop distribution that is based on [Hortonworks Data Platform 3.0](https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.0.0/release-notes/content/relnotes.html)
159
-
* HDInsight cluster version 3.6 uses a Hadoop distribution that is based on [Hortonworks Data Platform 2.6](https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.0/bk_release-notes/content/ch_relnotes.html).
160
-
* HDInsight cluster version 3.5 uses a Hadoop distribution that is based on [Hortonworks Data Platform 2.5](https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.0/bk_release-notes/content/ch_relnotes_v250.html). HDInsight cluster version 3.5 is the _default_ Hadoop cluster that is created in the Azure portal.
161
-
* HDInsight cluster version 3.4 uses a Hadoop distribution that is based on [Hortonworks Data Platform 2.4](https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0/bk_HDP_RelNotes/content/ch_relnotes_v240.html).
162
-
* HDInsight cluster version 3.3 uses a Hadoop distribution that is based on [Hortonworks Data Platform 2.3](https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk_HDP_RelNotes/content/ch_relnotes_v230.html).
163
-
164
-
*[Apache Storm release notes](https://storm.apache.org/2015/11/05/storm0100-released.html) are available on the Apache website.
165
-
*[Apache Hive release notes](https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12332384&styleName=Text&projectId=12310843) are available on the Apache website.
166
-
* HDInsight cluster version 3.2 uses a Hadoop distribution that is based on [Hortonworks Data Platform 2.2][hdp-2-2].
167
-
168
-
* Release notes for specific Apache components are available as follows: [Hive 0.14](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310843&version=12326450), [Pig 0.14](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310730&version=12326954), [HBase 0.98.4](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753&version=12326810), [Phoenix 4.2.0](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315120&version=12327581), [M/R 2.6](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310941&version=12327180), [HDFS 2.6](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310942&version=12327181), [YARN 2.6](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313722&version=12327197), [Common](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310240&version=12327179), [Tez 0.5.2](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314426&version=12328742), [Ambari 2.0](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12312020&version=12327486), [Storm 0.9.3](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12327112), and [Oozie 4.1.0](https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12324960&projectId=12311620).
169
-
* HDInsight cluster version 3.1 uses a Hadoop distribution that is based on [Hortonworks Data Platform 2.1.7][hdp-2-1-7]. HDInsight 3.1 clusters created before November, 7, 2014, are based on [Hortonworks Data Platform 2.1.1][hdp-2-1-1].
170
-
* HDInsight cluster version 3.0 uses a Hadoop distribution that is based on [Hortonworks Data Platform 2.0][hdp-2-0-8].
171
-
* HDInsight cluster version 2.1 uses a Hadoop distribution that is based on [Hortonworks Data Platform 1.3][hdp-1-3-0].
172
-
* HDInsight cluster version 1.6 uses a Hadoop distribution that is based on [Hortonworks Data Platform 1.1][hdp-1-1-0].
173
-
174
155
## Default node configuration and virtual machine sizes for clusters
175
156
176
157
For more information on which virtual machine SKUs to select for your cluster, see [Azure HDInsight cluster configuration details](hdinsight-supported-node-configuration.md).
@@ -179,15 +160,4 @@ For more information on which virtual machine SKUs to select for your cluster, s
179
160
180
161
-[Cluster setup for Apache Hadoop, Spark, and more on HDInsight](hdinsight-hadoop-provision-linux-clusters.md)
181
162
-[Work in Apache Hadoop on HDInsight from a Windows PC](hdinsight-hadoop-windows-tools.md)
title: Hortonworks release notes associated with Azure HDInsight versions
3
+
description: Learn the Apache Hadoop components and versions in Azure HDInsight.
4
+
author: hrasheed-msft
5
+
ms.author: hrasheed
6
+
ms.reviewer: jasonh
7
+
ms.service: hdinsight
8
+
ms.topic: conceptual
9
+
ms.date: 04/22/2020
10
+
---
11
+
12
+
# Hortonworks release notes associated with HDInsight versions
13
+
14
+
The section provides links to release notes for the Hortonworks Data Platform distributions and Apache components that are used with HDInsight.
15
+
16
+
## Current versions
17
+
18
+
* HDInsight cluster version 4.0 uses a Hadoop distribution that is based on [Hortonworks Data Platform 3.0](https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.0.0/release-notes/content/relnotes.html).
19
+
20
+
* HDInsight cluster version 3.6 uses a Hadoop distribution that is based on [Hortonworks Data Platform 2.6](https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.0/bk_release-notes/content/ch_relnotes.html).
21
+
22
+
## Older versions
23
+
24
+
* HDInsight cluster version 3.5 uses a Hadoop distribution that is based on [Hortonworks Data Platform 2.5](https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.0/bk_release-notes/content/ch_relnotes_v250.html). HDInsight cluster version 3.5 is the _default_ Hadoop cluster that is created in the Azure portal.
25
+
26
+
* HDInsight cluster version 3.4 uses a Hadoop distribution that is based on [Hortonworks Data Platform 2.4](https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0/bk_HDP_RelNotes/content/ch_relnotes_v240.html).
27
+
28
+
* HDInsight cluster version 3.3 uses a Hadoop distribution that is based on [Hortonworks Data Platform 2.3](https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk_HDP_RelNotes/content/ch_relnotes_v230.html).
29
+
30
+
*[Apache Storm release notes](https://storm.apache.org/2015/11/05/storm0100-released.html) are available on the Apache website.
31
+
*[Apache Hive release notes](https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12332384&styleName=Text&projectId=12310843) are available on the Apache website.
32
+
33
+
* HDInsight cluster version 3.2 uses a Hadoop distribution that is based on [Hortonworks Data Platform 2.2][hdp-2-2].
34
+
35
+
* Release notes for specific Apache components are available as follows: [Hive 0.14](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310843&version=12326450), [Pig 0.14](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310730&version=12326954), [HBase 0.98.4](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753&version=12326810), [Phoenix 4.2.0](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315120&version=12327581), [M/R 2.6](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310941&version=12327180), [HDFS 2.6](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310942&version=12327181), [YARN 2.6](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313722&version=12327197), [Common](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310240&version=12327179), [Tez 0.5.2](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314426&version=12328742), [Ambari 2.0](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12312020&version=12327486), [Storm 0.9.3](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12327112), and [Oozie 4.1.0](https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12324960&projectId=12311620).
36
+
37
+
* HDInsight cluster version 3.1 uses a Hadoop distribution that is based on [Hortonworks Data Platform 2.1.7][hdp-2-1-7]. HDInsight 3.1 clusters created before November, 7, 2014, are based on [Hortonworks Data Platform 2.1.1][hdp-2-1-1].
38
+
39
+
* HDInsight cluster version 3.0 uses a Hadoop distribution that is based on [Hortonworks Data Platform 2.0][hdp-2-0-8].
40
+
41
+
* HDInsight cluster version 2.1 uses a Hadoop distribution that is based on [Hortonworks Data Platform 1.3][hdp-1-3-0].
42
+
43
+
* HDInsight cluster version 1.6 uses a Hadoop distribution that is based on [Hortonworks Data Platform 1.1][hdp-1-1-0].
44
+
45
+
## Next steps
46
+
47
+
*[Apache Hadoop components on HDInsight](./hdinsight-component-versioning.md)
48
+
*[Cluster setup for Apache Hadoop, Spark, and more on HDInsight](hdinsight-hadoop-provision-linux-clusters.md)
0 commit comments