You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/hdinsight/hadoop/apache-hadoop-use-hive-curl.md
-3Lines changed: 0 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -21,9 +21,6 @@ Learn how to use the WebHCat REST API to run Apache Hive queries with Apache Had
21
21
22
22
* A Linux-based Hadoop on HDInsight cluster version 3.4 or greater.
23
23
24
-
> [!IMPORTANT]
25
-
> Linux is the only operating system used on HDInsight version 3.4 or greater. For more information, see [HDInsight retirement on Windows](../hdinsight-component-versioning.md#hdinsight-windows-retirement).
26
-
27
24
* A REST client. This document uses Windows PowerShell and [Curl](https://curl.haxx.se/) examples.
Copy file name to clipboardExpand all lines: articles/hdinsight/hadoop/apache-hadoop-use-hive-powershell.md
+1-4Lines changed: 1 addition & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -25,9 +25,6 @@ This document provides an example of using Azure PowerShell in the Azure Resourc
25
25
26
26
* A Linux-based Apache Hadoop on HDInsight cluster version 3.4 or greater.
27
27
28
-
> [!IMPORTANT]
29
-
> Linux is the only operating system used on HDInsight version 3.4 or greater. For more information, see [HDInsight retirement on Windows](../hdinsight-component-versioning.md#hdinsight-windows-retirement).
@@ -56,7 +53,7 @@ The following steps demonstrate how to use these cmdlets to run a job in your HD
56
53
57
54
.\hivejob.ps1
58
55
59
-
When the script runs, you are prompted to enter the cluster name and the HTTPS/Cluster Admin account credentials. You may also be prompted to log in to your Azure subscription.
56
+
When the script runs, you are prompted to enter the cluster name and the HTTPS/Cluster Admin account credentials. You may also be prompted to sign in to your Azure subscription.
60
57
61
58
3. When the job completes, it returns information similar to the following text:
Copy file name to clipboardExpand all lines: articles/hdinsight/hadoop/apache-hadoop-use-mapreduce-powershell.md
-3Lines changed: 0 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -23,9 +23,6 @@ This document provides an example of using Azure PowerShell to run a MapReduce j
23
23
24
24
***An Azure HDInsight (Hadoop on HDInsight) cluster**
25
25
26
-
> [!IMPORTANT]
27
-
> Linux is the only operating system used on HDInsight version 3.4 or greater. For more information, see [HDInsight retirement on Windows](../hdinsight-component-versioning.md#hdinsight-windows-retirement).
Copy file name to clipboardExpand all lines: articles/hdinsight/hadoop/apache-hadoop-use-mapreduce-ssh.md
-3Lines changed: 0 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -24,9 +24,6 @@ Learn how to submit MapReduce jobs from a Secure Shell (SSH) connection to HDIns
24
24
25
25
* A Linux-based HDInsight (Hadoop on HDInsight) cluster
26
26
27
-
> [!IMPORTANT]
28
-
> Linux is the only operating system used on HDInsight version 3.4 or greater. For more information, see [HDInsight retirement on Windows](../hdinsight-component-versioning.md#hdinsight-windows-retirement).
29
-
30
27
* An SSH client. For more information, see [Use SSH with HDInsight](../hdinsight-hadoop-linux-use-ssh-unix.md)
Copy file name to clipboardExpand all lines: articles/hdinsight/hdinsight-analyze-twitter-data-linux.md
-2Lines changed: 0 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -17,8 +17,6 @@ Learn how to use [Apache Hive](https://hive.apache.org/) to process Twitter data
17
17
18
18
> [!IMPORTANT]
19
19
> The steps in this document were tested on HDInsight 3.6.
20
-
>
21
-
> Linux is the only operating system used on HDInsight version 3.4 or greater. For more information, see [HDInsight retirement on Windows](hdinsight-component-versioning.md#hdinsight-windows-retirement).
# What are the Apache Hadoop components and versions available with HDInsight?
14
15
15
16
Learn about the [Apache Hadoop](https://hadoop.apache.org/) ecosystem components and versions in Microsoft Azure HDInsight, as well as the Enterprise Security Package. Also, learn how to check Hadoop component versions in HDInsight.
@@ -54,10 +55,7 @@ The component versions associated with HDInsight cluster versions are listed in
54
55
55
56
## Check for current Hadoop component version information
56
57
57
-
The Hadoop ecosystem component versions associated with HDInsight cluster versions can change with updates to HDInsight. To check the Hadoop components and to verify which versions are being used for a cluster, use the Ambari REST API. The **GetComponentInformation** command retrieves information about service components. For details, see the [Apache Ambari documentation][ambari-docs].
58
-
59
-
> [!IMPORTANT]
60
-
> Linux is the only operating system used on HDInsight version 3.4 or later. For more information, see [Windows retirement on HDInsight](#hdinsight-windows-retirement).
58
+
The Hadoop ecosystem component versions associated with HDInsight cluster versions can change with updates to HDInsight. To check the Hadoop components and to verify which versions are being used for a cluster, use the Ambari REST API. The **GetComponentInformation** command retrieves information about service components. For details, see the [Apache Ambari documentation](https://github.com/apache/ambari/blob/trunk/ambari-server/docs/api/v1/index.md).
61
59
62
60
### Release notes
63
61
@@ -74,7 +72,7 @@ The following table lists the versions of HDInsight that are available in the Az
74
72
| HDInsight version | HDP version | VM OS | Release date | Support expiration date | Retirement date | High availability | Availability on the Azure portal |
@@ -138,63 +136,10 @@ Currently, only the following cluster types support the Enterprise Security Pack
138
136
139
137
The Enterprise Security Package supports using Azure Data Lake Storage as both the primary storage and the add-on storage.
140
138
141
-
### Pricing and SLA
139
+
### Pricing and service level agreement
142
140
143
141
For information on pricing and SLA for the Enterprise Security Package, see [HDInsight pricing](https://azure.microsoft.com/pricing/details/hdinsight/).
144
142
145
-
## HDInsight Windows retirement
146
-
147
-
Microsoft Azure HDInsight version 3.3 was the last version of HDInsight on Windows. The retirement date for HDInsight on Windows is July 31, 2018. If you have any HDInsight clusters on Windows 3.3 or earlier, you must migrate to HDInsight on Linux (HDInsight version 3.5 or later) before July 31, 2018. Migrating to the Linux OS enables you to retain the ability to create or resize your HDInsight clusters. Support for HDInsight version 3.3 on Windows expired on June 27, 2016.
148
-
149
-
Starting with HDInsight version 3.4, Microsoft has released HDInsight only on the Linux OS. As a result, some of the components within HDInsight are available for Linux only. These include [Apache Ranger](https://ranger.apache.org/), [Apache Kafka](https://kafka.apache.org/), Interactive Query, [Apache Spark](https://spark.apache.org/), HDInsight applications, and Azure Data Lake Storage as the primary file system. Future releases of HDInsight are available only on the Linux OS. There will be no future releases of HDInsight on Windows.
150
-
151
-
## FAQs
152
-
153
-
### What is the timeline for retiring HDInsight on Windows?
154
-
155
-
July 31, 2018, is the retirement date for HDInsight on Windows. If the planned retirement date is different for your region, you are notified separately.
156
-
157
-
### What is the impact of retiring HDInsight on Windows for existing customers?
158
-
159
-
After HDInsight on Windows is retired, you can't create a new HDInsight Windows cluster, or resize an existing HDInsight Windows cluster. Support for HDInsight version 3.3 expired on June 27, 2016. Therefore, there is no support or bug fixes for HDInsight 3.3 or earlier versions. Future releases of HDInsight are available only on the Linux OS. There will be no future releases of HDInsight on Windows.
160
-
161
-
### Which versions of HDInsight on Windows are affected?
162
-
163
-
Azure HDInsight version 3.3 is the last version of HDInsight for Windows. Before HDInsight on Windows is retired, all HDInsight Windows clusters version 3.3 or earlier must be migrated to HDInsight on Linux version 3.5 or later. Migrating your clusters to HDInsight on Linux enables you to retain the ability to create new clusters or resize existing clusters.
164
-
165
-
### What do I need to do?
166
-
167
-
Migrate your HDInsight Windows clusters to a supported HDInsight Linux cluster before July 31, 2018. Learn more in the [HDInsight migration document](hdinsight-migrate-from-windows-to-linux.md). For details about Azure HDInsight versions, see the list of [supported versions](hdinsight-component-versioning.md#supported-hdinsight-versions).
168
-
169
-
### Where do I find the cluster OS type?
170
-
171
-
In the Azure portal, go to the HDInsight Cluster overview page and locate **Cluster type** under **Essentials**. The cluster OS types are listed on that page.
172
-
173
-
### I can’t migrate to an HDInsight Linux cluster by July 31, 2018. What is the impact to my HDInsight Windows cluster?
174
-
175
-
The HDInsight Windows cluster runs as-is, but you cannot create a new HDInsight Windows cluster, or resize an existing HDInsight Windows cluster.
176
-
177
-
### My cluster has a .NET dependency. How do I resolve this dependency on Linux?
178
-
179
-
You can resolve your Linux cluster dependency by using the [Mono project](https://www.mono-project.com/). This open-source implementation of .NET is available for HDInsight Linux clusters. Learn more in the [HDInsight migration document](hdinsight-migrate-from-windows-to-linux.md).
180
-
181
-
### I'm a new customer for HDInsight on Windows. How can I create an HDInsight Windows cluster?
182
-
183
-
As of July 3, 2017, only existing HDInsight Windows customers can create new HDInsight Windows clusters. New customers cannot create an HDInsight Windows cluster in the Azure portal by using PowerShell or the SDK. We recommend that new customers create a Linux HDInsight cluster. Existing customers can create new HDInsight Windows clusters until the HDInsight on Windows retirement date.
184
-
185
-
### Is there a pricing impact associated with moving from HDInsight on Windows to HDInsight on Linux?
186
-
187
-
No, the pricing is the same for HDInsight on either OS.
188
-
189
-
### What are the customer advantages associated with the move to only using HDInsight on Linux?
190
-
191
-
* Faster time-to-market for open-source big data technologies through the HDInsight service
192
-
* A large community and ecosystem for support
193
-
* Ability to exercise active development by the open-source community for Hadoop and other big data technologies
194
-
195
-
### Does HDInsight on Linux provide additional functionality beyond what is available in HDInsight on Windows?
196
-
197
-
Starting with HDInsight version 3.4, Microsoft has released HDInsight only on the Linux OS. As a result, some of the components within HDInsight are available for Linux only. These include Apache Ranger, Kafka, Interactive Query, Spark, HDInsight applications, and Azure Data Lake Storage as the primary file system.
198
143
199
144
## Service level agreement for HDInsight cluster versions
200
145
@@ -256,8 +201,8 @@ The following tables list the default virtual machine (VM) sizes for HDInsight c
256
201
|ZooKeeper: recommended VM sizes||A4 v2||A2 v2|||A2M v2|
257
202
|||A8 v2||A4 v2|||D3 v2|
258
203
|||A2m v2||A8 v2|||E8 v3|
259
-
|Edge: default VM size||||||D4 v2||
260
-
|Edge: recommended VM size||||||D4 v2||
204
+
|ML Services: default VM size||||||D4 v2||
205
+
|ML Services: recommended VM size||||||D4 v2||
261
206
|||||||D12 v2||
262
207
|||||||D13 v2||
263
208
|||||||D14 v2||
@@ -276,8 +221,8 @@ The following tables list the default virtual machine (VM) sizes for HDInsight c
Copy file name to clipboardExpand all lines: articles/hdinsight/hdinsight-hadoop-access-yarn-app-logs-linux.md
-3Lines changed: 0 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -15,9 +15,6 @@ ms.author: hrasheed
15
15
16
16
Learn how to access the logs for [Apache Hadoop YARN](https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html) (Yet Another Resource Negotiator) applications on an [Apache Hadoop](https://hadoop.apache.org/) cluster in Azure HDInsight.
17
17
18
-
> [!IMPORTANT]
19
-
> The steps in this document require an HDInsight cluster that uses Linux. Linux is the only operating system used on HDInsight version 3.6 or greater. For more information, see [HDInsight component versioning](hdinsight-component-versioning.md#hdinsight-windows-retirement).
20
-
21
18
## <aname="YARNTimelineServer"></a>YARN Timeline Server
22
19
23
20
The [Apache Hadoop YARN Timeline Server](https://hadoop.apache.org/docs/r2.7.3/hadoop-yarn/hadoop-yarn-site/TimelineServer.html) provides generic information on completed applications
Copy file name to clipboardExpand all lines: articles/hdinsight/hdinsight-hadoop-add-hive-libraries.md
-3Lines changed: 0 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -35,9 +35,6 @@ For **Linux-based clusters**: [https://hdiconfigactions.blob.core.windows.net/li
35
35
36
36
For **Windows-based clusters**: [https://hdiconfigactions.blob.core.windows.net/setupcustomhivelibsv01/setup-customhivelibs-v01.ps1](https://hdiconfigactions.blob.core.windows.net/setupcustomhivelibsv01/setup-customhivelibs-v01.ps1)
37
37
38
-
> [!IMPORTANT]
39
-
> Linux is the only operating system used on HDInsight version 3.4 or greater. For more information, see [HDInsight retirement on Windows](hdinsight-component-versioning.md#hdinsight-windows-retirement).
40
-
41
38
**Requirements**
42
39
43
40
* The scripts must be applied to both the **Head nodes** and **Worker nodes**.
Copy file name to clipboardExpand all lines: articles/hdinsight/hdinsight-hadoop-collect-debug-heap-dump-linux.md
-3Lines changed: 0 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -17,9 +17,6 @@ ms.author: hrasheed
17
17
18
18
Heap dumps contain a snapshot of the application's memory, including the values of variables at the time the dump was created. So they are useful for diagnosing problems that occur at run-time.
19
19
20
-
> [!IMPORTANT]
21
-
> The steps in this document only work with HDInsight clusters that use Linux. Linux is the only operating system used on HDInsight version 3.4 or greater. For more information, see [HDInsight retirement on Windows](hdinsight-component-versioning.md#hdinsight-windows-retirement).
22
-
23
20
## <aname="whichServices"></a>Services
24
21
25
22
You can enable heap dumps for the following services:
Copy file name to clipboardExpand all lines: articles/hdinsight/hdinsight-hadoop-create-linux-clusters-curl-rest.md
+1-4Lines changed: 1 addition & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -19,9 +19,6 @@ Learn how to create an HDInsight cluster using an Azure Resource Manager templat
19
19
20
20
The Azure REST API allows you to perform management operations on services hosted in the Azure platform, including the creation of new resources such as HDInsight clusters.
21
21
22
-
> [!IMPORTANT]
23
-
> Linux is the only operating system used on HDInsight version 3.4 or greater. For more information, see [HDInsight retirement on Windows](hdinsight-component-versioning.md#hdinsight-windows-retirement).
24
-
25
22
> [!NOTE]
26
23
> The steps in this document use the [curl (https://curl.haxx.se/)](https://curl.haxx.se/) utility to communicate with the Azure REST API.
27
24
@@ -211,7 +208,7 @@ This example is used in the steps in this document. Replace the example *values*
211
208
>
212
209
> For more information on node sizes and associated costs, see [HDInsight pricing](https://azure.microsoft.com/pricing/details/hdinsight/).
213
210
214
-
## Log in to your Azure subscription
211
+
## Sign in to your Azure subscription
215
212
216
213
Follow the steps documented in [Get started with Azure CLI](https://docs.microsoft.com/cli/azure/get-started-with-az-cli2) and connect to your subscription using the `az login` command.
0 commit comments