Skip to content

Commit 1cedb5c

Browse files
authored
Merge pull request #78721 from dagiro/freshness112
freshness112
2 parents 9c9b473 + 8f65499 commit 1cedb5c

19 files changed

+13
-128
lines changed

articles/hdinsight/hadoop/apache-hadoop-use-hive-curl.md

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -21,9 +21,6 @@ Learn how to use the WebHCat REST API to run Apache Hive queries with Apache Had
2121

2222
* A Linux-based Hadoop on HDInsight cluster version 3.4 or greater.
2323

24-
> [!IMPORTANT]
25-
> Linux is the only operating system used on HDInsight version 3.4 or greater. For more information, see [HDInsight retirement on Windows](../hdinsight-component-versioning.md#hdinsight-windows-retirement).
26-
2724
* A REST client. This document uses Windows PowerShell and [Curl](https://curl.haxx.se/) examples.
2825

2926
> [!NOTE]

articles/hdinsight/hadoop/apache-hadoop-use-hive-powershell.md

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -25,9 +25,6 @@ This document provides an example of using Azure PowerShell in the Azure Resourc
2525

2626
* A Linux-based Apache Hadoop on HDInsight cluster version 3.4 or greater.
2727

28-
> [!IMPORTANT]
29-
> Linux is the only operating system used on HDInsight version 3.4 or greater. For more information, see [HDInsight retirement on Windows](../hdinsight-component-versioning.md#hdinsight-windows-retirement).
30-
3128
* A client with Azure PowerShell.
3229

3330
[!INCLUDE [upgrade-powershell](../../../includes/hdinsight-use-latest-powershell.md)]
@@ -56,7 +53,7 @@ The following steps demonstrate how to use these cmdlets to run a job in your HD
5653

5754
.\hivejob.ps1
5855

59-
When the script runs, you are prompted to enter the cluster name and the HTTPS/Cluster Admin account credentials. You may also be prompted to log in to your Azure subscription.
56+
When the script runs, you are prompted to enter the cluster name and the HTTPS/Cluster Admin account credentials. You may also be prompted to sign in to your Azure subscription.
6057

6158
3. When the job completes, it returns information similar to the following text:
6259

articles/hdinsight/hadoop/apache-hadoop-use-mapreduce-powershell.md

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -23,9 +23,6 @@ This document provides an example of using Azure PowerShell to run a MapReduce j
2323

2424
* **An Azure HDInsight (Hadoop on HDInsight) cluster**
2525

26-
> [!IMPORTANT]
27-
> Linux is the only operating system used on HDInsight version 3.4 or greater. For more information, see [HDInsight retirement on Windows](../hdinsight-component-versioning.md#hdinsight-windows-retirement).
28-
2926
* **A workstation with Azure PowerShell**.
3027

3128
## <a id="powershell"></a>Run a MapReduce job

articles/hdinsight/hadoop/apache-hadoop-use-mapreduce-ssh.md

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -24,9 +24,6 @@ Learn how to submit MapReduce jobs from a Secure Shell (SSH) connection to HDIns
2424

2525
* A Linux-based HDInsight (Hadoop on HDInsight) cluster
2626

27-
> [!IMPORTANT]
28-
> Linux is the only operating system used on HDInsight version 3.4 or greater. For more information, see [HDInsight retirement on Windows](../hdinsight-component-versioning.md#hdinsight-windows-retirement).
29-
3027
* An SSH client. For more information, see [Use SSH with HDInsight](../hdinsight-hadoop-linux-use-ssh-unix.md)
3128

3229
## <a id="ssh"></a>Connect with SSH

articles/hdinsight/hdinsight-analyze-twitter-data-linux.md

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,8 +17,6 @@ Learn how to use [Apache Hive](https://hive.apache.org/) to process Twitter data
1717

1818
> [!IMPORTANT]
1919
> The steps in this document were tested on HDInsight 3.6.
20-
>
21-
> Linux is the only operating system used on HDInsight version 3.4 or greater. For more information, see [HDInsight retirement on Windows](hdinsight-component-versioning.md#hdinsight-windows-retirement).
2220
2321
## Get the data
2422

articles/hdinsight/hdinsight-component-versioning.md

Lines changed: 9 additions & 76 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ ms.custom: hdinsightactive,hdiseo17may2017
1010
ms.topic: overview
1111
ms.date: 06/07/2019
1212
---
13+
1314
# What are the Apache Hadoop components and versions available with HDInsight?
1415

1516
Learn about the [Apache Hadoop](https://hadoop.apache.org/) ecosystem components and versions in Microsoft Azure HDInsight, as well as the Enterprise Security Package. Also, learn how to check Hadoop component versions in HDInsight.
@@ -54,10 +55,7 @@ The component versions associated with HDInsight cluster versions are listed in
5455
5556
## Check for current Hadoop component version information
5657

57-
The Hadoop ecosystem component versions associated with HDInsight cluster versions can change with updates to HDInsight. To check the Hadoop components and to verify which versions are being used for a cluster, use the Ambari REST API. The **GetComponentInformation** command retrieves information about service components. For details, see the [Apache Ambari documentation][ambari-docs].
58-
59-
> [!IMPORTANT]
60-
> Linux is the only operating system used on HDInsight version 3.4 or later. For more information, see [Windows retirement on HDInsight](#hdinsight-windows-retirement).
58+
The Hadoop ecosystem component versions associated with HDInsight cluster versions can change with updates to HDInsight. To check the Hadoop components and to verify which versions are being used for a cluster, use the Ambari REST API. The **GetComponentInformation** command retrieves information about service components. For details, see the [Apache Ambari documentation](https://github.com/apache/ambari/blob/trunk/ambari-server/docs/api/v1/index.md).
6159

6260
### Release notes
6361

@@ -74,7 +72,7 @@ The following table lists the versions of HDInsight that are available in the Az
7472
| HDInsight version | HDP version | VM OS | Release date | Support expiration date | Retirement date | High availability | Availability on the Azure portal |
7573
| --- | --- | --- | --- | --- | --- | --- | --- |
7674
| HDInsight 4.0 |HDP 3.0 |Ubuntu 16.0.4 LTS |September 24, 2018 | | |Yes |Yes |
77-
| HDInsight 3.6 |HDP 2.6 |Ubuntu 16.0.4 LTS |April 4, 2017 | June 30th, 2020 |December 31st, 2020 |Yes |Yes |
75+
| HDInsight 3.6 |HDP 2.6 |Ubuntu 16.0.4 LTS |April 4, 2017 | June 30, 2020 |December 31, 2020 |Yes |Yes |
7876

7977

8078
> [!NOTE]
@@ -138,63 +136,10 @@ Currently, only the following cluster types support the Enterprise Security Pack
138136

139137
The Enterprise Security Package supports using Azure Data Lake Storage as both the primary storage and the add-on storage.
140138

141-
### Pricing and SLA
139+
### Pricing and service level agreement
142140

143141
For information on pricing and SLA for the Enterprise Security Package, see [HDInsight pricing](https://azure.microsoft.com/pricing/details/hdinsight/).
144142

145-
## HDInsight Windows retirement
146-
147-
Microsoft Azure HDInsight version 3.3 was the last version of HDInsight on Windows. The retirement date for HDInsight on Windows is July 31, 2018. If you have any HDInsight clusters on Windows 3.3 or earlier, you must migrate to HDInsight on Linux (HDInsight version 3.5 or later) before July 31, 2018. Migrating to the Linux OS enables you to retain the ability to create or resize your HDInsight clusters. Support for HDInsight version 3.3 on Windows expired on June 27, 2016.
148-
149-
Starting with HDInsight version 3.4, Microsoft has released HDInsight only on the Linux OS. As a result, some of the components within HDInsight are available for Linux only. These include [Apache Ranger](https://ranger.apache.org/), [Apache Kafka](https://kafka.apache.org/), Interactive Query, [Apache Spark](https://spark.apache.org/), HDInsight applications, and Azure Data Lake Storage as the primary file system. Future releases of HDInsight are available only on the Linux OS. There will be no future releases of HDInsight on Windows.
150-
151-
## FAQs
152-
153-
### What is the timeline for retiring HDInsight on Windows?
154-
155-
July 31, 2018, is the retirement date for HDInsight on Windows. If the planned retirement date is different for your region, you are notified separately.
156-
157-
### What is the impact of retiring HDInsight on Windows for existing customers?
158-
159-
After HDInsight on Windows is retired, you can't create a new HDInsight Windows cluster, or resize an existing HDInsight Windows cluster. Support for HDInsight version 3.3 expired on June 27, 2016. Therefore, there is no support or bug fixes for HDInsight 3.3 or earlier versions. Future releases of HDInsight are available only on the Linux OS. There will be no future releases of HDInsight on Windows.
160-
161-
### Which versions of HDInsight on Windows are affected?
162-
163-
Azure HDInsight version 3.3 is the last version of HDInsight for Windows. Before HDInsight on Windows is retired, all HDInsight Windows clusters version 3.3 or earlier must be migrated to HDInsight on Linux version 3.5 or later. Migrating your clusters to HDInsight on Linux enables you to retain the ability to create new clusters or resize existing clusters.
164-
165-
### What do I need to do?
166-
167-
Migrate your HDInsight Windows clusters to a supported HDInsight Linux cluster before July 31, 2018. Learn more in the [HDInsight migration document](hdinsight-migrate-from-windows-to-linux.md). For details about Azure HDInsight versions, see the list of [supported versions](hdinsight-component-versioning.md#supported-hdinsight-versions).
168-
169-
### Where do I find the cluster OS type?
170-
171-
In the Azure portal, go to the HDInsight Cluster overview page and locate **Cluster type** under **Essentials**. The cluster OS types are listed on that page.
172-
173-
### I can’t migrate to an HDInsight Linux cluster by July 31, 2018. What is the impact to my HDInsight Windows cluster?
174-
175-
The HDInsight Windows cluster runs as-is, but you cannot create a new HDInsight Windows cluster, or resize an existing HDInsight Windows cluster.
176-
177-
### My cluster has a .NET dependency. How do I resolve this dependency on Linux?
178-
179-
You can resolve your Linux cluster dependency by using the [Mono project](https://www.mono-project.com/). This open-source implementation of .NET is available for HDInsight Linux clusters. Learn more in the [HDInsight migration document](hdinsight-migrate-from-windows-to-linux.md).
180-
181-
### I'm a new customer for HDInsight on Windows. How can I create an HDInsight Windows cluster?
182-
183-
As of July 3, 2017, only existing HDInsight Windows customers can create new HDInsight Windows clusters. New customers cannot create an HDInsight Windows cluster in the Azure portal by using PowerShell or the SDK. We recommend that new customers create a Linux HDInsight cluster. Existing customers can create new HDInsight Windows clusters until the HDInsight on Windows retirement date.
184-
185-
### Is there a pricing impact associated with moving from HDInsight on Windows to HDInsight on Linux?
186-
187-
No, the pricing is the same for HDInsight on either OS.
188-
189-
### What are the customer advantages associated with the move to only using HDInsight on Linux?
190-
191-
* Faster time-to-market for open-source big data technologies through the HDInsight service
192-
* A large community and ecosystem for support
193-
* Ability to exercise active development by the open-source community for Hadoop and other big data technologies
194-
195-
### Does HDInsight on Linux provide additional functionality beyond what is available in HDInsight on Windows?
196-
197-
Starting with HDInsight version 3.4, Microsoft has released HDInsight only on the Linux OS. As a result, some of the components within HDInsight are available for Linux only. These include Apache Ranger, Kafka, Interactive Query, Spark, HDInsight applications, and Azure Data Lake Storage as the primary file system.
198143

199144
## Service level agreement for HDInsight cluster versions
200145

@@ -256,8 +201,8 @@ The following tables list the default virtual machine (VM) sizes for HDInsight c
256201
|ZooKeeper: recommended VM sizes||A4 v2||A2 v2|||A2M v2|
257202
|||A8 v2||A4 v2|||D3 v2|
258203
|||A2m v2||A8 v2|||E8 v3|
259-
|Edge: default VM size||||||D4 v2||
260-
|Edge: recommended VM size||||||D4 v2||
204+
|ML Services: default VM size||||||D4 v2||
205+
|ML Services: recommended VM size||||||D4 v2||
261206
|||||||D12 v2||
262207
|||||||D13 v2||
263208
|||||||D14 v2||
@@ -276,8 +221,8 @@ The following tables list the default virtual machine (VM) sizes for HDInsight c
276221
| Worker: recommended VM sizes |D3,<br/> D4,<br/> D12 |D3,<br/> D4,<br/> D12 | D13,<br/> D14 |D3,<br/> D4,<br/> D12 |D4,<br/> D12,<br/> D13,<br/> D14 | D4,<br/> D12,<br/> D13,<br/> D14 |
277222
| ZooKeeper: default VM size | |A4 v2 | A4 v2| A4 v2 | | A2 v2|
278223
| ZooKeeper: recommended VM sizes | |A2,<br/> A3,<br/> A4 | |A2,<br/> A3,<br/> A4 | | |
279-
| Edge: default VM sizes | | | | | |D4 |
280-
| Edge: recommended VM sizes | | | | | |D4,<br/> D12,<br/> D13,<br/> D14 |
224+
| ML Services: default VM sizes | | | | | |D4 |
225+
| ML Services: recommended VM sizes | | | | | |D4,<br/> D12,<br/> D13,<br/> D14 |
281226

282227
> [!NOTE]
283228
> - Head is known as *Nimbus* for the Storm cluster type.
@@ -288,14 +233,6 @@ The following tables list the default virtual machine (VM) sizes for HDInsight c
288233
- [Cluster setup for Apache Hadoop, Spark, and more on HDInsight](hdinsight-hadoop-provision-linux-clusters.md)
289234
- [Work in Apache Hadoop on HDInsight from a Windows PC](hdinsight-hadoop-windows-tools.md)
290235

291-
[Supported HDInsight versions]:(#supported-hdinsight-versions)
292-
293-
[image-hdi-versioning-versionscreen]: ./media/hdinsight-component-versioning/hdi-versioning-version-screen.png
294-
295-
[wa-forums]: https://azure.microsoft.com/support/forums/
296-
297-
[connect-excel-with-hive-ODBC]: hdinsight-connect-excel-hive-ODBC-driver.md
298-
299236
[hdp-2-2]: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.2.9/bk_HDP_RelNotes/content/ch_relnotes_v229.html
300237

301238
[hdp-2-1-7]: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.1.7-Win/bk_releasenotes_HDP-Win/content/ch_relnotes-HDP-2.1.7.html
@@ -306,8 +243,4 @@ The following tables list the default virtual machine (VM) sizes for HDInsight c
306243

307244
[hdp-1-3-0]: https://docs.hortonworks.com/HDPDocuments/HDP1/HDP-1.3.0/bk_releasenotes_hdp_1.x/content/ch_relnotes-hdp1.3.0_1.html
308245

309-
[hdp-1-1-0]: https://docs.hortonworks.com/HDPDocuments/HDP1/HDP-1.3.0/bk_releasenotes_hdp_1.x/content/ch_relnotes-hdp1.1.1.16_1.html
310-
311-
[ambari-docs]: https://github.com/apache/ambari/blob/trunk/ambari-server/docs/api/v1/index.md
312-
313-
[zookeeper]: https://zookeeper.apache.org/
246+
[hdp-1-1-0]: https://docs.hortonworks.com/HDPDocuments/HDP1/HDP-1.3.0/bk_releasenotes_hdp_1.x/content/ch_relnotes-hdp1.1.1.16_1.html

articles/hdinsight/hdinsight-hadoop-access-yarn-app-logs-linux.md

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -15,9 +15,6 @@ ms.author: hrasheed
1515

1616
Learn how to access the logs for [Apache Hadoop YARN](https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html) (Yet Another Resource Negotiator) applications on an [Apache Hadoop](https://hadoop.apache.org/) cluster in Azure HDInsight.
1717

18-
> [!IMPORTANT]
19-
> The steps in this document require an HDInsight cluster that uses Linux. Linux is the only operating system used on HDInsight version 3.6 or greater. For more information, see [HDInsight component versioning](hdinsight-component-versioning.md#hdinsight-windows-retirement).
20-
2118
## <a name="YARNTimelineServer"></a>YARN Timeline Server
2219

2320
The [Apache Hadoop YARN Timeline Server](https://hadoop.apache.org/docs/r2.7.3/hadoop-yarn/hadoop-yarn-site/TimelineServer.html) provides generic information on completed applications

articles/hdinsight/hdinsight-hadoop-add-hive-libraries.md

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -35,9 +35,6 @@ For **Linux-based clusters**: [https://hdiconfigactions.blob.core.windows.net/li
3535

3636
For **Windows-based clusters**: [https://hdiconfigactions.blob.core.windows.net/setupcustomhivelibsv01/setup-customhivelibs-v01.ps1](https://hdiconfigactions.blob.core.windows.net/setupcustomhivelibsv01/setup-customhivelibs-v01.ps1)
3737

38-
> [!IMPORTANT]
39-
> Linux is the only operating system used on HDInsight version 3.4 or greater. For more information, see [HDInsight retirement on Windows](hdinsight-component-versioning.md#hdinsight-windows-retirement).
40-
4138
**Requirements**
4239

4340
* The scripts must be applied to both the **Head nodes** and **Worker nodes**.

articles/hdinsight/hdinsight-hadoop-collect-debug-heap-dump-linux.md

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -17,9 +17,6 @@ ms.author: hrasheed
1717

1818
Heap dumps contain a snapshot of the application's memory, including the values of variables at the time the dump was created. So they are useful for diagnosing problems that occur at run-time.
1919

20-
> [!IMPORTANT]
21-
> The steps in this document only work with HDInsight clusters that use Linux. Linux is the only operating system used on HDInsight version 3.4 or greater. For more information, see [HDInsight retirement on Windows](hdinsight-component-versioning.md#hdinsight-windows-retirement).
22-
2320
## <a name="whichServices"></a>Services
2421

2522
You can enable heap dumps for the following services:

articles/hdinsight/hdinsight-hadoop-create-linux-clusters-curl-rest.md

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -19,9 +19,6 @@ Learn how to create an HDInsight cluster using an Azure Resource Manager templat
1919

2020
The Azure REST API allows you to perform management operations on services hosted in the Azure platform, including the creation of new resources such as HDInsight clusters.
2121

22-
> [!IMPORTANT]
23-
> Linux is the only operating system used on HDInsight version 3.4 or greater. For more information, see [HDInsight retirement on Windows](hdinsight-component-versioning.md#hdinsight-windows-retirement).
24-
2522
> [!NOTE]
2623
> The steps in this document use the [curl (https://curl.haxx.se/)](https://curl.haxx.se/) utility to communicate with the Azure REST API.
2724
@@ -211,7 +208,7 @@ This example is used in the steps in this document. Replace the example *values*
211208
>
212209
> For more information on node sizes and associated costs, see [HDInsight pricing](https://azure.microsoft.com/pricing/details/hdinsight/).
213210
214-
## Log in to your Azure subscription
211+
## Sign in to your Azure subscription
215212

216213
Follow the steps documented in [Get started with Azure CLI](https://docs.microsoft.com/cli/azure/get-started-with-az-cli2) and connect to your subscription using the `az login` command.
217214

0 commit comments

Comments
 (0)