Skip to content

Commit b06be4b

Browse files
Merge pull request #218698 from sreekzz/patch-124
Removed Storm Contents Phase 2
2 parents 341630f + f058e1c commit b06be4b

10 files changed

+15
-60
lines changed

articles/hdinsight/hadoop/apache-hadoop-etl-at-scale.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ description: Learn how extract, transform, and load is used in HDInsight with Ap
44
ms.service: hdinsight
55
ms.topic: how-to
66
ms.custom: hdinsightactive,seoapr2020
7-
ms.date: 04/01/2022
7+
ms.date: 11/17/2022
88
---
99

1010
# Extract, transform, and load (ETL) at scale
@@ -66,7 +66,7 @@ Azure Data Lake Storage is a managed, hyperscale repository for analytics data.
6666

6767
Data is usually ingested into Data Lake Storage through Azure Data Factory. You can also use Data Lake Storage SDKs, the AdlCopy service, Apache DistCp, or Apache Sqoop. The service you choose depends on where the data is. If it's in an existing Hadoop cluster, you might use Apache DistCp, the AdlCopy service, or Azure Data Factory. For data in Azure Blob storage, you might use Azure Data Lake Storage .NET SDK, Azure PowerShell, or Azure Data Factory.
6868

69-
Data Lake Storage is optimized for event ingestion through Azure Event Hubs or Apache Storm.
69+
Data Lake Storage is optimized for event ingestion through Azure Event Hubs.
7070

7171
### Considerations for both storage options
7272

articles/hdinsight/hdinsight-hadoop-create-linux-clusters-curl-rest.md

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ description: Learn how to create HDInsight clusters by submitting Azure Resource
44
ms.service: hdinsight
55
ms.topic: how-to
66
ms.custom: hdinsightactive, devx-track-azurecli
7-
ms.date: 08/05/2022
7+
ms.date: 11/17/2022
88
---
99

1010
# Create Apache Hadoop clusters using the Azure REST API
@@ -35,7 +35,6 @@ The following JSON document is a merger of the template and parameters files fro
3535
"type": "string",
3636
"allowedValues": ["hadoop",
3737
"hbase",
38-
"storm",
3938
"spark"],
4039
"metadata": {
4140
"description": "The type of the HDInsight cluster to create."

articles/hdinsight/hdinsight-hadoop-customize-cluster-bootstrap.md

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ description: Learn how to customize HDInsight cluster configuration programmatic
44
ms.service: hdinsight
55
ms.topic: how-to
66
ms.custom: hdinsightactive, devx-track-azurepowershell
7-
ms.date: 05/31/2022
7+
ms.date: 11/17/2022
88
---
99

1010
# Customize HDInsight clusters using Bootstrap
@@ -30,7 +30,6 @@ For example, using these programmatic methods, you can configure options in thes
3030
* mapred-site
3131
* oozie-site.xml
3232
* oozie-env.xml
33-
* storm-site.xml
3433
* tez-site.xml
3534
* webhcat-site.xml
3635
* yarn-site.xml

articles/hdinsight/hdinsight-hadoop-linux-information.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ description: Get implementation tips for using Linux-based HDInsight (Hadoop) cl
44
ms.service: hdinsight
55
ms.custom: hdinsightactive,seoapr2020
66
ms.topic: conceptual
7-
ms.date: 04/29/2020
7+
ms.date: 11/17/2022
88
---
99

1010
# Information about using HDInsight on Linux
@@ -111,7 +111,7 @@ In HDInsight, the data storage resources (Azure Blob Storage and Azure Data Lake
111111

112112
### <a name="URI-and-scheme"></a>URI and scheme
113113

114-
Some commands may require you to specify the scheme as part of the URI when accessing a file. For example, the Storm-HDFS component requires you to specify the scheme. When using non-default storage (storage added as "additional" storage to the cluster), you must always use the scheme as part of the URI.
114+
Some commands may require you to specify the scheme as part of the URI when accessing a file. When using non-default storage (storage added as "additional" storage to the cluster), you must always use the scheme as part of the URI.
115115

116116
When using [**Azure Storage**](./hdinsight-hadoop-use-blob-storage.md), use one of the following URI schemes:
117117

@@ -243,4 +243,4 @@ To use a different version of a component, upload the version you need and use i
243243

244244
* [Manage HDInsight clusters by using the Apache Ambari REST API](./hdinsight-hadoop-manage-ambari-rest-api.md)
245245
* [Use Apache Hive with HDInsight](hadoop/hdinsight-use-hive.md)
246-
* [Use MapReduce jobs with HDInsight](hadoop/hdinsight-use-mapreduce.md)
246+
* [Use MapReduce jobs with HDInsight](hadoop/hdinsight-use-mapreduce.md)

articles/hdinsight/hdinsight-hadoop-use-data-lake-storage-gen1.md

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ description: Learn how to query data from Azure Data Lake Storage Gen1 and to st
44
ms.service: hdinsight
55
ms.topic: how-to
66
ms.custom: hdinsightactive,hdiseo17may2017,seoapr2020, devx-track-azurepowershell
7-
ms.date: 09/15/2022
7+
ms.date: 11/17/2022
88
---
99

1010
# Use Data Lake Storage Gen1 with Azure HDInsight clusters
@@ -40,7 +40,6 @@ Currently, only some of the HDInsight cluster types/versions support using Data
4040
| HDInsight version 3.4 | No | Yes | |
4141
| HDInsight version 3.3 | No | No | |
4242
| HDInsight version 3.2 | No | Yes | |
43-
| Storm | | |You can use Data Lake Storage Gen1 to write data from a Storm topology. You can also use Data Lake Storage Gen1 for reference data that can then be read by a Storm topology.|
4443

4544
> [!WARNING]
4645
> HDInsight HBase is not supported with Azure Data Lake Storage Gen1

articles/hdinsight/hdinsight-key-scenarios-to-monitor.md

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ description: How to monitor health and performance of Apache Hadoop clusters in
44
ms.service: hdinsight
55
ms.topic: how-to
66
ms.custom: hdinsightactive
7-
ms.date: 04/28/2022
7+
ms.date: 11/17/2022
88
---
99

1010
# Monitor cluster performance in Azure HDInsight
@@ -72,7 +72,6 @@ If your cluster's backing store is Azure Data Lake Storage (ADLS), your throttli
7272

7373
* [Performance tuning guidance for Apache Hive on HDInsight and Azure Data Lake Storage](../data-lake-store/data-lake-store-performance-tuning-hive.md)
7474
* [Performance tuning guidance for MapReduce on HDInsight and Azure Data Lake Storage](../data-lake-store/data-lake-store-performance-tuning-mapreduce.md)
75-
* [Performance tuning guidance for Apache Storm on HDInsight and Azure Data Lake Storage](../data-lake-store/data-lake-store-performance-tuning-storm.md)
7675

7776
## Troubleshoot sluggish node performance
7877

articles/hdinsight/hdinsight-scaling-best-practices.md

Lines changed: 1 addition & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ author: yeturis
66
ms.service: hdinsight
77
ms.topic: how-to
88
ms.custom: seoapr2020
9-
ms.date: 07/21/2022
9+
ms.date: 11/17/2022
1010
---
1111

1212
# Manually scale Azure HDInsight clusters
@@ -66,37 +66,6 @@ The impact of changing the number of data nodes varies for each type of cluster
6666

6767
For more information on using the HBase shell, see [Get started with an Apache HBase example in HDInsight](hbase/apache-hbase-tutorial-get-started-linux.md).
6868

69-
* Apache Storm
70-
71-
You can seamlessly add or remove data nodes while Storm is running. However, after a successful completion of the scaling operation, you'll need to rebalance the topology. Rebalancing allows the topology to readjust [parallelism settings](https://storm.apache.org/documentation/Understanding-the-parallelism-of-a-Storm-topology.html) based on the new number of nodes in the cluster. To rebalance running topologies, use one of the following options:
72-
73-
* Storm web UI
74-
75-
Use the following steps to rebalance a topology using the Storm UI.
76-
77-
1. Open `https://CLUSTERNAME.azurehdinsight.net/stormui` in your web browser, where `CLUSTERNAME` is the name of your Storm cluster. If prompted, enter the HDInsight cluster administrator (admin) name and password you specified when creating the cluster.
78-
79-
1. Select the topology you wish to rebalance, then select the **Rebalance** button. Enter the delay before the rebalance operation is done.
80-
81-
:::image type="content" source="./media/hdinsight-scaling-best-practices/hdinsight-portal-scale-cluster-storm-rebalance.png" alt-text="HDInsight Storm scale rebalance":::
82-
83-
* Command-line interface (CLI) tool
84-
85-
Connect to the server and use the following command to rebalance a topology:
86-
87-
```bash
88-
storm rebalance TOPOLOGYNAME
89-
```
90-
91-
You can also specify parameters to override the parallelism hints originally provided by the topology. For example, the code below reconfigures the `mytopology` topology to 5 worker processes, 3 executors for the blue-spout component, and 10 executors for the yellow-bolt component.
92-
93-
```bash
94-
## Reconfigure the topology "mytopology" to use 5 worker processes,
95-
## the spout "blue-spout" to use 3 executors, and
96-
## the bolt "yellow-bolt" to use 10 executors
97-
$ storm rebalance mytopology -n 5 -e blue-spout=3 -e yellow-bolt=10
98-
```
99-
10069
* Kafka
10170

10271
You should rebalance partition replicas after scaling operations. For more information, see the [High availability of data with Apache Kafka on HDInsight](./kafka/apache-kafka-high-availability.md) document.

articles/hdinsight/hdinsight-virtual-network-architecture.md

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ title: Azure HDInsight virtual network architecture
33
description: Learn the resources available when you create an HDInsight cluster in an Azure Virtual Network.
44
ms.service: hdinsight
55
ms.topic: conceptual
6-
ms.date: 04/01/2022
6+
ms.date: 11/17/2022
77
---
88

99
# Azure HDInsight virtual network architecture
@@ -16,12 +16,9 @@ Azure HDInsight clusters have different types of virtual machines, or nodes. Eac
1616

1717
| Type | Description |
1818
| --- | --- |
19-
| Head node | For all cluster types except Apache Storm, the head nodes host the processes that manage execution of the distributed application. The head node is also the node that you can SSH into and execute applications that are then coordinated to run across the cluster resources. The number of head nodes is fixed at two for all cluster types. |
2019
| ZooKeeper node | Zookeeper coordinates tasks between the nodes that are doing data processing. It also does leader election of the head node, and keeps track of which head node is running a specific master service. The number of ZooKeeper nodes is fixed at three. |
2120
| Worker node | Represents the nodes that support data processing functionality. Worker nodes can be added or removed from the cluster to scale computing capability and manage costs. |
2221
| Region node | For the HBase cluster type, the region node (also referred to as a Data Node) runs the Region Server. Region Servers serve and manage a portion of the data managed by HBase. Region nodes can be added or removed from the cluster to scale computing capability and manage costs.|
23-
| Nimbus node | For the Storm cluster type, the Nimbus node provides functionality similar to the Head node. The Nimbus node assigns tasks to other nodes in a cluster through Zookeeper, which coordinates the running of Storm topologies. |
24-
| Supervisor node | For the Storm cluster type, the supervisor node executes the instructions provided by the Nimbus node to do the processing. |
2522

2623
## Resource naming conventions
2724

articles/hdinsight/log-analytics-migration.md

Lines changed: 1 addition & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ ms.service: hdinsight
55
ms.topic: how-to
66
ms.author: sairamyeturi
77
author: yeturis
8-
ms.date: 09/02/2022
8+
ms.date: 11/17/2022
99
---
1010

1111
# Log Analytics migration guide for Azure HDInsight clusters
@@ -267,13 +267,6 @@ The following charts show the table mappings from the classic Azure Monitoring I
267267
| HDInsightHBaseMetrics | <ul><li>**Description**: This table contains JMX metrics from HBase. It contains all the same JMX metrics from the tables listed in the Old Schema column. In contrast from the old tables, each row contains one metric.</li><li>**Old table**: metrics\_regionserver\_CL, metrics\_regionserver\_wal\_CL, metrics\_regionserver\_ipc\_CL, metrics\_regionserver\_os\_CL, metrics\_regionserver\_replication\_CL, metrics\_restserver\_CL, metrics\_restserver\_jvm\_CL, metrics\_hmaster\_assignmentmanager\_CL, metrics\_hmaster\_ipc\_CL, metrics\_hmaser\_os\_CL, metrics\_hmaster\_balancer\_CL, metrics\_hmaster\_jvm\_CL, metrics\_hmaster\_CL,metrics\_hmaster\_fs\_CL</li></ul>|
268268
| HDInsightHBaseLogs | <ul><li>**Description**: This table contains logs from HBase and its related components: Phoenix and HDFS.</li><li>**Old table**: log\_regionserver\_CL, log\_restserver\_CL, log\_phoenixserver\_CL, log\_hmaster\_CL, log\_hdfsnamenode\_CL, log\_garbage\_collector\_CL</li></ul>|
269269

270-
## Storm workload
271-
272-
| New Table | Details |
273-
| --- | --- |
274-
| HDInsightStormMetrics | <ul><li>**Description**: This table contains the same JMX metrics as the tables in the Old Tables section. Its rows contain one metric per record.</li><li>**Old table**: metrics\_stormnimbus\_CL, metrics\_stormsupervisor\_CL</li></ul>|
275-
| HDInsightStormTopologyMetrics | <ul><li>**Description**: This table contains topology level metrics from Storm. It's the same shape as the table listed in Old Tables section.</li><li>**Old table**: metrics\_stormrest\_CL</li></ul>|
276-
| HDInsightStormLogs | <ul><li>**Description**: This table contains all logs generated from Storm.</li><li>**Old table**: log\_supervisor\_CL, log\_nimbus\_CL</li></ul>|
277270

278271
## Oozie workload
279272

articles/hdinsight/manage-clusters-runbooks.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ description: Learn how to create and delete Azure HDInsight clusters with script
44
ms.service: hdinsight
55
ms.custom: hdinsightactive
66
ms.topic: tutorial
7-
ms.date: 12/27/2019
7+
ms.date: 11/17/2022
88
---
99

1010
# Tutorial: Create Azure HDInsight clusters with Azure Automation
@@ -108,7 +108,7 @@ If you don’t have an Azure subscription, create a [free account](https://azure
108108
#Automation credential for user to SSH into cluster
109109
$sshCreds = Get-AutomationPSCredential –Name 'ssh-password'
110110
111-
$clusterType = "Hadoop" #Use any supported cluster type (Hadoop, HBase, Storm, etc.)
111+
$clusterType = "Hadoop" #Use any supported cluster type (Hadoop, HBase, etc.)
112112
$clusterOS = "Linux"
113113
$clusterWorkerNodes = 3
114114
$clusterNodeSize = "Standard_D3_v2"
@@ -162,4 +162,4 @@ When no longer needed, delete the Azure Automation Account that was created to a
162162
## Next steps
163163
164164
> [!div class="nextstepaction"]
165-
> [Manage Apache Hadoop clusters in HDInsight by using Azure PowerShell](hdinsight-administer-use-powershell.md)
165+
> [Manage Apache Hadoop clusters in HDInsight by using Azure PowerShell](hdinsight-administer-use-powershell.md)

0 commit comments

Comments
 (0)