Skip to content

Commit d9dbf91

Browse files
authored
Merge pull request #101112 from dagiro/freshness175
freshness175
2 parents 953d397 + 71ea4b9 commit d9dbf91

File tree

1 file changed

+12
-22
lines changed

1 file changed

+12
-22
lines changed

articles/hdinsight/storm/apache-storm-develop-csharp-event-hub-topology.md

Lines changed: 12 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -2,14 +2,13 @@
22
title: Process events from Event Hubs with Storm - Azure HDInsight
33
description: Learn how to process data from Azure Event Hubs with a C# Storm topology created in Visual Studio, by using the HDInsight tools for Visual Studio.
44
author: hrasheed-msft
5+
ms.author: hrasheed
56
ms.reviewer: jasonh
6-
77
ms.service: hdinsight
88
ms.topic: conceptual
9-
ms.date: 11/27/2017
10-
ms.author: hrasheed
11-
ROBOTS: NOINDEX
9+
ms.date: 01/14/2020
1210
---
11+
1312
# Process events from Azure Event Hubs with Apache Storm on HDInsight (C#)
1413

1514
Learn how to work with Azure Event Hubs from [Apache Storm](https://storm.apache.org/) on HDInsight. This document uses a C# Storm topology to read and write data from Event Hubs
@@ -21,20 +20,12 @@ Learn how to work with Azure Event Hubs from [Apache Storm](https://storm.apache
2120

2221
The steps in this document use SCP.NET, a NuGet package that makes it easy to create C# topologies and components for use with Storm on HDInsight.
2322

24-
> [!IMPORTANT]
25-
> While the steps in this document rely on a Windows development environment with Visual Studio, the compiled project can be submitted to a Storm on HDInsight cluster that uses Linux. Only Linux-based clusters created after October 28, 2016, support SCP.NET topologies.
26-
2723
HDInsight 3.4 and greater use Mono to run C# topologies. The example used in this document works with HDInsight 3.6. If you plan on creating your own .NET solutions for HDInsight, check the [Mono compatibility](https://www.mono-project.com/docs/about-mono/compatibility/) document for potential incompatibilities.
2824

2925
### Cluster versioning
3026

3127
The Microsoft.SCP.Net.SDK NuGet package you use for your project must match the major version of Storm installed on HDInsight. HDInsight versions 3.5 and 3.6 use Storm 1.x, so you must use SCP.NET version 1.0.x.x with these clusters.
3228

33-
> [!IMPORTANT]
34-
> The example in this document expects an HDInsight 3.5 or 3.6 cluster.
35-
>
36-
> Linux is the only operating system used on HDInsight version 3.4 or greater.
37-
3829
C# topologies must also target .NET 4.5.
3930

4031
## How to work with Event Hubs
@@ -104,9 +95,6 @@ You can download a complete version of the project created in this article from
10495

10596
* An Apache Storm cluster on HDInsight. See [Create Apache Hadoop clusters using the Azure portal](../hdinsight-hadoop-create-linux-clusters-portal.md) and select **Storm** for **Cluster type**.
10697

107-
> [!WARNING]
108-
> The example used in this document requires Storm on HDInsight version 3.5 or 3.6. This does not work with older versions of HDInsight, due to breaking class name changes. For a version of this example that works with older clusters, see [GitHub](https://github.com/Azure-Samples/hdinsight-dotnet-java-storm-eventhub/releases).
109-
11098
* An [Azure event hub](../../event-hubs/event-hubs-create.md).
11199

112100
* The [Azure .NET SDK](https://azure.microsoft.com/downloads/).
@@ -128,7 +116,7 @@ Create a directory named `eventhubspout`, and save the file into the directory.
128116

129117
Event Hubs is the data source for this example. Use the information in the "Create an event hub" section of [Get started with Event Hubs](../../event-hubs/event-hubs-create.md).
130118

131-
1. After the event hub has been created, view the **EventHub** settings in the Azure portal, and select **Shared access policies**. Select **+ Add** to add the following policies:
119+
1. After the event hub has been created, view the **EventHub** settings in the Azure portal, and select **Shared access policies**. Select **+ Add** to create the following policies:
132120

133121
| Name | Permissions |
134122
| --- | --- |
@@ -141,11 +129,11 @@ Event Hubs is the data source for this example. Use the information in the "Crea
141129

142130
## Configure the EventHubWriter
143131

144-
1. If you have not already installed the latest version of the HDInsight tools for Visual Studio, see [Get started using HDInsight tools for Visual Studio](../hadoop/apache-hadoop-visual-studio-tools-get-started.md).
132+
1. If you haven't already installed the latest version of the HDInsight tools for Visual Studio, see [Get started using HDInsight tools for Visual Studio](../hadoop/apache-hadoop-visual-studio-tools-get-started.md).
145133

146134
2. Download the solution from [eventhub-storm-hybrid](https://github.com/Azure-Samples/hdinsight-dotnet-java-storm-eventhub).
147135

148-
3. In the **EventHubWriter** project, open the **App.config** file. Use the information from the event hub that you configured earlier to fill in the value for the following keys:
136+
3. Open **EventHubExample.sln**. In the **EventHubWriter** project, open the **App.config** file. Use the information from the event hub that you configured earlier to fill in the value for the following keys:
149137

150138
| Key | Value |
151139
| --- | --- |
@@ -199,9 +187,11 @@ Event Hubs is the data source for this example. Use the information in the "Crea
199187

200188
9. In the **Executors** section, select one of the links in the **Port** column. This displays information logged by the component. The logged information is similar to the following text:
201189

202-
2017-03-02 14:51:29.255 m.s.p.TaskHost [INFO] Received C# STDOUT: 2017-03-02 14:51:29,255 [1] INFO EventHubReader_LogBolt [(null)] - Received data: {"deviceValue":1830978598,"deviceId":"8566ccbc-034d-45db-883d-d8a31f34068e"}
203-
2017-03-02 14:51:29.283 m.s.p.TaskHost [INFO] Received C# STDOUT: 2017-03-02 14:51:29,283 [1] INFO EventHubReader_LogBolt [(null)] - Received data: {"deviceValue":1756413275,"deviceId":"647a5eff-823d-482f-a8b4-b95b35ae570b"}
204-
2017-03-02 14:51:29.313 m.s.p.TaskHost [INFO] Received C# STDOUT: 2017-03-02 14:51:29,312 [1] INFO EventHubReader_LogBolt [(null)] - Received data: {"deviceValue":1108478910,"deviceId":"206a68fa-8264-4d61-9100-bfdb68ee8f0a"}
190+
```output
191+
2017-03-02 14:51:29.255 m.s.p.TaskHost [INFO] Received C# STDOUT: 2017-03-02 14:51:29,255 [1] INFO EventHubReader_LogBolt [(null)] - Received data: {"deviceValue":1830978598,"deviceId":"8566ccbc-034d-45db-883d-d8a31f34068e"}
192+
2017-03-02 14:51:29.283 m.s.p.TaskHost [INFO] Received C# STDOUT: 2017-03-02 14:51:29,283 [1] INFO EventHubReader_LogBolt [(null)] - Received data: {"deviceValue":1756413275,"deviceId":"647a5eff-823d-482f-a8b4-b95b35ae570b"}
193+
2017-03-02 14:51:29.313 m.s.p.TaskHost [INFO] Received C# STDOUT: 2017-03-02 14:51:29,312 [1] INFO EventHubReader_LogBolt [(null)] - Received data: {"deviceValue":1108478910,"deviceId":"206a68fa-8264-4d61-9100-bfdb68ee8f0a"}
194+
```
205195
206196
## Stop the topologies
207197
@@ -215,7 +205,7 @@ To stop the topologies, select each topology in the **Storm Topology Viewer**, t
215205
216206
## Next steps
217207
218-
In this document, you have learned how to use the Java Event Hubs spout and bolt from a C# topology to work with data in Azure Event Hubs. To learn more about creating C# topologies, see the following:
208+
In this document, you've learned how to use the Java Event Hubs spout and bolt from a C# topology to work with data in Azure Event Hubs. To learn more about creating C# topologies, see the following:
219209
220210
* [Develop C# topologies for Apache Storm on HDInsight using Visual Studio](apache-storm-develop-csharp-visual-studio-topology.md)
221211
* [SCP programming guide](apache-storm-scp-programming-guide.md)

0 commit comments

Comments
 (0)