Skip to content

Commit 356eeb0

Browse files
Freshness.
1 parent 0a3a219 commit 356eeb0

File tree

3 files changed

+34
-34
lines changed

3 files changed

+34
-34
lines changed

articles/event-grid/event-hubs-integration.md

Lines changed: 34 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,11 @@
11
---
22
title: 'Tutorial: Send Event Hubs data to data warehouse - Event Grid'
3-
description: Shows how to migrate Event Hubs captured data from Azure Blob Storage to Azure Synapse Analytics, specifically a dedicated SQL pool, using Azure Event Grid and Azure Functions.
3+
description: Learn to migrate Event Hubs captured data from Azure Blob Storage to Azure Synapse Analytics using Azure Event Grid and Azure Functions.
44
ms.topic: tutorial
5-
ms.date: 01/31/2024
5+
ms.date: 06/26/2025
66
ms.devlang: csharp
77
ms.custom: devx-track-csharp
8+
#customer intent: As a developer, I want to learn to migrate data captured by event hubs from storage to analytics to support apps.
89
---
910

1011
# Tutorial: Migrate Event Hubs captured data from Azure Storage to Azure Synapse Analytics using Azure Event Grid and Azure Functions
@@ -31,22 +32,21 @@ In this article, you take the following steps:
3132
> - Verify captured data in Azure Synapse Analytics
3233
3334
## Prerequisites
34-
To complete this tutorial, you must have:
3535

36-
- This article assumes that you're familiar with Event Grid and Event Hubs (especially the Capture feature). If you aren't familiar with Azure Event Grid, see [Introduction to Azure Event Grid](overview.md). To learn about the Capture feature of Azure Event Hubs, see [Capture events through Azure Event Hubs in Azure Blob Storage or Azure Data Lake Storage](../event-hubs/event-hubs-capture-overview.md).
36+
- Knowledge of Azure Event Grid and Azure Event Hubs, especially the Capture feature. If you aren't familiar with Azure Event Grid, see [Introduction to Azure Event Grid](overview.md). To learn about the Capture feature of Azure Event Hubs, see [Capture events through Azure Event Hubs in Azure Blob Storage or Azure Data Lake Storage](../event-hubs/event-hubs-capture-overview.md).
3737
- An Azure subscription. If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/) before you begin.
38-
- [Visual Studio](https://www.visualstudio.com/vs/) with workloads for: .NET desktop development, Azure development, ASP.NET and web development, Node.js development, and Python development.
39-
- Download the [EventHubsCaptureEventGridDemo sample project](https://github.com/Azure/azure-event-hubs/tree/master/samples/e2e/EventHubsCaptureEventGridDemo) to your computer.
38+
- [Visual Studio](https://www.visualstudio.com/vs/) with workloads for .NET desktop development, Azure development, ASP.NET and web development, Node.js development, and Python development.
39+
- The [EventHubsCaptureEventGridDemo sample project](https://github.com/Azure/azure-event-hubs/tree/master/samples/e2e/EventHubsCaptureEventGridDemo). The sample includes:
4040

41-
- WindTurbineDataGenerator A simple publisher that sends sample wind turbine data to an event hub with the Capture feature enabled.
42-
- FunctionDWDumperAn Azure function that receives a notification from Azure Event Grid when an Avro file is captured to the Azure Storage blob. It receives the blob’s URI path, reads its contents, and pushes this data to Azure Synapse Analytics (dedicated SQL pool).
41+
- WindTurbineDataGenerator. A simple publisher that sends sample wind turbine data to an event hub with the Capture feature enabled.
42+
- FunctionDWDumper. An Azure function that receives a notification from Azure Event Grid when an Avro file is captured to the Azure Storage blob. It receives the blob URI, reads its contents, and pushes this data to Azure Synapse Analytics.
4343

4444
## Deploy the infrastructure
4545

46-
In this step, you deploy the required infrastructure with a [Resource Manager template](https://github.com/Azure/azure-docs-json-samples/blob/master/event-grid/EventHubsDataMigration.json). When you deploy the template, the following resources are created:
46+
In this section, deploy the required infrastructure with a [Resource Manager template](https://github.com/Azure/azure-docs-json-samples/blob/master/event-grid/EventHubsDataMigration.json). When you deploy the template, the following resources are created:
4747

48-
- Event hub with the Capture feature enabled.
49-
- Storage account for the captured files.
48+
- Event hub with the Capture feature enabled
49+
- Storage account for the captured files
5050
- App service plan for hosting the function app
5151
- Function app for processing the event
5252
- SQL Server for hosting the data warehouse
@@ -69,7 +69,7 @@ In this step, you deploy the required infrastructure with a [Resource Manager te
6969

7070
1. Create a storage account by selecting **Create storage**. Azure Cloud Shell requires an Azure storage account to store some files.
7171

72-
::image type="content" source="media/event-hubs-functions-synapse-analytics/create-storage-cloud-shell.png" alt-text="Screenshot showing the creation of storage for Cloud Shell.":::
72+
:::image type="content" source="media/event-hubs-functions-synapse-analytics/create-storage-cloud-shell.png" alt-text="Screenshot showing the creation of storage for Cloud Shell.":::
7373

7474
1. Wait until the Cloud Shell is initialized.
7575

@@ -102,14 +102,14 @@ In this step, you deploy the required infrastructure with a [Resource Manager te
102102
"tags": null,
103103
"type": "Microsoft.Resources/resourceGroups"
104104
}
105-
```
105+
```
106106
107-
1. Deploy all the resources mentioned in the previous section (event hub, storage account, functions app, Azure Synapse Analytics) by running the following CLI command:
107+
1. Deploy all the resources mentioned in the previous section by running the following CLI command. The resources include an event hub, storage account, functions app, and Azure Synapse Analytics.
108108
109-
1. Copy and paste the command into the Cloud Shell window. Alternatively, you can copy/paste into an editor of your choice, set values, and then copy the command to the Cloud Shell. If you see an error due to an Azure resource name, delete the resource group, fix the name, and retry the command again.
109+
1. Copy and paste the command into the Cloud Shell window. Alternatively, you can copy into a text editor, set values, and then copy the command to the Cloud Shell. If you see an error due to an Azure resource name, delete the resource group, fix the name, and retry the command.
110110
111111
> [!IMPORTANT]
112-
> Specify values for the following entities before running the command:
112+
> Specify values for the following entities before you run the command:
113113
>
114114
> - Name of the resource group you created earlier.
115115
> - Name for the event hub namespace.
@@ -128,14 +128,14 @@ In this step, you deploy the required infrastructure with a [Resource Manager te
128128
--parameters eventHubNamespaceName=<event-hub-namespace> eventHubName=hubdatamigration sqlServerName=<sql-server-name> sqlServerUserName=<user-name> sqlServerPassword=<password> sqlServerDatabaseName=<database-name> storageName=<unique-storage-name> functionAppName=<app-name>
129129
```
130130
131-
1. Press **ENTER** in the Cloud Shell window to run the command. This process might take a while since you're creating a bunch of resources. In the result of the command, ensure that there are no failures.
131+
1. Press **Enter** in the Cloud Shell window to run the command. This process might take a while since you're creating a bunch of resources. In the result of the command, ensure that there are no failures.
132132
133-
1. Close the Cloud Shell by selecting the **Cloud Shell** button in the portal (or) **X** button in the top-right corner of the Cloud Shell window.
133+
1. Close the Cloud Shell by selecting the **Cloud Shell** button in the portal or the **X** button in the top-right corner of the Cloud Shell window.
134134
135135
### Verify that the resources are created
136136
137137
1. In the Azure portal, in the search box, enter *Resource groups*. From the results, select **Resource groups**.
138-
1. Filter the list of resource groups by entering the name of your resource group in the search box.
138+
1. If necessary, in the search box, enter the name of your resource group to filter the resource groups.
139139
1. Select your resource group in the list.
140140
141141
:::image type="content" source="media/event-hubs-functions-synapse-analytics/select-resource-group.png" alt-text="Screenshot showing the selection of your resource group.":::
@@ -148,12 +148,12 @@ In this step, you deploy the required infrastructure with a [Resource Manager te
148148
149149
In this section, you create a table in the dedicated SQL pool you created earlier.
150150
151-
1. In the list of resources in the resource group, select your **dedicated SQL pool**.
152-
1. On the **Dedicated SQL pool** page, in the **Common Tasks** section on the left menu, select **Query editor (preview)**.
151+
1. In the list of resources in the resource group, select your *dedicated SQL pool*.
152+
1. On the **Dedicated SQL pool** page, under **Common Tasks**, select **Query editor (preview)**.
153153
154154
:::image type="content" source="media/event-hubs-functions-synapse-analytics/sql-data-warehouse-page.png" alt-text="Screenshot showing the selection of Query Editor on a Dedicated SQL pool page in the Azure portal.":::
155155
156-
1. Enter the name of **user** and **password** for the SQL server, and select **OK**. If you see a message about allowing your client to access the SQL server, select **Allowlist IP &lt;your IP Address&gt; on server &lt;your SQL server&gt;**, and then select **OK**.
156+
1. Enter the user name and password for the SQL server, and select **OK**. If you see a message about allowing your client to access the SQL server, select **Allowlist IP &lt;your IP Address&gt; on server &lt;your SQL server&gt;**, and then select **OK**.
157157
1. In the query window, copy and run the following SQL script:
158158
159159
```sql
@@ -177,11 +177,11 @@ First, get the publish profile for the Functions app from the Azure portal. Then
177177

178178
### Get the publish profile
179179

180-
1. On the **Resource Group** page, select the **Azure Functions app** in the list of resources.
180+
1. On the **Resource Group** page, select the **Azure Functions app**.
181181

182182
:::image type="content" source="media/event-hubs-functions-synapse-analytics/select-function-app.png" lightbox="media/event-hubs-functions-synapse-analytics/select-function-app.png" alt-text="Screenshot showing the selection of the function app in the list of resources for a resource group.":::
183183

184-
1. On the **Function App** page for your app, select **Get publish profile** on the command bar.
184+
1. On the **Function App** page for your app, select **Get publish profile**.
185185

186186
:::image type="content" source="media/event-hubs-functions-synapse-analytics/get-publish-profile.png" lightbox="media/event-hubs-functions-synapse-analytics/get-publish-profile.png" alt-text="Screenshot showing the selection of the **Get Publish Profile** button on the command bar of the function app page.":::
187187

@@ -197,7 +197,7 @@ First, get the publish profile for the Functions app from the Azure portal. Then
197197

198198
:::image type="content" source="media/event-hubs-functions-synapse-analytics/import-profile.png" alt-text="Screenshot showing the selection **Import Profile** on the **Publish** dialog box.":::
199199

200-
1. On the **Import profile** tab, select the publish settings file that you saved earlier in the **FunctionEGDWDumper** folder, and then select **Finish**.
200+
1. On the **Import profile** tab, select the publish settings file that you saved earlier in the **FunctionEGDWDumper** folder. Then select **Finish**.
201201
1. When Visual Studio configures the profile, select **Publish**. Confirm that the publishing succeeded.
202202
1. In the web browser that has the **Azure Function** page open, select **Functions** in the middle pane. Confirm that the **EventGridTriggerMigrateData** function shows up in the list. If you don't see it, try publishing from Visual Studio again, and then refresh the page in the portal.
203203

@@ -208,18 +208,18 @@ After publishing the function, you're ready to subscribe to the event.
208208
## Subscribe to the event
209209

210210
1. In a new tab or new window of a web browser, sign in to the [Azure portal](https://portal.azure.com).
211-
1. In the Azure portal, select **Resource groups** on the left menu.
212-
1. Filter the list of resource groups by entering the name of your resource group in the search box.
211+
1. In the Azure portal, search for and select **Resource groups**.
212+
1. If necessary, in the search box, enter the name of your resource group to filter the resource groups.
213213
1. Select your resource group in the list.
214214
1. Select the **Event Hubs namespace** from the list of resources.
215-
1. On the **Event Hubs Namespace** page, select **Events** on the left menu, and then select **+ Event Subscription** on the toolbar.
215+
1. On the **Event Hubs Namespace** page, select **Events**, and then select **+ Event Subscription** on the toolbar.
216216

217217
:::image type="content" source="media/event-hubs-functions-synapse-analytics/event-hub-add-subscription-link.png" alt-text="Screenshot of the Events page for an Event Hubs namespace with Add event subscription link selected. ":::
218218

219219
1. On the **Create Event Subscription** page, follow these steps:
220220

221221
1. Enter a name for the **event subscription**.
222-
1. Enter a name for the **system topic**. A system topic provides an endpoint for the sender to send events. For more information, see [System topics](system-topics.md)
222+
1. Enter a name for the **system topic**. A system topic provides an endpoint for the sender to send events. For more information, see [System topics](system-topics.md).
223223
1. For **Endpoint Type**, select **Azure Function**.
224224
1. For **Endpoint**, select the link.
225225
1. On the **Select Azure Function** page, follow these steps if they aren't automatically filled.
@@ -241,16 +241,16 @@ After publishing the function, you're ready to subscribe to the event.
241241

242242
## Run the app to generate data
243243

244-
You finished setting up your event hub, dedicate SQL pool (formerly SQL Data Warehouse), Azure function app, and event subscription. Before running an application that generates data for event hub, you need to configure a few values.
244+
You finished setting up your event hub, dedicate SQL pool (formerly SQL Data Warehouse), Azure function app, and event subscription. Before you run an application that generates data for event hub, configure a few values.
245245

246246
1. In the Azure portal, navigate to your resource group as you did earlier.
247247
1. Select the Event Hubs namespace.
248-
1. In the **Event Hubs Namespace** page, select **Shared access policies** on the left menu.
248+
1. In the **Event Hubs Namespace** page, select **Shared access policies**.
249249
1. Select **RootManageSharedAccessKey** in the list of policies.
250250

251251
:::image type="content" source="media/event-hubs-functions-synapse-analytics/event-hub-namespace-shared-access-policies.png" alt-text="Screenshot showing the Shared access policies page for an Event Hubs namespace.":::
252252

253-
1. Select the copy button next to the **Connection string-primary key** text box.
253+
1. Select the copy button next to **Primary connection string**.
254254
1. Go back to your Visual Studio solution.
255255
1. Right-click **WindTurbineDataGenerator** project, and select **Set as Startup project**.
256256
1. In the WindTurbineDataGenerator project, open **program.cs**.
@@ -272,7 +272,7 @@ You finished setting up your event hub, dedicate SQL pool (formerly SQL Data War
272272
:::image type="content" source="media/event-hubs-functions-synapse-analytics/query-results.png" alt-text="Screenshot showing the query results.":::
273273

274274
> [!IMPORTANT]
275-
> We use connection string to authenticate to Azure Event Hubs namespace to keep the tutorial simple. We recommend that you use Microsoft Entra ID authentication in production environments. When using an application, you can enable managed identity for the application and assign the identity an appropriate role (Azure Event Hubs Owner, Azure Event Hubs Data Sender, or Azure Event Hubs Data Receiver) on the Event Hubs namespace. For more information, see [Authorize access to Event Hubs using Microsoft Entra ID](../event-hubs/authorize-access-azure-active-directory.md).
275+
> In this example, using the connection string to authenticate to Azure Event Hubs namespace keeps the tutorial simple. We recommend that you use Microsoft Entra ID authentication in production environments. When you use an application, enable managed identity for the application and assign the identity an appropriate role (Azure Event Hubs Owner, Azure Event Hubs Data Sender, or Azure Event Hubs Data Receiver) on the Event Hubs namespace. For more information, see [Authorize access to Event Hubs using Microsoft Entra ID](../event-hubs/authorize-access-azure-active-directory.md).
276276
277277
## Monitor the solution
278278

@@ -281,7 +281,7 @@ This section helps you with monitoring or troubleshooting the solution.
281281
### View captured data in the storage account
282282

283283
1. Navigate to the resource group and select the storage account used for capturing event data.
284-
1. On the **Storage account** page, select **Storage browser** on the left menu.
284+
1. On the **Storage account** page, select **Storage browser**.
285285
1. Expand **BLOB CONTAINERS**, and select **windturbinecapture**.
286286
1. Open the folder named same as your **Event Hubs namespace** in the right pane.
287287
1. Open the folder named same as your event hub (**hubdatamigration**).
-33.7 KB
Loading
-10.9 KB
Loading

0 commit comments

Comments
 (0)