You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/stream-analytics/capture-event-hub-data-parquet.md
+17-8Lines changed: 17 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,12 +1,13 @@
1
1
---
2
-
title: Capture data from Event Hubs into Azure Data Lake Storage Gen2 in Parquet format
2
+
title: Event Hubs to Azure Data Lake in Parquet format
3
3
description: Learn how to use the node code editor to automatically capture the streaming data in Event Hubs in an Azure Data Lake Storage Gen2 account in Parquet format.
4
4
author: xujxu
5
5
ms.author: xujiang1
6
6
ms.service: azure-stream-analytics
7
7
ms.topic: how-to
8
8
ms.custom: mvc
9
-
ms.date: 08/15/2023
9
+
ms.date: 01/23/2025
10
+
# Customer intent: I want to lean how to use the no-code editor to automatically capture streaming data in Azure Event Hubs to an Azure Data Lake Storage Gen2 account in the Parquet format.
10
11
---
11
12
# Capture data from Event Hubs in Parquet format
12
13
This article explains how to use the no code editor to automatically capture streaming data in Event Hubs in an Azure Data Lake Storage Gen2 account in the Parquet format.
@@ -18,9 +19,13 @@ This article explains how to use the no code editor to automatically capture str
18
19
If you don't have an event hub, create one by following instructions from [Quickstart: Create an event hub](../event-hubs/event-hubs-create.md).
19
20
20
21
If you don't have a Data Lake Storage Gen2 account, create one by following instructions from [Create a storage account](../storage/blobs/create-data-lake-storage-account.md)
21
-
- The data in your Event Hubs must be serialized in either JSON, CSV, or Avro format. For testing purposes, select **Generate data (preview)** on the left menu, select **Stocks data** for dataset, and then select **Send**.
22
+
- The data in your Event Hubs instance (event hub) must be serialized in either JSON, CSV, or Avro format. On the **Event Hubs Instance** page for your event hub, follow these steps:
23
+
1. On the left menu, select **Data Explorer**.
24
+
1. In the middle pane, select **Send events**.
25
+
1. In the **Send events** pane, for **Select dataset**, select **Stocks data**.
26
+
1. Select **Send**.
22
27
23
-
:::image type="content" source="./media/capture-event-hub-data-parquet/stocks-data.png" alt-text="Screenshot showing the Generate data page to generate sample stocks data." lightbox="./media/capture-event-hub-data-parquet/stocks-data.png":::
28
+
:::image type="content" source="./media/capture-event-hub-data-parquet/stocks-data.png" alt-text="Screenshot showing the Generate data page to generate sample stocks data." lightbox="./media/capture-event-hub-data-parquet/stocks-data.png":::
24
29
25
30
## Configure a job to capture data
26
31
@@ -33,7 +38,7 @@ Use the following steps to configure a Stream Analytics job to capture data in A
33
38
1. Enter a **name** for your Stream Analytics job, and then select **Create**.
34
39
35
40
:::image type="content" source="./media/capture-event-hub-data-parquet/new-stream-analytics-job-name.png" alt-text="Screenshot showing the New Stream Analytics job window where you enter the job name." :::
36
-
1. Specify the **Serialization** type of your data in the Event Hubs and the **Authentication method** that the job uses to connect to Event Hubs. Then select **Connect**.
41
+
1. Specify the **Serialization** type of your data in the Event Hubs and the **Authentication method** that the job uses to connect to Event Hubs. For this tutorial, keep the default settings. Then select **Connect**.
1. When the connection is established successfully, you see:
@@ -69,11 +74,15 @@ Use the following steps to configure a Stream Analytics job to capture data in A
69
74
70
75
## Verify output
71
76
72
-
1. On the Event Hubs instance page for your event hub, select **Generate data**, select **Stocks data** for dataset, and then select **Send** to send some sample data to the event hub.
77
+
1. On the Event Hubs instance page for your event hub, follow these steps:
78
+
1. On the left menu, select **Data Explorer**.
79
+
1. In the middle pane, select **Send events**.
80
+
1. In the **Send events** pane, for **Select dataset**, select **Stocks data**.
81
+
1. Select **Send**.
73
82
1. Verify that the Parquet files are generated in the Azure Data Lake Storage container.
74
83
75
84
:::image type="content" source="./media/capture-event-hub-data-parquet/verify-captured-data.png" alt-text="Screenshot showing the generated Parquet files in the Azure Data Lake Storage container." lightbox="./media/capture-event-hub-data-parquet/verify-captured-data.png" :::
76
-
1.Select **Process data** on the left menu. Switch to the **Stream Analytics jobs** tab. Select **Open metrics** to monitor it.
85
+
1.Back on the Event Hubs instance page, select **Process data** on the left menu. Switch to the **Stream Analytics jobs** tab. Select **Open metrics** to monitor it. Add **Input metrics** to the chart using the **Add metric** on the toolbar. If you don't see the metrics in the chart, wait for a few minutes, and refresh the page.
77
86
78
87
:::image type="content" source="./media/capture-event-hub-data-parquet/open-metrics-link.png" alt-text="Screenshot showing Open Metrics link selected." lightbox="./media/capture-event-hub-data-parquet/open-metrics-link.png" :::
79
88
@@ -84,7 +93,7 @@ Use the following steps to configure a Stream Analytics job to capture data in A
Now you know how to use the Stream Analytics no code editor to create a job that captures Event Hubs data to Azure Data Lake Storage Gen2 in Parquet format. Next, you can learn more about Azure Stream Analytics and how to monitor the job that you created.
0 commit comments