Skip to content

Commit 4b637b2

Browse files
authored
Merge pull request #216297 from spelluru/ehubcapture1027
Freshness review, update steps and screenshots
2 parents fd60179 + 1bf4314 commit 4b637b2

File tree

8 files changed

+61
-38
lines changed

8 files changed

+61
-38
lines changed

articles/event-hubs/event-hubs-capture-enable-through-portal.md

Lines changed: 61 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -2,53 +2,72 @@
22
title: Event Hubs - Capture streaming events using Azure portal
33
description: This article describes how to enable capturing of events streaming through Azure Event Hubs by using the Azure portal.
44
ms.topic: quickstart
5-
ms.date: 09/16/2021
5+
ms.date: 10/27/2021
66
ms.custom: mode-ui
77
---
88

99
# Enable capturing of events streaming through Azure Event Hubs
1010

11-
Azure [Event Hubs Capture][capture-overview] enables you to automatically deliver the streaming data in Event Hubs to an [Azure Blob storage](https://azure.microsoft.com/services/storage/blobs/) or [Azure Data Lake Storage Gen1 or Gen 2](https://azure.microsoft.com/services/data-lake-store/) account of your choice.
12-
13-
You can configure Capture at the event hub creation time using the [Azure portal](https://portal.azure.com). You can either capture the data to an Azure [Blob storage](https://azure.microsoft.com/services/storage/blobs/) container, or to an [Azure Data Lake Storage Gen 1 or Gen 2](https://azure.microsoft.com/services/data-lake-store/) account.
14-
15-
For more information, see the [Event Hubs Capture overview][capture-overview].
11+
Azure [Event Hubs Capture][capture-overview] enables you to automatically deliver the streaming data in Event Hubs to an [Azure Blob storage](https://azure.microsoft.com/services/storage/blobs/) or [Azure Data Lake Storage Gen1 or Gen 2](https://azure.microsoft.com/services/data-lake-store/) account of your choice.You can configure capture settings using the [Azure portal](https://portal.azure.com) when creating an event hub or for an existing event hub. For conceptual information on this feature, see [Event Hubs Capture overview][capture-overview].
1612

1713
> [!IMPORTANT]
1814
> - The destination storage (Azure Storage or Azure Data Lake Storage) account must be in the same subscription as the event hub.
1915
> - Event Hubs doesn't support capturing events in a **premium** storage account.
2016
2117

22-
## Capture data to Azure Storage
18+
## Enable Capture when you create an event hub
2319

24-
When you create an event hub, you can enable Capture by clicking the **On** button in the **Create Event Hub** portal screen. You then specify a Storage Account and container by clicking **Azure Storage** in the **Capture Provider** box. Because Event Hubs Capture uses service-to-service authentication with storage, you do not need to specify a storage connection string. The resource picker selects the resource URI for your storage account automatically. If you use Azure Resource Manager, you must supply this URI explicitly as a string.
20+
If you don't have an Event Hubs namespace to work with, create a **standard** tier namespace by following steps from the article: [Create an Event Hubs namespace](event-hubs-create.md#create-an-event-hubs-namespace). Make sure that you select **Standard** for the **pricing tier**. The basic tier doesn't support the Capture feature.
2521

26-
The default time window is 5 minutes. The minimum value is 1, the maximum 15. The **Size** window has a range of 10-500 MB.
22+
To create an event hub within the namespace, follow these steps:
2723

28-
You can enable or disable emitting empty files when no events occur during the Capture window.
24+
1. On the **Overview** page for your namespace, select **+ Event hub** on the command bar.
25+
26+
:::image type="content" source="./media/event-hubs-quickstart-portal/create-event-hub4.png" lightbox="./media/event-hubs-quickstart-portal/create-event-hub4.png" alt-text="Screenshot of the selection of Add event hub button on the command bar.":::
27+
2. On the **Create event hub** page, type a name for your event hub, then select **Next: Capture** at the bottom of the page.
28+
29+
:::image type="content" source="./media/event-hubs-capture-enable-through-portal/create-event-hub-basics-page.png" alt-text="Screenshot of the Create event hub page.":::
30+
1. On the **Capture** tab, select **On** for **Capture**.
31+
1. Drag the slider to set the **Time window** in minutes. The default time window is 5 minutes. The minimum value is 1 and the maximum is 15.
32+
1. Drag the slider to set the **Size window (MB)**. The default value is 300 MB. The minimum value is 10 MB and the maximum value is 500 MB.
33+
1. Specify whether you want Event Hubs to **emit empty files when no events occur during the Capture time window**.
34+
35+
See one of the following sections based on the type of storage you want to use to store captured files.
36+
37+
## Capture data to Azure Storage
2938

30-
![Time window for capture][1]
39+
1. For **Capture Provider**, select **Azure Storage Account** (default).
40+
1. For **Azure Storage Container**, click the **Select the container** link.
3141

42+
:::image type="content" source="./media/event-hubs-capture-enable-through-portal/select-container-link.png" alt-text="Screenshot that shows the Create event hub page with the Select container link.":::
43+
1. On the **Storage accounts** page, select the storage account that you want to use to capture data.
44+
1. On the **Containers** page, select the container where you want to store captured files, and then click **Select**.
3245

46+
Because Event Hubs Capture uses service-to-service authentication with storage, you don't need to specify a storage connection string. The resource picker selects the resource URI for your storage account automatically. If you use Azure Resource Manager, you must supply this URI explicitly as a string.
47+
1. Now, on the **Create event hub** page, confirm that the selected container shows up.
48+
1. For **Capture file name format**, specify format for the captured file names.
49+
1. Select **Review + create** at the bottom of the page.
50+
1. On the **Review + create** page, review settings, and select **Create** to create the event hub.
51+
3352
## Capture data to Azure Data Lake Storage Gen 2
3453

35-
1. Follow [Create a storage account](../storage/common/storage-account-create.md?tabs=azure-portal#create-a-storage-account) article to create an Azure Storage account. Set **Hierarchical namespace** to **Enabled** on the **Advanced** tab to make it an Azure Data Lake Storage Gen 2 account. The Azure Storage account must be in the same subscription as the event hub.
36-
2. When creating an event hub, do the following steps:
54+
Follow [Create a storage account](../storage/common/storage-account-create.md?tabs=azure-portal#create-a-storage-account) article to create an Azure Storage account. Set **Hierarchical namespace** to **Enabled** on the **Advanced** tab to make it an Azure Data Lake Storage Gen 2 account. The Azure Storage account must be in the same subscription as the event hub.
3755

38-
1. Select **On** for **Capture**.
39-
2. Select **Azure Storage** as the capture provider. The **Azure Data Lake Store** option you see for the **Capture provider** is for the Gen 1 of Azure Data Lake Storage. To use a Gen 2 of Azure Data Lake Storage, you select **Azure Storage**.
40-
2. Select the **Select Container** button.
56+
1. Select **Azure Storage** as the capture provider. The **Azure Data Lake Store** option you see for the **Capture provider** is for the Gen 1 of Azure Data Lake Storage. To use a Gen 2 of Azure Data Lake Storage, you select **Azure Storage**.
57+
2. For **Azure Storage Container**, click the **Select the container** link.
4158

42-
![Enable capture to Data Lake Storage Gen 2](./media/event-hubs-capture-enable-through-portal/data-lake-storage-gen2.png)
59+
:::image type="content" source="./media/event-hubs-capture-enable-through-portal/select-container-link.png" alt-text="Screenshot that shows the Create event hub page with the Select container link.":::
4360
3. Select the **Azure Data Lake Storage Gen 2** account from the list.
4461

45-
![Select Data Lake Storage Gen 2](./media/event-hubs-capture-enable-through-portal/select-data-lake-storage-gen2.png)
46-
4. Select the **container** (file system in Data Lake Storage Gen 2).
62+
:::image type="content" source="./media/event-hubs-capture-enable-through-portal/select-data-lake-storage-gen2.png" alt-text="Screenshot showing the selection of Data Lake Storage Gen 2 account.":::
63+
4. Select the **container** (file system in Data Lake Storage Gen 2), and then click **Select** at the bottom of the page.
4764

48-
![Select file system in the storage](./media/event-hubs-capture-enable-through-portal/select-file-system-data-lake-storage.png)
49-
5. On the **Create Event Hub** page, select **Create**.
65+
:::image type="content" source="./media/event-hubs-capture-enable-through-portal/select-file-system-data-lake-storage.png" alt-text="Screenshot that shows the Containers page.":::
66+
1. For **Capture file name format**, specify format for the captured file names.
67+
1. Select **Review + create** at the bottom of the page.
5068

51-
![Select Create button](./media/event-hubs-capture-enable-through-portal/create-event-hub-data-lake-storage.png)
69+
:::image type="content" source="./media/event-hubs-capture-enable-through-portal/create-event-hub-data-lake-storage.png" alt-text="Screenshot that shows the Create event hub page with all the fields specified.":::
70+
1. On the **Review + create** page, review settings, and select **Create** to create the event hub.
5271

5372
> [!NOTE]
5473
> The container you create in a Azure Data Lake Storage Gen 2 using this user interface (UI) is shown under **File systems** in **Storage Explorer**. Similarly, the file system you create in a Data Lake Storage Gen 2 account shows up as a container in this UI.
@@ -58,6 +77,9 @@ You can enable or disable emitting empty files when no events occur during the C
5877

5978
To capture data to Azure Data Lake Storage Gen 1, you create a Data Lake Storage Gen 1 account, and an event hub:
6079

80+
> [!IMPORTANT]
81+
> On Feb 29, 2024 Azure Data Lake Storage Gen1 will be retired. For more information, see the [official announcement](https://azure.microsoft.com/updates/action-required-switch-to-azure-data-lake-storage-gen2-by-29-february-2024/). If you use Azure Data Lake Storage Gen1, make sure to migrate to Azure Data Lake Storage Gen2 prior to that date. For more information, see [Azure Data Lake Storage migration guidelines and patterns](../storage/blobs/data-lake-storage-migrate-gen1-to-gen2.md).
82+
6183
### Create an Azure Data Lake Storage Gen 1 account and folders
6284

6385
1. Create a Data Lake Storage account, following the instructions in [Get started with Azure Data Lake Storage Gen 1 using the Azure portal](../data-lake-store/data-lake-store-get-started-portal.md).
@@ -67,31 +89,32 @@ To capture data to Azure Data Lake Storage Gen 1, you create a Data Lake Storage
6789
### Create an event hub
6890

6991
1. The event hub must be in the same Azure subscription as the Azure Data Lake Storage Gen 1 account you created. Create the event hub, clicking the **On** button under **Capture** in the **Create Event Hub** portal page.
70-
2. In the **Create Event Hub** portal page, select **Azure Data Lake Store** from the **Capture Provider** box.
92+
2. On the **Create Event Hub** page, select **Azure Data Lake Store** from the **Capture Provider** box.
7193
3. In **Select Store** next to the **Data Lake Store** drop-down list, specify the Data Lake Storage Gen 1 account you created previously, and in the **Data Lake Path** field, enter the path to the data folder you created.
7294

73-
![Select Data Lake Storage account][3]
74-
75-
76-
## Add or configure Capture on an existing event hub
77-
78-
You can configure Capture on existing event hubs that are in Event Hubs namespaces. To enable Capture on an existing event hub, or to change your Capture settings, click the namespace to load the overview screen, then click the event hub for which you want to enable or change the Capture setting. Finally, click the **Capture** option on the left side of the open page and then edit the settings, as shown in the following figures:
95+
:::image type="content" source="./media/event-hubs-capture-enable-through-portal/event-hubs-capture3.png" alt-text="Screenshot showing the selection of Data Lake Storage Account Gen 1.":::
7996

80-
### Azure Blob Storage
8197

82-
:::image type="content" source="./media/event-hubs-capture-enable-through-portal/event-hubs-capture2.png" alt-text="Configure capture to an Azure storage for an existing event hub.":::
98+
## Configure Capture for an existing event hub
8399

84-
### Azure Data Lake Storage Gen 2
85-
It's same as above (for Azure Blob Storage) except that you will be selecting a container from an Azure Data Lake Storage Gen 2 account.
100+
You can configure Capture on existing event hubs that are in Event Hubs namespaces. To enable Capture on an existing event hub, or to change your Capture settings, follow these steps:
86101

87-
### Azure Data Lake Storage Gen 1
102+
1. On the home page for your namespace, select **Event Hubs** under **Entities** on the left menu.
103+
1. Select the event hub for which you want to configure the Capture feature.
88104

89-
![Configure Azure Data Lake Storage][4]
105+
:::image type="content" source="./media/event-hubs-capture-enable-through-portal/select-event-hub.png" alt-text="Screenshot showing the selection of an event hub in the list of event hubs.":::
106+
1. On the **Event Hubs Instance** page, select **Capture** on the left menu.
90107

91-
[1]: ./media/event-hubs-capture-enable-through-portal/event-hubs-capture1.png
92-
[3]: ./media/event-hubs-capture-enable-through-portal/event-hubs-capture3.png
93-
[4]: ./media/event-hubs-capture-enable-through-portal/event-hubs-capture4.png
108+
:::image type="content" source="./media/event-hubs-capture-enable-through-portal/view-capture-page.png" alt-text="Screenshot showing the Capture page for your event hub.":::
109+
1. On the **Capture** page, select **Avro** for **Output event serialization format**. The **Parquet** format is supported only via Azure Stream Analytics integration. For more information, see [Capture Event Hubs data in parquet format and analyze with Azure Synapse Analytics](../stream-analytics/event-hubs-parquet-capture-tutorial.md).
110+
1. Select **On** for **Capture**.
94111

112+
:::image type="content" source="./media/event-hubs-capture-enable-through-portal/enable-capture.png" alt-text="Screenshot showing the Capture page for your event hub with the Capture feature enabled.":::
113+
1. To configure other settings, see the sections:
114+
- [Capture data to Azure Storage](#capture-data-to-azure-storage)
115+
- [Capture data to Azure Data Lake Storage Gen 2](#capture-data-to-azure-data-lake-storage-gen-2)
116+
- [Capture data to Azure Data Lake Storage Gen 1](#capture-data-to-azure-data-lake-storage-gen-1)
117+
95118
## Next steps
96119

97120
- Learn more about Event Hubs capture by reading the [Event Hubs Capture overview][capture-overview].
23 KB
Loading
-6.68 KB
Loading
59.9 KB
Loading
23.7 KB
Loading
63.8 KB
Loading
37.5 KB
Loading
51.8 KB
Loading

0 commit comments

Comments
 (0)