Skip to content

Commit 6193d21

Browse files
authored
Merge pull request #271849 from spelluru/ehubadlsgen10410
ADLS Gen 1 is retired, so don't use it capturing event data
2 parents 1b72f9f + 01409f8 commit 6193d21

File tree

3 files changed

+16
-87
lines changed

3 files changed

+16
-87
lines changed

articles/event-hubs/event-hubs-capture-enable-through-portal.md

Lines changed: 7 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ ms.custom: mode-ui
88

99
# Enable capturing of events streaming through Azure Event Hubs
1010

11-
Azure [Event Hubs Capture][capture-overview] enables you to automatically deliver the streaming data in Event Hubs to an [Azure Blob storage](https://azure.microsoft.com/services/storage/blobs/) or [Azure Data Lake Storage Gen1 or Gen 2](https://azure.microsoft.com/services/data-lake-store/) account of your choice.You can configure capture settings using the [Azure portal](https://portal.azure.com) when creating an event hub or for an existing event hub. For conceptual information on this feature, see [Event Hubs Capture overview][capture-overview].
11+
Azure [Event Hubs Capture][capture-overview] enables you to automatically deliver the streaming data in Event Hubs to an [Azure Blob storage](https://azure.microsoft.com/services/storage/blobs/) or [Azure Data Lake Storage Gen 2](https://azure.microsoft.com/services/data-lake-store/) account of your choice. You can configure capture settings using the [Azure portal](https://portal.azure.com) when creating an event hub or for an existing event hub. For conceptual information on this feature, see [Event Hubs Capture overview][capture-overview].
1212

1313
> [!IMPORTANT]
1414
> Event Hubs doesn't support capturing events in a **premium** storage account.
@@ -33,6 +33,10 @@ To create an event hub within the namespace, follow these steps:
3333

3434
See one of the following sections based on the type of storage you want to use to store captured files.
3535

36+
37+
> [!IMPORTANT]
38+
> Azure Data Lake Storage Gen1 is retired, so don't use it for capturing event data. For more information, see the [official announcement](https://azure.microsoft.com/updates/action-required-switch-to-azure-data-lake-storage-gen2-by-29-february-2024/). If you are using Azure Data Lake Storage Gen1, migrate to Azure Data Lake Storage Gen2. For more information, see [Azure Data Lake Storage migration guidelines and patterns](../storage/blobs/data-lake-storage-migrate-gen1-to-gen2.md).
39+
3640
## Capture data to Azure Storage
3741

3842
1. For **Capture Provider**, select **Azure Storage Account** (default).
@@ -55,7 +59,7 @@ See one of the following sections based on the type of storage you want to use t
5559

5660
Follow [Create a storage account](../storage/common/storage-account-create.md?tabs=azure-portal#create-a-storage-account) article to create an Azure Storage account. Set **Hierarchical namespace** to **Enabled** on the **Advanced** tab to make it an Azure Data Lake Storage Gen 2 account. The Azure Storage account must be in the same subscription as the event hub.
5761

58-
1. Select **Azure Storage** as the capture provider. The **Azure Data Lake Store** option you see for the **Capture provider** is for the Gen 1 of Azure Data Lake Storage. To use a Gen 2 of Azure Data Lake Storage, you select **Azure Storage**.
62+
1. Select **Azure Storage** as the capture provider. To use Azure Data Lake Storage Gen2, you select **Azure Storage**.
5963
2. For **Azure Storage Container**, click the **Select the container** link.
6064

6165
:::image type="content" source="./media/event-hubs-capture-enable-through-portal/select-container-link.png" alt-text="Screenshot that shows the Create event hub page with the Select container link.":::
@@ -75,28 +79,6 @@ Follow [Create a storage account](../storage/common/storage-account-create.md?ta
7579
> The container you create in an Azure Data Lake Storage Gen 2 using this user interface (UI) is shown under **File systems** in **Storage Explorer**. Similarly, the file system you create in a Data Lake Storage Gen 2 account shows up as a container in this UI.
7680
7781

78-
## Capture data to Azure Data Lake Storage Gen 1
79-
80-
To capture data to Azure Data Lake Storage Gen 1, you create a Data Lake Storage Gen 1 account, and an event hub:
81-
82-
> [!IMPORTANT]
83-
> On Feb 29, 2024 Azure Data Lake Storage Gen1 will be retired. For more information, see the [official announcement](https://azure.microsoft.com/updates/action-required-switch-to-azure-data-lake-storage-gen2-by-29-february-2024/). If you use Azure Data Lake Storage Gen1, make sure to migrate to Azure Data Lake Storage Gen2 prior to that date. For more information, see [Azure Data Lake Storage migration guidelines and patterns](../storage/blobs/data-lake-storage-migrate-gen1-to-gen2.md).
84-
85-
### Create an Azure Data Lake Storage Gen 1 account and folders
86-
87-
1. Create a Data Lake Storage account, following the instructions in [Get started with Azure Data Lake Storage Gen 1 using the Azure portal](../data-lake-store/data-lake-store-get-started-portal.md).
88-
2. Follow the instructions in the [Assign permissions to Event Hubs](../data-lake-store/data-lake-store-archive-eventhub-capture.md#assign-permissions-to-event-hubs) section to create a folder within the Data Lake Storage Gen 1 account in which you want to capture the data from Event Hubs, and assign permissions to Event Hubs so that it can write data into your Data Lake Storage Gen 1 account.
89-
90-
91-
### Create an event hub
92-
93-
1. The event hub must be in the same Azure subscription as the Azure Data Lake Storage Gen 1 account you created. Create the event hub, clicking the **On** button under **Capture** in the **Create Event Hub** portal page.
94-
2. On the **Create Event Hub** page, select **Azure Data Lake Store** from the **Capture Provider** box.
95-
3. In **Select Store** next to the **Data Lake Store** drop-down list, specify the Data Lake Storage Gen 1 account you created previously, and in the **Data Lake Path** field, enter the path to the data folder you created.
96-
97-
:::image type="content" source="./media/event-hubs-capture-enable-through-portal/event-hubs-capture3.png" alt-text="Screenshot showing the selection of Data Lake Storage Account Gen 1.":::
98-
99-
10082
## Configure Capture for an existing event hub
10183

10284
You can configure Capture on existing event hubs that are in Event Hubs namespaces. To enable Capture on an existing event hub, or to change your Capture settings, follow these steps:
@@ -114,8 +96,7 @@ You can configure Capture on existing event hubs that are in Event Hubs namespac
11496
:::image type="content" source="./media/event-hubs-capture-enable-through-portal/enable-capture.png" alt-text="Screenshot showing the Capture page for your event hub with the Capture feature enabled.":::
11597
1. To configure other settings, see the sections:
11698
- [Capture data to Azure Storage](#capture-data-to-azure-storage)
117-
- [Capture data to Azure Data Lake Storage Gen 2](#capture-data-to-azure-data-lake-storage-gen-2)
118-
- [Capture data to Azure Data Lake Storage Gen 1](#capture-data-to-azure-data-lake-storage-gen-1)
99+
- [Capture data to Azure Data Lake Storage Gen 2](#capture-data-to-azure-data-lake-storage-gen-2)
119100

120101
## Next steps
121102

articles/event-hubs/event-hubs-resource-manager-namespace-event-hub-enable-capture.md

Lines changed: 9 additions & 61 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
title: Create an event hub with capture enabled - Azure Event Hubs | Microsoft Docs
3-
description: Create an Azure Event Hubs namespace with one event hub and enable Capture using Azure Resource Manager template
3+
description: Create an Azure Event Hubs namespace with one event hub and enable Capture using Azure Resource Manager template.
44
ms.topic: quickstart
55
ms.date: 08/26/2022
66
ms.custom: mode-other, devx-track-azurecli, devx-track-arm-template
@@ -19,13 +19,14 @@ For more information about patterns and practices for Azure Resources naming con
1919

2020
For the complete templates, select the following GitHub links:
2121

22-
- [Event hub and enable Capture to Storage template][Event Hub and enable Capture to Storage template]
23-
- [Event hub and enable Capture to Azure Data Lake Store template][Event Hub and enable Capture to Azure Data Lake Store template]
22+
- [Create an event hub and enable Capture to Storage template][Event Hub and enable Capture to Storage template]
23+
- [Create an event hub and enable Capture to Azure Data Lake Store template][Event Hub and enable Capture to Azure Data Lake Store template]
2424

2525
> [!NOTE]
2626
> To check for the latest templates, visit the [Azure Quickstart Templates][Azure Quickstart Templates] gallery and search for Event Hubs.
27-
>
28-
>
27+
28+
> [!IMPORTANT]
29+
> Azure Data Lake Storage Gen1 is retired, so don't use it for capturing event data. For more information, see the [official announcement](https://azure.microsoft.com/updates/action-required-switch-to-azure-data-lake-storage-gen2-by-29-february-2024/). If you are using Azure Data Lake Storage Gen1, migrate to Azure Data Lake Storage Gen2. For more information, see [Azure Data Lake Storage migration guidelines and patterns](../storage/blobs/data-lake-storage-migrate-gen1-to-gen2.md).
2930
3031
## What will you deploy?
3132

@@ -166,7 +167,7 @@ The size interval at which Capture starts capturing the data.
166167

167168
### captureNameFormat
168169

169-
The name format used by Event Hubs Capture to write the Avro files. Note that a Capture name format must contain `{Namespace}`, `{EventHub}`, `{PartitionId}`, `{Year}`, `{Month}`, `{Day}`, `{Hour}`, `{Minute}`, and `{Second}` fields. These can be arranged in any order, with or without delimiters.
170+
The name format used by Event Hubs Capture to write the Avro files. The capture name format must contain `{Namespace}`, `{EventHub}`, `{PartitionId}`, `{Year}`, `{Month}`, `{Day}`, `{Hour}`, `{Minute}`, and `{Second}` fields. These fields can be arranged in any order, with or without delimiters.
170171

171172
```json
172173
"captureNameFormat": {
@@ -221,8 +222,6 @@ The blob container in which to capture your event data.
221222
}
222223
```
223224

224-
Use the following parameters if you choose Azure Data Lake Store Gen 1 as your destination. You must set permissions on your Data Lake Store path, in which you want to Capture the event. To set permissions, see [Capture data to Azure Data Lake Storage Gen 1](event-hubs-capture-enable-through-portal.md#capture-data-to-azure-data-lake-storage-gen-1).
225-
226225
### subscriptionId
227226

228227
Subscription ID for the Event Hubs namespace and Azure Data Lake Store. Both these resources must be under the same subscription ID.
@@ -251,7 +250,7 @@ The Azure Data Lake Store name for the captured events.
251250

252251
### dataLakeFolderPath
253252

254-
The destination folder path for the captured events. This is the folder in your Data Lake Store to which the events will be pushed during the capture operation. To set permissions on this folder, see [Use Azure Data Lake Store to capture data from Event Hubs](../data-lake-store/data-lake-store-archive-eventhub-capture.md).
253+
The destination folder path for the captured events. This path is the folder in your Data Lake Store to which the events are pushed during the capture operation. To set permissions on this folder, see [Use Azure Data Lake Store to capture data from Event Hubs](../data-lake-store/data-lake-store-archive-eventhub-capture.md).
255254

256255
```json
257256
"dataLakeFolderPath": {
@@ -264,7 +263,7 @@ The destination folder path for the captured events. This is the folder in your
264263

265264
## Azure Storage or Azure Data Lake Storage Gen 2 as destination
266265

267-
Creates a namespace of type **EventHub**, with one event hub, and also enables Capture to Azure Blob Storage or Azure Data Lake Storage Gen2.
266+
Creates a namespace of type `Microsoft.EventHub/Namespaces`, with one event hub, and also enables Capture to Azure Blob Storage or Azure Data Lake Storage Gen2.
268267

269268
```json
270269
"resources":[
@@ -324,57 +323,6 @@ Creates a namespace of type **EventHub**, with one event hub, and also enables C
324323
]
325324
```
326325

327-
## Azure Data Lake Storage Gen1 as destination
328-
329-
Creates a namespace of type **EventHub**, with one event hub, and also enables Capture to Azure Data Lake Storage Gen1. If you're using Gen2 of Data Lake Storage, see the previous section.
330-
331-
```json
332-
"resources": [
333-
{
334-
"apiVersion": "2017-04-01",
335-
"name": "[parameters('namespaceName')]",
336-
"type": "Microsoft.EventHub/Namespaces",
337-
"location": "[variables('location')]",
338-
"sku": {
339-
"name": "Standard",
340-
"tier": "Standard"
341-
},
342-
"resources": [
343-
{
344-
"apiVersion": "2017-04-01",
345-
"name": "[parameters('eventHubName')]",
346-
"type": "EventHubs",
347-
"dependsOn": [
348-
"[concat('Microsoft.EventHub/namespaces/', parameters('namespaceName'))]"
349-
],
350-
"properties": {
351-
"path": "[parameters('eventHubName')]",
352-
"captureDescription": {
353-
"enabled": "true",
354-
"skipEmptyArchives": false,
355-
"encoding": "[parameters('archiveEncodingFormat')]",
356-
"intervalInSeconds": "[parameters('captureTime')]",
357-
"sizeLimitInBytes": "[parameters('captureSize')]",
358-
"destination": {
359-
"name": "EventHubArchive.AzureDataLake",
360-
"properties": {
361-
"DataLakeSubscriptionId": "[parameters('subscriptionId')]",
362-
"DataLakeAccountName": "[parameters('dataLakeAccountName')]",
363-
"DataLakeFolderPath": "[parameters('dataLakeFolderPath')]",
364-
"ArchiveNameFormat": "[parameters('captureNameFormat')]"
365-
}
366-
}
367-
}
368-
}
369-
}
370-
]
371-
}
372-
]
373-
```
374-
375-
> [!NOTE]
376-
> You can enable or disable emitting empty files when no events occur during the Capture window by using the **skipEmptyArchives** property.
377-
378326
## Commands to run deployment
379327

380328
[!INCLUDE [app-service-deploy-commands](../../includes/app-service-deploy-commands.md)]

0 commit comments

Comments
 (0)