You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/event-hubs/get-started-capture-python-v2.md
+12-11Lines changed: 12 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -18,9 +18,9 @@ ms.author: spelluru
18
18
19
19
# Capture Event Hubs data in Azure Storage and read it by using Python
20
20
21
-
You can configure an event hub so that the data sent to an event hub is captured in an Azure storage account or Azure Data Lake Storage. This article shows you how to write Python code to send events to an event hub and read the captured data from Azure blob storage. For more information about this feature, see [Event Hubs Capture feature overview](event-hubs-capture-overview.md).
21
+
You can configure an event hub so that the data that's sent to an event hub is captured in an Azure storage account or Azure Data Lake Storage. This article shows you how to write Python code to send events to an event hub and read the captured data from Azure Blob storage. For more information about this feature, see [Event Hubs Capture feature overview](event-hubs-capture-overview.md).
22
22
23
-
This sample uses the [Azure Python SDK](https://azure.microsoft.com/develop/python/) to demonstrate the Capture feature. The *sender.py* app sends simulated environmental telemetry to event hubs in JSON format. The event hub is configured to use the Capture feature to write this data to Blob storage in batches. The *capturereader.py* app reads these blobs and creates an append file for each device. The app then writes the data into CSV files.
23
+
This quickstart uses the [Azure Python SDK](https://azure.microsoft.com/develop/python/) to demonstrate the Capture feature. The *sender.py* app sends simulated environmental telemetry to event hubs in JSON format. The event hub is configured to use the Capture feature to write this data to Blob storage in batches. The *capturereader.py* app reads these blobs and creates an append file for each device. The app then writes the data into CSV files.
24
24
25
25
> [!IMPORTANT]
26
26
> This quickstart uses version 5 of the Azure Event Hubs Python SDK. For a quickstart that uses version 1 of the Python SDK, see [Quickstart: Event Hubs Capture walkthrough - Python](event-hubs-capture-python.md). If you're using version 1 of the SDK, we recommend that you migrate your code to the latest version. For more information, see the [migration guide](https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/eventhub/azure-eventhub/migration_guide.md).
@@ -38,8 +38,9 @@ In this quickstart, you:
38
38
39
39
- Python 2.7, and 3.5 or later, with PIP installed and updated.
40
40
- An Azure subscription. If you don't have one, [create a free account](https://azure.microsoft.com/free/) before you begin.
41
-
-[Create an Event Hubs namespace and an event hub in the namespace](event-hubs-create.md). Record the name of the Event Hubs namespace, the name of the event hub, and the primary access key for the namespace. To get the access key, see [Get an Event Hubs connection string](event-hubs-get-connection-string.md#get-connection-string-from-the-portal). The default key name is *RootManageSharedAccessKey*. For this quickstart, you need only the primary key. You don't need the connection string.
42
-
- Do the following:
41
+
- An active Event Hubs namespace and event hub.
42
+
[Create an Event Hubs namespace and an event hub in the namespace](event-hubs-create.md). Record the name of the Event Hubs namespace, the name of the event hub, and the primary access key for the namespace. To get the access key, see [Get an Event Hubs connection string](event-hubs-get-connection-string.md#get-connection-string-from-the-portal). The default key name is *RootManageSharedAccessKey*. For this quickstart, you need only the primary key. You don't need the connection string.
43
+
- An Azure storage account, a blob container in the storage account, and a connection string to the storage account. If you don't have these items, do the following:
43
44
1.[Create an Azure storage account](../storage/common/storage-quickstart-create-account.md?tabs=azure-portal)
44
45
1.[Create a blob container in the storage account](../storage/blobs/storage-quickstart-blobs-portal.md#create-a-container)
45
46
1.[Get the connection string to the storage account](../storage/common/storage-configure-connection-string.md#view-and-copy-a-connection-string)
@@ -52,7 +53,7 @@ In this section, you create a Python script that sends 200 events (10 devices *
52
53
53
54
1. Open your favorite Python editor, such as [Visual Studio Code][Visual Studio Code].
54
55
2. Create a script called *sender.py*.
55
-
3. Paste the following code into sender.py. See the code comments for details.
56
+
3. Paste the following code into *sender.py*.
56
57
57
58
```python
58
59
import time
@@ -88,15 +89,15 @@ In this section, you create a Python script that sends 200 events (10 devices *
88
89
* Replace `EVENTHUBSNAMESPACECONNECTIONSTRING`with the connection string for your Event Hubs namespace.
89
90
* Replace `EVENTHUBNAME`with the name of your event hub.
90
91
5. Run the script to send events to the event hub.
91
-
6. In the Azure portal, you can verify that the event hub has received the messages. Switch to **Messages** view in the **Metrics** section. Refresh the page to update the chart. It might take a few seconds forit to show that the messages have been received.
92
+
6. In the Azure portal, you can verify that the event hub has received the messages. Switch to **Messages** view in the **Metrics** section. Refresh the page to update the chart. It might take a few seconds forthe page to display that the messages have been received.
92
93
93
94
[](./media/get-started-capture-python-v2/messages-portal.png#lightbox)
94
95
95
96
## Create a Python script to read your Capture files
96
-
In this example, the captured data is stored in Azure Blob storage. The script in this section reads the capture data files from your Azure storage account and generates CSV files for you to easily openand view the contents. You will see 10 files in the current working directory of the application. These files will contain the environmental readings for the 10 devices.
97
+
In this example, the captured data is stored in Azure Blob storage. The script in this section reads the captured data files from your Azure storage account and generates CSV files for you to easily openand view. You will see 10 files in the current working directory of the application. These files will contain the environmental readings for the 10 devices.
97
98
98
99
1. In your Python editor, create a script called *capturereader.py*. This script reads the captured files and creates a filefor each device to write the data only for that device.
99
-
2. Paste the following code into *capturereader.py*. See the code comments for details.
100
+
2. Paste the following code into *capturereader.py*.
100
101
101
102
```python
102
103
import os
@@ -132,7 +133,7 @@ In this example, the captured data is stored in Azure Blob storage. The script i
132
133
133
134
def startProcessing():
134
135
print('Processor started using path: '+ os.getcwd())
blob_list= container.list_blobs() # List all the blobs in the container.
138
139
for blob in blob_list:
@@ -153,7 +154,7 @@ In this example, the captured data is stored in Azure Blob storage. The script i
153
154
154
155
startProcessing()
155
156
```
156
-
3. Replace `AZURESTORAGECONNECTIONSTRING`with the connection string for your Azure storage account. The name of container you created in this quickstart is*capture*. If you used a different name for the container, replace *capture*with the name of the container in the storage account.
157
+
3. Replace `AZURESTORAGECONNECTIONSTRING`with the connection string for your Azure storage account. The name of the container you created in this quickstart is*capture*. If you used a different name for the container, replace *capture*with the name of the container in the storage account.
157
158
158
159
## Run the scripts
159
160
1. Open a command prompt that has Python in its path, and then run these commands to install Python prerequisite packages:
@@ -163,7 +164,7 @@ In this example, the captured data is stored in Azure Blob storage. The script i
163
164
pip install azure-eventhub
164
165
pip install avro-python3
165
166
```
166
-
2. Change your directory to wherever you saved *sender.py*and*capturereader.py*, and run this command:
167
+
2. Change your directory to the directory where you saved *sender.py*and*capturereader.py*, and run this command:
0 commit comments