Skip to content

Commit d868c5a

Browse files
authored
Merge pull request #95285 from viv-liu/viviali-export-cleanup
Cleaning up export doc
2 parents 596a9c6 + b418176 commit d868c5a

File tree

5 files changed

+44
-49
lines changed

5 files changed

+44
-49
lines changed

articles/iot-central-pnp/toc.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -83,7 +83,7 @@
8383
href: ../iot-central/core/howto-version-device-template-pnp.md?toc=/azure/iot-central-pnp/toc.json&bc=/azure/iot-central-pnp/breadcrumb/toc.json
8484
- name: Transform
8585
items:
86-
- name: Configure continuous data export
86+
- name: Export your data
8787
href: ../iot-central/core/howto-export-data-pnp.md?toc=/azure/iot-central-pnp/toc.json&bc=/azure/iot-central-pnp/breadcrumb/toc.json
8888
- name: Create webhooks on rules
8989
href: ../iot-central/core/howto-create-webhooks-pnp.md?toc=/azure/iot-central-pnp/toc.json&bc=/azure/iot-central-pnp/breadcrumb/toc.json
@@ -147,7 +147,7 @@
147147
href: /azure/iot-hub/iot-hub-devguide-sdks#azure-iot-device-sdks
148148
- name: Resources
149149
items:
150-
- name: REST API
150+
- name: REST API Reference
151151
href: https://docs.microsoft.com/rest/api/iotcentral/
152152
- name: Customer data requests
153153
href: ../iot-central/core/iot-central-customer-data-requests.md?toc=/azure/iot-central-pnp/toc.json&bc=/azure/iot-central-pnp/breadcrumb/toc.json

articles/iot-central/core/concepts-architecture-pnp.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,7 @@ An application can have one or more simulated and real devices based on each dev
7777

7878
## Data export
7979

80-
In an Azure IoT Central application, you can [continuously export your data](howto-export-data-pnp.md?toc=/azure/iot-central-pnp/toc.json&bc=/azure/iot-central-pnp/breadcrumb/toc.json) to your own Azure Event Hubs and Azure Service Bus instances. You can also periodically export your data to your Azure Blob storage account. IoT Central can export measurements, devices, and device templates.
80+
In an Azure IoT Central application, you can [continuously export your data](howto-export-data-pnp.md?toc=/azure/iot-central-pnp/toc.json&bc=/azure/iot-central-pnp/breadcrumb/toc.json) to your own Azure Event Hubs, Azure Service Bus, and Azure Blob Storage instances. IoT Central can export telemetry, devices, and device templates.
8181

8282
## Batch device updates
8383

articles/iot-central/core/concepts-architecture.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -78,7 +78,7 @@ An application can have one or more simulated and real devices based on each dev
7878

7979
## Data export
8080

81-
In an Azure IoT Central application, you can [continuously export your data](howto-export-data-event-hubs-service-bus.md) to your own Azure Event Hubs, and Azure Service Bus instances. You can also periodically export your data to your Azure Blob storage account. IoT Central can export measurements, devices, and device templates.
81+
In an Azure IoT Central application, you can [continuously export your data](howto-export-data-event-hubs-service-bus.md) to your own Azure Event Hubs, Azure Service Bus, and Azure Blob Storage instances. IoT Central can export measurements, devices, and device templates.
8282

8383
## Batch device updates
8484

articles/iot-central/core/howto-export-data-pnp.md

Lines changed: 39 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -7,16 +7,16 @@ ms.author: viviali
77
ms.date: 10/15/2019
88
ms.topic: conceptual
99
ms.service: iot-central
10-
manager: peterpr
10+
manager: corywink
1111
---
1212

13-
# Export your Azure IoT Central data(preview features)
13+
# Export your Azure IoT Central data (preview features)
1414

1515
[!INCLUDE [iot-central-pnp-original](../../../includes/iot-central-pnp-original-note.md)]
1616

1717
*This topic applies to administrators.*
1818

19-
This article describes how to use the continuous data export feature in Azure IoT Central to export your data to Azure Event Hubs, Azure Service Bus, or Azure Blob storage instances. Data is exported in JSON format and can include telemetry, device information, and device template information. Use the exported data for:
19+
This article describes how to use the continuous data export feature in Azure IoT Central to export your data to **Azure Event Hubs**, **Azure Service Bus**, or **Azure Blob storage** instances. Data is exported in JSON format and can include telemetry, device information, and device template information. Use the exported data for:
2020

2121
- Warm-path insights and analytics. This option includes triggering custom rules in Azure Stream Analytics, triggering custom workflows in Azure Logic Apps, or passing it through Azure Functions to be transformed.
2222
- Cold-path analytics such as training models in Azure Machine Learning or long-term trend analysis in Microsoft Power BI.
@@ -103,30 +103,33 @@ Now that you have a destination to export data to, follow these steps to set up
103103

104104
9. After a few minutes, your data appears in your chosen destination.
105105

106-
## Data format
106+
## Export contents and format
107107

108-
Data is exported to your event hub or Service Bus queue or topic in near-realtime.
108+
Exported telemetry data contains the entirety of the message your devices sent to IoT Central, not just the telemetry values themselves. Exported devices data contains changes to properties and metadata of all devices, and exported device templates contains changes to all device templates.
109109

110-
Data is exported to your storage account once per minute, with each file containing the batch of changes since the last exported file. Exported data is placed in three folders in JSON format. The default paths in your storage account are:
110+
For Event Hubs and Service Bus, data is exported in near-realtime. The data sits in the body property and is in JSON format (see below for examples).
111+
112+
For Blob Storage, data is exported once per minute, with each file containing the batch of changes since the last exported file. Exported data is placed in three folders in JSON format. The default paths in your storage account are:
111113

112114
- Telemetry: _{container}/{app-id}/telemetry/{YYYY}/{MM}/{dd}/{hh}/{mm}/{filename}_
113115
- Devices: _{container}/{app-id}/devices/{YYYY}/{MM}/{dd}/{hh}/{mm}/{filename}_
114116
- Device templates: _{container}/{app-id}/deviceTemplates/{YYYY}/{MM}/{dd}/{hh}/{mm}/{filename}_
115117

116118
You can browse the exported files in the Azure portal by navigating to the file and choosing the **Edit blob** tab.
117119

118-
Exported telemetry data contains the entirety of the message your devices sent to IoT Central, not just the telemetry values themselves. Exported devices data contains changes to properties and metadata of all devices, and exported device templates contains changes to all device templates. The exported data is in the body property and is in JSON format.
119120

120-
### Telemetry
121+
## Telemetry
122+
123+
For Event Hubs and Service Bus, a new message is exported quickly after IoT Central receives the message from a device, and each exported message contains the full message the device sent in the body property in JSON format.
121124

122-
A new message is exported quickly after IoT Central receives the message from a device.
125+
For Blob Storage, messages are batched and exported once per minute. The exported files use the same format as the message files exported by [IoT Hub message routing](../../iot-hub/iot-hub-csharp-csharp-process-d2c.md) to blob storage.
123126

124-
- Each exported message in Event Hubs and Service Bus contains the full message the device sent in the body property in JSON format.
125-
- The exported files in blob storage use the same format as the message files exported by [IoT Hub message routing](../../iot-hub/iot-hub-csharp-csharp-process-d2c.md) to blob storage. Ensure that your devices are sending messages that have `contentType: application/JSON` and `contentEncoding:utf-8` (or `utf-16`, `utf-32`). See the [IoT Hub documentation](../../iot-hub/iot-hub-devguide-routing-query-syntax.md#message-routing-query-based-on-message-body) for an example.
127+
> [!NOTE]
128+
> For Blob Storage, ensure that your devices are sending messages that have `contentType: application/JSON` and `contentEncoding:utf-8` (or `utf-16`, `utf-32`). See the [IoT Hub documentation](../../iot-hub/iot-hub-devguide-routing-query-syntax.md#message-routing-query-based-on-message-body) for an example.
126129
127-
The devices that send the telemetry are represented by device IDs (see the following sections). To get the names of the devices, export device data and correlate each message by using the **connectionDeviceId** that matches the **deviceId** of the device message.
130+
The device that sent the telemetry is represented by the device ID (see the following sections). To get the names of the devices, export device data and correlate each message by using the **connectionDeviceId** that matches the **deviceId** of the device message.
128131

129-
The following example shows a message about telemetry data received in event hub or Service Bus queue or topic.
132+
This is an example message received in an event hub or Service Bus queue or topic.
130133

131134
```json
132135
{
@@ -155,7 +158,7 @@ The following example shows a message about telemetry data received in event hub
155158
}
156159
```
157160

158-
The following example shows a record in JSON format in blob storage:
161+
This is an example record exported to blob storage:
159162

160163
```json
161164
{
@@ -179,26 +182,25 @@ The following example shows a record in JSON format in blob storage:
179182
}
180183
```
181184

182-
### Devices
183-
184-
Messages containing device data are sent to your event hub or Service Bus queue or topic once every few minutes. A new snapshot is written to blob storage once per minute. Each message or snapshot includes data about:
185+
## Devices
185186

186-
- New devices that were added
187-
- Devices with changed property values
188-
189-
Each message or record in a snapshot represents one or more changes to a device since the last exported message. Information includes:
187+
Each message or record in a snapshot represents one or more changes to a device and its properties since the last exported message. This includes:
190188

191189
- `@id` of the device in IoT Central
192190
- `name` of the device
193-
- `deviceId` from [Device Provisioning Service](/azure/iot-central/core/howto-connect-nodejs)
194-
- Device template information
191+
- `deviceId` which matches the `connectionDeviceId` from the telemetry stream above
192+
- `instanceOf` which is the device template Id
195193
- Property values
196194

197-
Devices deleted since the last batch aren't exported. Currently, there are no indicators in exported messages for deleted devices.
195+
The device template that each device belongs to is represented by the `instanceOf`. To get the name and additional information about the device template, be sure to export device template data too.
196+
197+
Deleted devices aren't exported. Currently, there are no indicators in exported messages for deleted devices.
198+
199+
For Event Hubs and Service Bus, messages containing device data are sent to your event hub or Service Bus queue or topic in near real-time, as it appears in IoT Central.
198200

199-
The device template that each device belongs to is represented by a device template ID. To get the name of the device template, be sure to export device template data too.
201+
For Blob Storage, a new snapshot containing all the changes since the last one written is exported once per minute.
200202

201-
The following example shows a message about device data in event hub or Service Bus queue or topic:
203+
This is an example message about devices and properties data in event hub or Service Bus queue or topic:
202204

203205
```json
204206
{
@@ -257,7 +259,7 @@ The following example shows a message about device data in event hub or Service
257259
}
258260
```
259261

260-
Exported files contain a single line per record. The following example shows a record in JSON format.
262+
This is an example snapshot containing devices and properties data in Blob Storage. Exported files contain a single line per record.
261263

262264
```json
263265
{
@@ -303,27 +305,24 @@ Exported files contain a single line per record. The following example shows a r
303305
}
304306
```
305307

306-
### Device templates
307-
308-
Messages containing device templates data are sent to your event hub or Service Bus queue or topic once every few minutes. A new snapshot is written once per minute to blob storage. Therefore, every few minutes, a batch of messages arrives with data about:
309-
310-
Messages containing device template data are sent to your event hub or Service Bus queue or topic once every few minutes. A new snapshot is written to blob storage once per minute. Each message or snapshot includes data about:
311-
312-
- New device templates that were added or versioned
313-
- Device templates with changed capability models, cloud properties, overrides, and initial values
308+
## Device templates
314309

315310
Each message or snapshot record represents one or more changes to a device template since the last exported message. Information sent in each message or record includes:
316311

317-
- `@id` of the device template
312+
- `@id` of the device template which matches the `instanceOf` of the devices stream above
318313
- `name` of the device template
319314
- `version` of the device template
320315
- The device `capabilityModel` including its `interfaces`, and the telemetry, properties, and commands definitions
321316
- `cloudProperties` definitions
322317
- Overrides and initial values, inline with the `capabilityModel`
323318

324-
Device templates deleted since the last batch aren't exported. Currently, there are no indicators in exported messages for deleted device templates.
319+
Deleted device templates aren't exported. Currently, there are no indicators in exported messages for deleted device templates.
320+
321+
For Event Hubs and Service Bus, messages containing device template data are sent to your event hub or Service Bus queue or topic in near real-time, as it appears in IoT Central.
322+
323+
For Blob Storage, a new snapshot containing all the changes since the last one written is exported once per minute.
325324

326-
The following example shows a device template message in event hub or Service Bus queue or topic:
325+
This is an example message about device templates data in event hub or Service Bus queue or topic:
327326

328327
```json
329328
{
@@ -470,7 +469,7 @@ The following example shows a device template message in event hub or Service Bu
470469
}
471470
```
472471

473-
Exported files contain a single line per record. The following example shows a record in JSON format.
472+
This is an example snapshot containing devices and properties data in Blob Storage. Exported files contain a single line per record.
474473

475474
```json
476475
{
@@ -606,7 +605,7 @@ Exported files contain a single line per record. The following example shows a r
606605

607606
## Next steps
608607

609-
Now that you know how to export your data to Azure Event Hubs and Azure Service Bus, continue to the next step:
608+
Now that you know how to export your data to Azure Event Hubs, Azure Service Bus, and Azure Blob Storage, continue to the next step:
610609

611610
> [!div class="nextstepaction"]
612611
> [How to trigger Azure Functions](howto-trigger-azure-functions.md?toc=/azure/iot-central-pnp/toc.json&bc=/azure/iot-central-pnp/breadcrumb/toc.json)

articles/iot-central/core/tutorial-create-telemetry-rules-pnp.md

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -104,10 +104,6 @@ Choose the rule you want to enable or disable. Add a filter in the **Scopes** se
104104

105105
## Next steps
106106

107-
Now that you have learned how to create rules in your Azure IoT Central application, the suggested next step is to learn [How to manage your devices](howto-manage-devices-pnp.md?toc=/azure/iot-central-pnp/toc.json&bc=/azure/iot-central-pnp/breadcrumb/toc.json)
108-
109-
## Next steps
110-
111107
In this tutorial, you learned how to:
112108

113109
* Create a telemetry-based rule
@@ -116,4 +112,4 @@ In this tutorial, you learned how to:
116112
Now that you've defined a threshold-based rule the suggested next step is to learn how to:
117113

118114
> [!div class="nextstepaction"]
119-
> [Configure continuous data export](./howto-export-data-pnp.md?toc=/azure/iot-central-pnp/toc.json&bc=/azure/iot-central-pnp/breadcrumb/toc.json).
115+
> [Create a webhook action](./howto-create-webhooks-pnp.md?toc=/azure/iot-central-pnp/toc.json&bc=/azure/iot-central-pnp/breadcrumb/toc.json).

0 commit comments

Comments
 (0)