Skip to content

Commit 5ad8a6e

Browse files
committed
acrolinx, typos, improved
1 parent b32dae9 commit 5ad8a6e

File tree

1 file changed

+24
-21
lines changed

1 file changed

+24
-21
lines changed

articles/time-series-insights/time-series-insights-update-storage-ingress.md

Lines changed: 24 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -8,25 +8,25 @@ ms.workload: big-data
88
ms.service: time-series-insights
99
services: time-series-insights
1010
ms.topic: conceptual
11-
ms.date: 02/07/2020
11+
ms.date: 02/10/2020
1212
ms.custom: seodec18
1313
---
1414

1515
# Data storage and ingress in Azure Time Series Insights Preview
1616

17-
This article describes updates to data storage and ingress for Azure Time Series Insights Preview. It describes the underlying storage structure, file format, and Time Series ID property. It also discusses the underlying ingress process, best practices, and current preview limitations.
17+
This article describes updates to data storage and ingress for Azure Time Series Insights Preview. It describes the underlying storage structure, file format, and Time Series ID property. The underlying ingress process, best practices, and current preview limitations are also described.
1818

1919
## Data ingress
2020

2121
Your Azure Time Series Insights environment contains an *ingestion engine* to collect, process, and store time-series data.
2222

23-
There are some considerations to take into account to ensure all incoming data is processed, to achieve high ingress scale, and minimize ingestion latency (the time taken by Time Series Insights to read and process data from the event source) when [planning your environment](time-series-insights-update-plan.md).
23+
There are some considerations to be mindful of to ensure all incoming data is processed, to achieve high ingress scale, and minimize *ingestion latency* (the time taken by Time Series Insights to read and process data from the event source) when [planning your environment](time-series-insights-update-plan.md).
2424

2525
Time Series Insights Preview data ingress policies determine where data can be sourced from and what format the data should have.
2626

2727
### Ingress policies
2828

29-
Data ingress involves how data is sent to an Azure Time Series Insights Preview environment.
29+
*Data ingress* involves how data is sent to an Azure Time Series Insights Preview environment.
3030

3131
Key configuration, formatting, and best practices are summarized below.
3232

@@ -42,13 +42,13 @@ Azure Time Series Insights Preview supports a maximum of two event sources per i
4242
> [!IMPORTANT]
4343
> * You may experience high initial latency when attaching an event source to your Preview environment.
4444
> Event source latency depends on the number of events currently in your IoT Hub or Event Hub.
45-
> * High latency will subside after event source data is first ingested. Contact us by submitting a support ticket through the Azure portal if you experience continued high latency.
45+
> * High latency will subside after event source data is first ingested. Submit a support ticket through the Azure portal if you experience ongoing high latency.
4646
4747
#### Supported data format and types
4848

4949
Azure Time Series Insights supports UTF-8 encoded JSON sent from Azure IoT Hub or Azure Event Hubs.
5050

51-
Below is the list of supported data types.
51+
The supported data types are:
5252

5353
| Data type | Description |
5454
|---|---|
@@ -59,9 +59,9 @@ Below is the list of supported data types.
5959

6060
#### Objects and arrays
6161

62-
You can send complex types such as objects and arrays as part of your event payload, but your data will undergo a flattening process when stored.
62+
You may send complex types such as objects and arrays as part of your event payload, but your data will undergo a flattening process when stored.
6363

64-
Detailed information describing how to shape your JSON events, sending complex type, and nested object flattening is available in [How to shape JSON for ingress and query](./time-series-insights-update-how-to-shape-events.md).
64+
Detailed information describing how to shape your JSON events, send complex type, and nested object flattening is available in [How to shape JSON for ingress and query](./time-series-insights-update-how-to-shape-events.md) to assist with planning and optimization.
6565

6666
### Ingress best practices
6767

@@ -77,6 +77,9 @@ We recommend that you employ the following best practices:
7777

7878
Azure Time Series Insights Preview ingress limitations are described below.
7979

80+
> [!TIP]
81+
> Read [Plan your Preview environment](https://docs.microsoft.com/azure/time-series-insights/time-series-insights-update-plan#review-preview-limits) for a comprehensive list of all Preview limits.
82+
8083
#### Per environment limitations
8184

8285
In general, ingress rates are viewed as the factor of the number of devices that are in your organization, event emission frequency, and the size of each event:
@@ -87,23 +90,23 @@ By default, Time Series Insights preview can ingest incoming data at a rate of *
8790

8891
> [!TIP]
8992
> * Environment support for ingesting speeds up to 16 MBps can be provided by request.
90-
> * Contact us if you require higher throughput by submitting a support ticket in the Azure portal.
93+
> * Contact us if you require higher throughput by submitting a support ticket through Azure Portal.
9194
9295
* **Example 1:**
9396

9497
Contoso Shipping has 100,000 devices that emit an event three times per minute. The size of an event is 200 bytes. They’re using an Event Hub with four partitions as the Time Series Insights event source.
9598

96-
* The ingestion rate for their Time Series Insights environment would be: 100,000 devices * 200 bytes/event * (3/60 event/sec) = 1 MBps.
99+
* The ingestion rate for their Time Series Insights environment would be: **100,000 devices * 200 bytes/event * (3/60 event/sec) = 1 MBps**.
97100
* The ingestion rate per partition would be 0.25 MBps.
98101
* Contoso Shipping’s ingestion rate would be within the preview scale limitation.
99102

100103
* **Example 2:**
101104

102105
Contoso Fleet Analytics has 60,000 devices that emit an event every second. They are using an IoT Hub 24 partition count of 4 as the Time Series Insights event source. The size of an event is 200 bytes.
103106

104-
* The environment ingestion rate would be: 20,000 devices * 200 bytes/event * 1 event/sec = 4 MBps.
107+
* The environment ingestion rate would be: **20,000 devices * 200 bytes/event * 1 event/sec = 4 MBps**.
105108
* The per partition rate would be 1 MBps.
106-
* Contoso Fleet Analytics would need to submit a request to Time Series Insights via the Azure portal for a dedicated environment to achieve this scale.
109+
* Contoso Fleet Analytics can submit a request to Time Series Insights through Azure Portal to increase the ingestion rate for their environment.
107110

108111
#### Hub partitions and per partition limits
109112

@@ -122,20 +125,18 @@ Azure Time Series Insights Preview currently has a general **per partition limit
122125

123126
#### IoT Hub-specific considerations
124127

125-
When a device is created in IoT Hub, it is permanently assigned to a partition. In doing so, IoT Hub is able to guarantee event ordering (since the assignment never changes).
126-
127-
This has implications for Time Series Insights instances that are ingesting data sent from IoT Hub downstream.
128+
When a device is created in IoT Hub, it's permanently assigned to a partition. In doing so, IoT Hub is able to guarantee event ordering (since the assignment never changes).
128129

129-
When messages from multiple devices are forwarded to the hub using the same gateway device ID, they may arrive in the same partition at the same time potentially exceeding the per partition scale limits.
130+
A fixed partition assignment also impacts Time Series Insights instances that are ingesting data sent from IoT Hub downstream. When messages from multiple devices are forwarded to the hub using the same gateway device ID, they may arrive in the same partition at the same time potentially exceeding the per partition scale limits.
130131

131132
**Impact**:
132133

133-
* If a single partition experiences a sustained rate of ingestion over the Preview limit, there is the potential that the Time Series Insights reader will not ever catch up before the IoT Hub data retention period has been exceeded. This would cause a loss of data.
134+
* If a single partition experiences a sustained rate of ingestion over the Preview limit, it's possible that Time Series Insights will not sync all device telemetry before the IoT Hub data retention period has been exceeded. As a result, sent data can be lost if the ingestion limits are consistently exceeded.
134135

135-
We recommend the following:
136+
To mitigate that circumstance, we recommend the following best practices:
136137

137-
* Calculate your per environment and per partition ingestion rate before deploying your solution.
138-
* Ensure that your IoT Hub devices (and thus partitions) are load-balanced to the furthest extend possible.
138+
* Calculate your per environment and per partition ingestion rates before deploying your solution.
139+
* Ensure that your IoT Hub devices are load-balanced to the furthest extent possible.
139140

140141
> [!IMPORTANT]
141142
> For environments using IoT Hub as an event source, calculate the ingestion rate using the number of hub devices in use to be sure that the rate falls below the 0.5 MBps per partition limitation in preview.
@@ -185,7 +186,9 @@ Azure Time Series Insights Preview publishes up to two copies of each event in y
185186
> [!NOTE]
186187
> You can also use Spark, Hadoop, and other familiar tools to process the raw Parquet files.
187188
188-
Time Series Insights Preview also re-partitions the Parquet files to optimize for the Time Series Insights query. This repartitioned copy of the data is also saved. During public review, data is stored indefinitely in your Azure Storage account.
189+
Time Series Insights Preview also repartitions the Parquet files to optimize for the Time Series Insights query. This repartitioned copy of the data is also saved.
190+
191+
During public Preview, data is stored indefinitely in your Azure Storage account.
189192

190193
#### Writing and editing Time Series Insights blobs
191194

0 commit comments

Comments
 (0)