You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/stream-analytics/stream-analytics-parsing-json.md
+18-18Lines changed: 18 additions & 18 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,19 +1,19 @@
1
1
---
2
2
title: Parsing JSON and AVRO in Azure Stream Analytics
3
-
description: This article describes how to operate on complex data types like arrays, JSON, CSV formatted data.
3
+
description: This article describes how to operate on complex data types like arrays, JSON, CSV formatted data when using Azure Stream Analytics.
4
4
ms.service: azure-stream-analytics
5
5
author: an-emma
6
6
ms.author: raan
7
-
ms.topic: conceptual
8
-
ms.date: 05/25/2023
9
-
ms.custom:
7
+
ms.topic: concept-article
8
+
ms.date: 01/23/2025
9
+
# Customer intent: I want to know about Azure Stream Analytics support for parsing JSON and AVRO data.
10
10
---
11
11
# Parse JSON and Avro data in Azure Stream Analytics
12
12
13
-
Azure Stream Analytics support processing events in CSV, JSON, and Avro data formats. Both JSON and Avro data can be structured and contain some complex types such as nested objects (records) and arrays.
13
+
The Azure Stream Analytics service supports processing events in CSV, JSON, and Avro data formats. Both JSON and Avro data can be structured and contain some complex types such as nested objects (records) and arrays.
14
14
15
15
>[!NOTE]
16
-
>AVRO files created by Event Hub Capture use a specific format that requires you to use the *custom deserializer* feature. For more information, see [Read input in any format using .NET custom deserializers](./custom-deserializer-examples.md).
16
+
>AVRO files created by Event Hubs Capture use a specific format that requires you to use the *custom deserializer* feature. For more information, see [Read input in any format using .NET custom deserializers](./custom-deserializer-examples.md).
17
17
18
18
19
19
@@ -44,7 +44,7 @@ Record data types are used to represent JSON and Avro arrays when corresponding
44
44
```
45
45
46
46
### Access nested fields in known schema
47
-
Use dot notation (.) to easily access nested fields directly from your query. For example, this query selects the Latitude and Longitude coordinates under the Location property in the preceding JSON data. The dot notation can be used to navigate multiple levels as shown below.
47
+
Use dot notation (.) to easily access nested fields directly from your query. For example, this query selects the Latitude and Longitude coordinates under the Location property in the preceding JSON data. The dot notation can be used to navigate multiple levels as shown in the following snippet:
48
48
49
49
```SQL
50
50
SELECT
@@ -82,9 +82,9 @@ The result is:
82
82
83
83
### Access nested fields when property name is a variable
84
84
85
-
Use the [GetRecordPropertyValue](/stream-analytics-query/getrecordpropertyvalue-azure-stream-analytics) function if the property name is a variable. This allows for building dynamic queries without hardcoding property names.
85
+
Use the [GetRecordPropertyValue](/stream-analytics-query/getrecordpropertyvalue-azure-stream-analytics) function if the property name is a variable. It allows for building dynamic queries without hardcoding property names.
86
86
87
-
For example, imagine the sample data stream needs **to be joined with reference data** containing thresholds for each device sensor. A snippet of such reference data is shown below.
87
+
For example, imagine the sample data stream needs **to be joined with reference data** containing thresholds for each device sensor. A snippet of such reference data is shown in the following snippet.
88
88
89
89
```json
90
90
{
@@ -99,7 +99,7 @@ For example, imagine the sample data stream needs **to be joined with reference
99
99
}
100
100
```
101
101
102
-
The goal here is to join our sample dataset from the top of the article to that reference data, and output one event for each sensor measure above its threshold. That means our single event above can generate multiple output events if multiple sensors are above their respective thresholds, thanks to the join. To achieve similar results without a join, see the section below.
102
+
The goal here's to join our sample dataset from the top of the article to that reference data, and output one event for each sensor measure above its threshold. That means our single event above can generate multiple output events if multiple sensors are above their respective thresholds, thanks to the join. To achieve similar results without a join, see the following example:
103
103
104
104
```SQL
105
105
SELECT
@@ -119,8 +119,8 @@ WHERE
119
119
The result is:
120
120
121
121
|DeviceID|SensorName|AlertMessage|
122
-
|-|-|-|
123
-
|12345|Humidity|Alert : Sensor above threshold|
122
+
| - | - | - |
123
+
|12345|Humidity| Alert: Sensor above threshold|
124
124
125
125
### Convert record fields into separate events
126
126
@@ -165,7 +165,7 @@ SELECT DeviceID, PropertyValue AS Humidity INTO HumidityOutput FROM Stage0 WHERE
165
165
```
166
166
167
167
### Parse JSON record in SQL reference data
168
-
When using Azure SQL Database as reference data in your job, it's possible to have a column that has data in JSON format. An example is shown below.
168
+
When using Azure SQL Database as reference data in your job, it's possible to have a column that has data in JSON format. An example is shown in the following example:
169
169
170
170
|DeviceID|Data|
171
171
|-|-|
@@ -180,7 +180,7 @@ return JSON.parse(string);
180
180
}
181
181
```
182
182
183
-
You can then create a step in your Stream Analytics query as shown below to access the fields of your JSON records.
183
+
You can then create a step in your Stream Analytics query as shown here to access the fields of your JSON records.
184
184
185
185
```SQL
186
186
WITH parseJson as
@@ -198,9 +198,9 @@ You can then create a step in your Stream Analytics query as shown below to acce
198
198
199
199
## Array data types
200
200
201
-
Array data types are an ordered collection of values. Some typical operations on array values are detailed below. These examples use the functions [GetArrayElement](/stream-analytics-query/getarrayelement-azure-stream-analytics), [GetArrayElements](/stream-analytics-query/getarrayelements-azure-stream-analytics), [GetArrayLength](/stream-analytics-query/getarraylength-azure-stream-analytics), and the [APPLY](/stream-analytics-query/apply-azure-stream-analytics) operator.
201
+
Array data types are an ordered collection of values. Some typical operations on array values are detailed here. These examples use the functions [GetArrayElement](/stream-analytics-query/getarrayelement-azure-stream-analytics), [GetArrayElements](/stream-analytics-query/getarrayelements-azure-stream-analytics), [GetArrayLength](/stream-analytics-query/getarraylength-azure-stream-analytics), and the [APPLY](/stream-analytics-query/apply-azure-stream-analytics) operator.
202
202
203
-
Here's an example of a event. Both `CustomSensor03` and `SensorMetadata` are of type **array**:
203
+
Here's an example of an event. Both `CustomSensor03` and `SensorMetadata` are of type **array**:
204
204
205
205
```json
206
206
{
@@ -294,7 +294,7 @@ The result is:
294
294
|12345|Manufacturer|ABC|
295
295
|12345|Version|1.2.45|
296
296
297
-
If the extracted fields need to appear in columns, it is possible to pivot the dataset using the [WITH](/stream-analytics-query/with-azure-stream-analytics) syntax in addition to the [JOIN](/stream-analytics-query/join-azure-stream-analytics) operation. That join requires a [time boundary](/stream-analytics-query/join-azure-stream-analytics#BKMK_DateDiff) condition that prevents duplication:
297
+
If the extracted fields need to appear in columns, it's possible to pivot the dataset using the [WITH](/stream-analytics-query/with-azure-stream-analytics) syntax in addition to the [JOIN](/stream-analytics-query/join-azure-stream-analytics) operation. That join requires a [time boundary](/stream-analytics-query/join-azure-stream-analytics#BKMK_DateDiff) condition that prevents duplication:
298
298
299
299
```SQL
300
300
WITH DynamicCTE AS (
@@ -322,5 +322,5 @@ The result is:
322
322
|-|-|-|-|-|
323
323
|12345|47|122|1.2.45|ABC|
324
324
325
-
## See Also
325
+
## Related content
326
326
[Data Types in Azure Stream Analytics](/stream-analytics-query/data-types-azure-stream-analytics)
0 commit comments