You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/data-explorer/ingest-json-formats.md
+43-43Lines changed: 43 additions & 43 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,6 @@
1
1
---
2
-
title: Ingest json formatted data into Azure Data Explorer
3
-
description: Learn about how to ingest json formatted data into Azure Data Explorer.
2
+
title: Ingest JSON formatted data into Azure Data Explorer
3
+
description: Learn about how to ingest JSON formatted data into Azure Data Explorer.
4
4
author: orspod
5
5
ms.author: orspodek
6
6
ms.reviewer: kerend
@@ -9,27 +9,27 @@ ms.topic: conceptual
9
9
ms.date: 01/23/2020
10
10
---
11
11
12
-
# Ingest json formatted sample data into Azure Data Explorer
12
+
# Ingest JSON formatted sample data into Azure Data Explorer
13
13
14
-
This article shows you how to ingest json formatted data into an Azure Data Explorer database. You'll start with simple examples of raw and mapped json, continue to multi-lined json, and then tackle more complex json schemas containing arrays and dictionaries.
14
+
This article shows you how to ingest JSON formatted data into an Azure Data Explorer database. You'll start with simple examples of raw and mapped JSON, continue to multi-lined JSON, and then tackle more complex JSON schemas containing arrays and dictionaries.
15
15
16
16
## Prerequisites
17
17
18
18
[A test cluster and database](create-cluster-database-portal.md)
19
19
20
-
## The json format
20
+
## The JSON format
21
21
22
-
Azure Data Explorer supports two json file formats:
23
-
*`json`: Line separated json. Each line in the input data has exactly one json record.
24
-
*`multijson`: Multi-lined json. The parser ignores the line separators and reads a record from the previous position to the end of a valid json.
22
+
Azure Data Explorer supports two JSON file formats:
23
+
*`json`: Line separated JSON. Each line in the input data has exactly one JSON record.
24
+
*`multijson`: Multi-lined JSON. The parser ignores the line separators and reads a record from the previous position to the end of a valid JSON.
25
25
26
-
### Ingest and map json formatted data
26
+
### Ingest and map JSON formatted data
27
27
28
-
Ingestion of json formatted data requires you to specify the *format* using [ingestion property](/azure/kusto/management/data-ingestion/index#ingestion-properties). Ingestion of json data requires [mapping](/azure/kusto/management/mappings), which maps a json source entry to its target column. When ingesting data, use the pre-defined `jsonMappingReference` ingestion property or specify the `jsonMapping`ingestion property. This article will use the `jsonMappingReference` ingestion property, which is pre-defined on the table used for ingestion. In the examples below, we'll start by ingesting json records as raw data to a single column table. Then we'll use the mapping to ingest each property to its mapped column.
28
+
Ingestion of JSON formatted data requires you to specify the *format* using [ingestion property](/azure/kusto/management/data-ingestion/index#ingestion-properties). Ingestion of JSON data requires [mapping](/azure/kusto/management/mappings), which maps a JSON source entry to its target column. When ingesting data, use the pre-defined `jsonMappingReference` ingestion property or specify the `jsonMapping`ingestion property. This article will use the `jsonMappingReference` ingestion property, which is pre-defined on the table used for ingestion. In the examples below, we'll start by ingesting JSON records as raw data to a single column table. Then we'll use the mapping to ingest each property to its mapped column.
29
29
30
-
### Simple json example
30
+
### Simple JSON example
31
31
32
-
The following example is a simple json, with a flat structure. The data has temperature and humidity information, collected by several devices. Each record is marked with an ID and timestamp.
32
+
The following example is a simple JSON, with a flat structure. The data has temperature and humidity information, collected by several devices. Each record is marked with an ID and timestamp.
33
33
34
34
```json
35
35
{
@@ -41,13 +41,13 @@ The following example is a simple json, with a flat structure. The data has temp
41
41
}
42
42
```
43
43
44
-
## Ingest raw json records
44
+
## Ingest raw JSON records
45
45
46
-
In this example, you ingest json records as raw data to a single column table. The data manipulation, using queries, and update policy is done after the data is ingested.
46
+
In this example, you ingest JSON records as raw data to a single column table. The data manipulation, using queries, and update policy is done after the data is ingested.
47
47
48
48
# [KQL](#tab/kusto-query-language)
49
49
50
-
Use Kusto query language to ingest data in a raw json format.
50
+
Use Kusto query language to ingest data in a raw JSON format.
51
51
52
52
1. Sign in to [https://dataexplorer.azure.com](https://dataexplorer.azure.com).
53
53
@@ -63,13 +63,13 @@ Use Kusto query language to ingest data in a raw json format.
63
63
64
64
This query creates a table with a single `Event` column of a [dynamic](/azure/kusto/query/scalar-data-types/dynamic) data type.
@@ -228,11 +228,11 @@ In this example, you ingest json records data. Each json property is mapped to a
228
228
.ingest into table Events h'https://kustosamplefiles.blob.core.windows.net/jsonsamplefiles/simple.json?st=2018-08-31T22%3A02%3A25Z&se=2020-09-01T22%3A02%3A00Z&sp=r&sv=2018-03-28&sr=b&sig=LQIbomcKI8Ooz425hWtjeq6d61uEaq21UVX7YrM61N4%3D' with (format=json, jsonMappingReference=FlatEventMapping)
229
229
```
230
230
231
-
The file 'simple.json' has a few line-separated json records. The format is `json`, and the mapping used in the ingest command is the `FlatEventMapping` you created.
231
+
The file 'simple.json' has a few line-separated JSON records. The format is `json`, and the mapping used in the ingest command is the `FlatEventMapping` you created.
232
232
233
233
# [C#](#tab/c-sharp)
234
234
235
-
1. Create a new table, with a similar schema to the json input data. We'll use this table for all the following examples and ingest commands.
235
+
1. Create a new table, with a similar schema to the JSON input data. We'll use this table for all the following examples and ingest commands.
236
236
237
237
```C#
238
238
var table = "Events";
@@ -251,7 +251,7 @@ In this example, you ingest json records data. Each json property is mapped to a
251
251
kustoClient.ExecuteControlCommand(command);
252
252
```
253
253
254
-
1. Create the json mapping.
254
+
1. Create the JSON mapping.
255
255
256
256
```C#
257
257
var tableMapping = "FlatEventMapping";
@@ -287,11 +287,11 @@ In this example, you ingest json records data. Each json property is mapped to a
The file 'simple.json' has a few line-separated json records. The format is `json`, and the mapping used in the ingest command is the `FlatEventMapping` you created.
290
+
The file 'simple.json' has a few line-separated JSON records. The format is `json`, and the mapping used in the ingest command is the `FlatEventMapping` you created.
291
291
292
292
# [Python](#tab/python)
293
293
294
-
1. Create a new table, with a similar schema to the json input data. We'll use this table for all the following examples and ingest commands.
294
+
1. Create a new table, with a similar schema to the JSON input data. We'll use this table for all the following examples and ingest commands.
295
295
296
296
```Python
297
297
TABLE = "RawEvents"
@@ -300,7 +300,7 @@ In this example, you ingest json records data. Each json property is mapped to a
The file 'simple.json' has a few line separated json records. The format is `json`, and the mapping used in the ingest command is the `FlatEventMapping` you created.
323
+
The file 'simple.json' has a few line separated JSON records. The format is `json`, and the mapping used in the ingest command is the `FlatEventMapping` you created.
324
324
---
325
325
326
-
## Ingest multi-lined json records
326
+
## Ingest multi-lined JSON records
327
327
328
-
In this example, you ingest multi-lined json records. Each json property is mapped to a single column in the table. The file 'multilined.json' has a few indented json records. The format `multijson` tells the engine to read records by the json structure.
328
+
In this example, you ingest multi-lined JSON records. Each JSON property is mapped to a single column in the table. The file 'multilined.json' has a few indented JSON records. The format `multijson` tells the engine to read records by the JSON structure.
329
329
330
330
# [KQL](#tab/kusto-query-language)
331
331
@@ -358,7 +358,7 @@ Ingest data into the `Events` table.
Array data types are an ordered collection of values. Ingestion of a json array is done by an [update policy](/azure/kusto/management/update-policy). The json is ingested as-is to an intermediate table. An update policy runs a pre-defined function on the `RawEvents` table, reingesting the results to the target table. We will ingest data with the following structure:
372
+
Array data types are an ordered collection of values. Ingestion of a JSON array is done by an [update policy](/azure/kusto/management/update-policy). The JSON is ingested as-is to an intermediate table. An update policy runs a pre-defined function on the `RawEvents` table, reingesting the results to the target table. We will ingest data with the following structure:
373
373
374
374
```json
375
375
{
@@ -536,9 +536,9 @@ Array data types are an ordered collection of values. Ingestion of a json array
536
536
537
537
---
538
538
539
-
## Ingest json records containing dictionaries
539
+
## Ingest JSON records containing dictionaries
540
540
541
-
Dictionary structured json contains key-value pairs. Json records undergo ingestion mapping using logical expression in the `JsonPath`. You can ingest data with the following structure:
541
+
Dictionary structured JSON contains key-value pairs. Json records undergo ingestion mapping using logical expression in the `JsonPath`. You can ingest data with the following structure:
0 commit comments