Skip to content

Commit efda070

Browse files
committed
PR reviewer changes
1 parent cf1ac96 commit efda070

File tree

2 files changed

+44
-44
lines changed

2 files changed

+44
-44
lines changed

articles/data-explorer/ingest-json-formats.md

Lines changed: 43 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
2-
title: Ingest json formatted data into Azure Data Explorer
3-
description: Learn about how to ingest json formatted data into Azure Data Explorer.
2+
title: Ingest JSON formatted data into Azure Data Explorer
3+
description: Learn about how to ingest JSON formatted data into Azure Data Explorer.
44
author: orspod
55
ms.author: orspodek
66
ms.reviewer: kerend
@@ -9,27 +9,27 @@ ms.topic: conceptual
99
ms.date: 01/23/2020
1010
---
1111

12-
# Ingest json formatted sample data into Azure Data Explorer
12+
# Ingest JSON formatted sample data into Azure Data Explorer
1313

14-
This article shows you how to ingest json formatted data into an Azure Data Explorer database. You'll start with simple examples of raw and mapped json, continue to multi-lined json, and then tackle more complex json schemas containing arrays and dictionaries.
14+
This article shows you how to ingest JSON formatted data into an Azure Data Explorer database. You'll start with simple examples of raw and mapped JSON, continue to multi-lined JSON, and then tackle more complex JSON schemas containing arrays and dictionaries.
1515

1616
## Prerequisites
1717

1818
[A test cluster and database](create-cluster-database-portal.md)
1919

20-
## The json format
20+
## The JSON format
2121

22-
Azure Data Explorer supports two json file formats:
23-
* `json`: Line separated json. Each line in the input data has exactly one json record.
24-
* `multijson`: Multi-lined json. The parser ignores the line separators and reads a record from the previous position to the end of a valid json.
22+
Azure Data Explorer supports two JSON file formats:
23+
* `json`: Line separated JSON. Each line in the input data has exactly one JSON record.
24+
* `multijson`: Multi-lined JSON. The parser ignores the line separators and reads a record from the previous position to the end of a valid JSON.
2525

26-
### Ingest and map json formatted data
26+
### Ingest and map JSON formatted data
2727

28-
Ingestion of json formatted data requires you to specify the *format* using [ingestion property](/azure/kusto/management/data-ingestion/index#ingestion-properties). Ingestion of json data requires [mapping](/azure/kusto/management/mappings), which maps a json source entry to its target column. When ingesting data, use the pre-defined `jsonMappingReference` ingestion property or specify the `jsonMapping`ingestion property. This article will use the `jsonMappingReference` ingestion property, which is pre-defined on the table used for ingestion. In the examples below, we'll start by ingesting json records as raw data to a single column table. Then we'll use the mapping to ingest each property to its mapped column.
28+
Ingestion of JSON formatted data requires you to specify the *format* using [ingestion property](/azure/kusto/management/data-ingestion/index#ingestion-properties). Ingestion of JSON data requires [mapping](/azure/kusto/management/mappings), which maps a JSON source entry to its target column. When ingesting data, use the pre-defined `jsonMappingReference` ingestion property or specify the `jsonMapping`ingestion property. This article will use the `jsonMappingReference` ingestion property, which is pre-defined on the table used for ingestion. In the examples below, we'll start by ingesting JSON records as raw data to a single column table. Then we'll use the mapping to ingest each property to its mapped column.
2929

30-
### Simple json example
30+
### Simple JSON example
3131

32-
The following example is a simple json, with a flat structure. The data has temperature and humidity information, collected by several devices. Each record is marked with an ID and timestamp.
32+
The following example is a simple JSON, with a flat structure. The data has temperature and humidity information, collected by several devices. Each record is marked with an ID and timestamp.
3333

3434
```json
3535
{
@@ -41,13 +41,13 @@ The following example is a simple json, with a flat structure. The data has temp
4141
}
4242
```
4343

44-
## Ingest raw json records
44+
## Ingest raw JSON records
4545

46-
In this example, you ingest json records as raw data to a single column table. The data manipulation, using queries, and update policy is done after the data is ingested.
46+
In this example, you ingest JSON records as raw data to a single column table. The data manipulation, using queries, and update policy is done after the data is ingested.
4747

4848
# [KQL](#tab/kusto-query-language)
4949

50-
Use Kusto query language to ingest data in a raw json format.
50+
Use Kusto query language to ingest data in a raw JSON format.
5151

5252
1. Sign in to [https://dataexplorer.azure.com](https://dataexplorer.azure.com).
5353

@@ -63,13 +63,13 @@ Use Kusto query language to ingest data in a raw json format.
6363
6464
This query creates a table with a single `Event` column of a [dynamic](/azure/kusto/query/scalar-data-types/dynamic) data type.
6565
66-
1. Create the json mapping.
66+
1. Create the JSON mapping.
6767
6868
```Kusto
6969
.create table RawEvents ingestion json mapping 'RawEventMapping' '[{"column":"Event","path":"$"}]'
7070
```
7171
72-
This command creates a mapping, and maps the json root path `$` to the `Event` column.
72+
This command creates a mapping, and maps the JSON root path `$` to the `Event` column.
7373
7474
1. Ingest data into the `RawEvents` table.
7575
@@ -82,7 +82,7 @@ Use Kusto query language to ingest data in a raw json format.
8282
8383
# [C#](#tab/c-sharp)
8484
85-
Use C# to ingest data in raw json format.
85+
Use C# to ingest data in raw JSON format.
8686
8787
1. Create the `RawEvents` table.
8888
@@ -111,7 +111,7 @@ Use C# to ingest data in raw json format.
111111
kustoClient.ExecuteControlCommand(command);
112112
```
113113
114-
1. Create the json mapping.
114+
1. Create the JSON mapping.
115115
116116
```C#
117117
var tableMapping = "RawEventMapping";
@@ -126,7 +126,7 @@ Use C# to ingest data in raw json format.
126126
127127
kustoClient.ExecuteControlCommand(command);
128128
```
129-
This command creates a mapping, and maps the json root path `$` to the `Event` column.
129+
This command creates a mapping, and maps the JSON root path `$` to the `Event` column.
130130
131131
1. Ingest data into the `RawEvents` table.
132132
@@ -159,7 +159,7 @@ Use C# to ingest data in raw json format.
159159
160160
# [Python](#tab/python)
161161
162-
Use Python to ingest data in raw json format.
162+
Use Python to ingest data in raw JSON format.
163163
164164
1. Create the `RawEvents` table.
165165
@@ -174,7 +174,7 @@ Use Python to ingest data in raw json format.
174174
dataframe_from_result_table(RESPONSE.primary_results[0])
175175
```
176176
177-
1. Create the json mapping.
177+
1. Create the JSON mapping.
178178
179179
```Python
180180
MAPPING = "RawEventMapping"
@@ -202,19 +202,19 @@ Use Python to ingest data in raw json format.
202202
203203
---
204204
205-
## Ingest mapped json records
205+
## Ingest mapped JSON records
206206
207-
In this example, you ingest json records data. Each json property is mapped to a single column in the table.
207+
In this example, you ingest JSON records data. Each JSON property is mapped to a single column in the table.
208208
209209
# [KQL](#tab/kusto-query-language)
210210
211-
1. Create a new table, with a similar schema to the json input data. We'll use this table for all the following examples and ingest commands.
211+
1. Create a new table, with a similar schema to the JSON input data. We'll use this table for all the following examples and ingest commands.
212212
213213
```Kusto
214214
.create table Events (Time: datetime, Device: string, MessageId: string, Temperature: double, Humidity: double)
215215
```
216216
217-
1. Create the json mapping.
217+
1. Create the JSON mapping.
218218
219219
```Kusto
220220
.create table Events ingestion json mapping 'FlatEventMapping' '[{"column":"Time","path":"$.timestamp"},{"column":"Device","path":"$.deviceId"},{"column":"MessageId","path":"$.messageId"},{"column":"Temperature","path":"$.temperature"},{"column":"Humidity","path":"$.humidity"}]'
@@ -228,11 +228,11 @@ In this example, you ingest json records data. Each json property is mapped to a
228228
.ingest into table Events h'https://kustosamplefiles.blob.core.windows.net/jsonsamplefiles/simple.json?st=2018-08-31T22%3A02%3A25Z&se=2020-09-01T22%3A02%3A00Z&sp=r&sv=2018-03-28&sr=b&sig=LQIbomcKI8Ooz425hWtjeq6d61uEaq21UVX7YrM61N4%3D' with (format=json, jsonMappingReference=FlatEventMapping)
229229
```
230230
231-
The file 'simple.json' has a few line-separated json records. The format is `json`, and the mapping used in the ingest command is the `FlatEventMapping` you created.
231+
The file 'simple.json' has a few line-separated JSON records. The format is `json`, and the mapping used in the ingest command is the `FlatEventMapping` you created.
232232
233233
# [C#](#tab/c-sharp)
234234
235-
1. Create a new table, with a similar schema to the json input data. We'll use this table for all the following examples and ingest commands.
235+
1. Create a new table, with a similar schema to the JSON input data. We'll use this table for all the following examples and ingest commands.
236236
237237
```C#
238238
var table = "Events";
@@ -251,7 +251,7 @@ In this example, you ingest json records data. Each json property is mapped to a
251251
kustoClient.ExecuteControlCommand(command);
252252
```
253253
254-
1. Create the json mapping.
254+
1. Create the JSON mapping.
255255
256256
```C#
257257
var tableMapping = "FlatEventMapping";
@@ -287,11 +287,11 @@ In this example, you ingest json records data. Each json property is mapped to a
287287
ingestClient.IngestFromSingleBlob(blobPath, deleteSourceOnSuccess: false, ingestionProperties: properties);
288288
```
289289
290-
The file 'simple.json' has a few line-separated json records. The format is `json`, and the mapping used in the ingest command is the `FlatEventMapping` you created.
290+
The file 'simple.json' has a few line-separated JSON records. The format is `json`, and the mapping used in the ingest command is the `FlatEventMapping` you created.
291291
292292
# [Python](#tab/python)
293293
294-
1. Create a new table, with a similar schema to the json input data. We'll use this table for all the following examples and ingest commands.
294+
1. Create a new table, with a similar schema to the JSON input data. We'll use this table for all the following examples and ingest commands.
295295
296296
```Python
297297
TABLE = "RawEvents"
@@ -300,7 +300,7 @@ In this example, you ingest json records data. Each json property is mapped to a
300300
dataframe_from_result_table(RESPONSE.primary_results[0])
301301
```
302302
303-
1. Create the json mapping.
303+
1. Create the JSON mapping.
304304
305305
```Python
306306
MAPPING = "FlatEventMapping"
@@ -320,12 +320,12 @@ In this example, you ingest json records data. Each json property is mapped to a
320320
BLOB_DESCRIPTOR, ingestion_properties=INGESTION_PROPERTIES)
321321
```
322322
323-
The file 'simple.json' has a few line separated json records. The format is `json`, and the mapping used in the ingest command is the `FlatEventMapping` you created.
323+
The file 'simple.json' has a few line separated JSON records. The format is `json`, and the mapping used in the ingest command is the `FlatEventMapping` you created.
324324
---
325325
326-
## Ingest multi-lined json records
326+
## Ingest multi-lined JSON records
327327
328-
In this example, you ingest multi-lined json records. Each json property is mapped to a single column in the table. The file 'multilined.json' has a few indented json records. The format `multijson` tells the engine to read records by the json structure.
328+
In this example, you ingest multi-lined JSON records. Each JSON property is mapped to a single column in the table. The file 'multilined.json' has a few indented JSON records. The format `multijson` tells the engine to read records by the JSON structure.
329329
330330
# [KQL](#tab/kusto-query-language)
331331
@@ -358,7 +358,7 @@ Ingest data into the `Events` table.
358358

359359
```Python
360360
MAPPING = "FlatEventMapping"
361-
BLOB_PATH = 'https://kustosamplefiles.blob.core.windows.net/jsonsamplefiles/multilined.json?st=2018-08-31T22%3A02%3A25Z&se=2020-09-01T22%3A02%3A00Z&sp=r&sv=2018-03-28&sr=b&sig=LQIbomcKI8Ooz425hWtjeq6d61uEaq21UVX7YrM61N4%3D'
361+
BLOB_PATH = 'https://kustosamplefiles.blob.core.windows.net/jsonsamplefiles/multilined.JSON?st=2018-08-31T22%3A02%3A25Z&se=2020-09-01T22%3A02%3A00Z&sp=r&sv=2018-03-28&sr=b&sig=LQIbomcKI8Ooz425hWtjeq6d61uEaq21UVX7YrM61N4%3D'
362362
INGESTION_PROPERTIES = IngestionProperties(database=DATABASE, table=TABLE, dataFormat=DataFormat.multijson, mappingReference=MAPPING)
363363
BLOB_DESCRIPTOR = BlobDescriptor(BLOB_PATH, FILE_SIZE)
364364
INGESTION_CLIENT.ingest_from_blob(
@@ -367,9 +367,9 @@ INGESTION_CLIENT.ingest_from_blob(
367367

368368
---
369369

370-
## Ingest json records containing arrays
370+
## Ingest JSON records containing arrays
371371

372-
Array data types are an ordered collection of values. Ingestion of a json array is done by an [update policy](/azure/kusto/management/update-policy). The json is ingested as-is to an intermediate table. An update policy runs a pre-defined function on the `RawEvents` table, reingesting the results to the target table. We will ingest data with the following structure:
372+
Array data types are an ordered collection of values. Ingestion of a JSON array is done by an [update policy](/azure/kusto/management/update-policy). The JSON is ingested as-is to an intermediate table. An update policy runs a pre-defined function on the `RawEvents` table, reingesting the results to the target table. We will ingest data with the following structure:
373373

374374
```json
375375
{
@@ -536,9 +536,9 @@ Array data types are an ordered collection of values. Ingestion of a json array
536536
537537
---
538538
539-
## Ingest json records containing dictionaries
539+
## Ingest JSON records containing dictionaries
540540
541-
Dictionary structured json contains key-value pairs. Json records undergo ingestion mapping using logical expression in the `JsonPath`. You can ingest data with the following structure:
541+
Dictionary structured JSON contains key-value pairs. Json records undergo ingestion mapping using logical expression in the `JsonPath`. You can ingest data with the following structure:
542542
543543
```json
544544
{
@@ -570,7 +570,7 @@ Dictionary structured json contains key-value pairs. Json records undergo ingest
570570

571571
# [KQL](#tab/kusto-query-language)
572572

573-
1. Create a json mapping.
573+
1. Create a JSON mapping.
574574

575575
```Kusto
576576
.create table Events ingestion json mapping 'KeyValueEventMapping' '[{"column":"Time","path":"$.event[?(@.Key == 'timestamp')]"},{"column":"Device","path":"$.event[?(@.Key == 'deviceId')]"},{"column":"MessageId","path":"$.event[?(@.Key == 'messageId')]"},{"column":"Temperature","path":"$.event[?(@.Key == 'temperature')]"},{"column":"Humidity","path":"$.event[?(@.Key == 'humidity')]"}]'
@@ -584,7 +584,7 @@ Dictionary structured json contains key-value pairs. Json records undergo ingest
584584
585585
# [C#](#tab/c-sharp)
586586
587-
1. Create a json mapping.
587+
1. Create a JSON mapping.
588588
589589
```C#
590590
var tableName = "Events";
@@ -621,7 +621,7 @@ Dictionary structured json contains key-value pairs. Json records undergo ingest
621621
622622
# [Python](#tab/python)
623623
624-
1. Create a json mapping.
624+
1. Create a JSON mapping.
625625
626626
```Python
627627
MAPPING = "KeyValueEventMapping"

articles/data-explorer/toc.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@
2727
href: visualize-power-bi.md
2828
- name: Concepts
2929
items:
30-
- name: Data Ingestion
30+
- name: Data ingestion
3131
items:
3232
- name: Data ingestion overview
3333
displayName: pipelines, connectors, plugins, Python, .NET, Java, Node, REST

0 commit comments

Comments
 (0)