Skip to content

Commit af9e6ee

Browse files
committed
Fixes in json file
1 parent e33f038 commit af9e6ee

File tree

1 file changed

+44
-44
lines changed

1 file changed

+44
-44
lines changed

articles/data-explorer/ingest-json-formats.md

Lines changed: 44 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -19,13 +19,13 @@ This article shows you how to ingest json formatted data into an Azure Data Expl
1919

2020
## The json format
2121

22-
ADX supports two json file formats:
22+
Azure Data Explorer supports two json file formats:
2323
* `json`: Line separated json. Each line in the input data has exactly one json record.
2424
* `multijson`: Multi-lined json. The parser ignores the line separators and reads a record from the previous position to the end of a valid json.
2525

2626
### Ingest and map json formatted data
2727

28-
Ingestion of json formatted data requires you to specify the *format* [ingestion property](/azure/kusto/management/data-ingestion/index#ingestion-properties). Ingestion of json data requires [mapping](/azure/kusto/management/mappings), which maps a json source entry to its target column. When ingesting data, use the pre-defined `jsonMappingReference` ingestion property or specify the `jsonMapping`ingestion property. This article will use the `jsonMappingReference` ingestion property which is pre-defined on the table used for ingestion. In the examples below, we'll start by ingesting json records as raw data to a single column table. Then we'll use the mapping to ingest each property to its mapped column.
28+
Ingestion of json formatted data requires you to specify the *format* using [ingestion property](/azure/kusto/management/data-ingestion/index#ingestion-properties). Ingestion of json data requires [mapping](/azure/kusto/management/mappings), which maps a json source entry to its target column. When ingesting data, use the pre-defined `jsonMappingReference` ingestion property or specify the `jsonMapping`ingestion property. This article will use the `jsonMappingReference` ingestion property which is pre-defined on the table used for ingestion. In the examples below, we'll start by ingesting json records as raw data to a single column table. Then we'll use the mapping to ingest each property to its mapped column.
2929

3030
### Simple json example
3131

@@ -63,7 +63,7 @@ Use Kusto query language to ingest data in a raw json format.
6363
6464
This query creates a table with a single `Event` column of a [dynamic](/azure/kusto/query/scalar-data-types/dynamic) data type.
6565
66-
1. Create the json mapping
66+
1. Create the json mapping.
6767
6868
```Kusto
6969
.create table RawEvents ingestion json mapping 'RawEventMapping' '[{"column":"Event","path":"$"}]'
@@ -73,13 +73,13 @@ Use Kusto query language to ingest data in a raw json format.
7373
7474
1. Ingest data into the `RawEvents` table.
7575
76-
> [!NOTE]
77-
> This shows the `ingest` control commands executed directly to the engine endpoint. In production scenarios, ingestion is executed to the Data Management service using client libraries or data connections. Read [Ingest data using the Azure Data Explorer Python library](/azure/data-explorer/python-ingest-data) and [Ingest data using the Azure Data Explorer .NET Standard SDK](/azure/data-explorer/net-standard-ingest-data) for a walk-through regarding ingesting data with these client libraries.
78-
7976
```Kusto
8077
.ingest into table RawEvents h'https://kustosamplefiles.blob.core.windows.net/jsonsamplefiles/simple.json?st=2018-08-31T22%3A02%3A25Z&se=2020-09-01T22%3A02%3A00Z&sp=r&sv=2018-03-28&sr=b&sig=LQIbomcKI8Ooz425hWtjeq6d61uEaq21UVX7YrM61N4%3D' with (format=json, jsonMappingReference=RawEventMapping)
8178
```
8279
80+
> [!NOTE]
81+
> This shows the `ingest` control commands executed directly to the engine endpoint. In production scenarios, ingestion is executed to the Data Management service using client libraries or data connections. Read [Ingest data using the Azure Data Explorer Python library](/azure/data-explorer/python-ingest-data) and [Ingest data using the Azure Data Explorer .NET Standard SDK](/azure/data-explorer/net-standard-ingest-data) for a walk-through regarding ingesting data with these client libraries.
82+
8383
# [C#](#tab/c-sharp)
8484
8585
Use C# to ingest data in raw json format.
@@ -222,7 +222,7 @@ In this example, you ingest json records data. Each json property is mapped to a
222222
223223
In this mapping, as defined by the table schema, the `timestamp` entries will be ingested to the column `Time` as `datetime` data types.
224224
225-
1. Ingest data into the table `Events`.
225+
1. Ingest data into the `Events` table.
226226
227227
```Kusto
228228
.ingest into table Events h'https://kustosamplefiles.blob.core.windows.net/jsonsamplefiles/simple.json?st=2018-08-31T22%3A02%3A25Z&se=2020-09-01T22%3A02%3A00Z&sp=r&sv=2018-03-28&sr=b&sig=LQIbomcKI8Ooz425hWtjeq6d61uEaq21UVX7YrM61N4%3D' with (format=json, jsonMappingReference=FlatEventMapping)
@@ -273,7 +273,7 @@ In this example, you ingest json records data. Each json property is mapped to a
273273
274274
In this mapping, as defined by the table schema, the `timestamp` entries will be ingested to the column `Time` as `datetime` data types.
275275
276-
1. Ingest data into the table `Events`.
276+
1. Ingest data into the `Events` table.
277277
278278
```C#
279279
var blobPath = "https://kustosamplefiles.blob.core.windows.net/jsonsamplefiles/simple.json?st=2018-08-31T22%3A02%3A25Z&se=2020-09-01T22%3A02%3A00Z&sp=r&sv=2018-03-28&sr=b&sig=LQIbomcKI8Ooz425hWtjeq6d61uEaq21UVX7YrM61N4%3D";
@@ -300,7 +300,7 @@ In this example, you ingest json records data. Each json property is mapped to a
300300
dataframe_from_result_table(RESPONSE.primary_results[0])
301301
```
302302
303-
1. Create the json mapping
303+
1. Create the json mapping.
304304
305305
```Python
306306
MAPPING = "FlatEventMapping"
@@ -309,7 +309,7 @@ In this example, you ingest json records data. Each json property is mapped to a
309309
dataframe_from_result_table(RESPONSE.primary_results[0])
310310
```
311311
312-
1. Ingest data into the table `Events`.
312+
1. Ingest data into the `Events` table.
313313
314314
```Python
315315
BLOB_PATH = 'https://kustosamplefiles.blob.core.windows.net/jsonsamplefiles/simple.json?st=2018-08-31T22%3A02%3A25Z&se=2020-09-01T22%3A02%3A00Z&sp=r&sv=2018-03-28&sr=b&sig=LQIbomcKI8Ooz425hWtjeq6d61uEaq21UVX7YrM61N4%3D'
@@ -329,41 +329,41 @@ In this example, you ingest multi-lined json records. Each json property is mapp
329329
330330
# [KQL](#tab/kusto-query-language)
331331
332-
1. Ingest data into the table `Events`.
332+
Ingest data into the `Events` table.
333333
334-
```Kusto
335-
.ingest into table Events h'https://kustosamplefiles.blob.core.windows.net/jsonsamplefiles/multilined.json?st=2018-08-31T22%3A02%3A25Z&se=2020-09-01T22%3A02%3A00Z&sp=r&sv=2018-03-28&sr=b&sig=LQIbomcKI8Ooz425hWtjeq6d61uEaq21UVX7YrM61N4%3D' with (format=multijson, jsonMappingReference=FlatEventMapping)
336-
```
334+
```Kusto
335+
.ingest into table Events h'https://kustosamplefiles.blob.core.windows.net/jsonsamplefiles/multilined.json?st=2018-08-31T22%3A02%3A25Z&se=2020-09-01T22%3A02%3A00Z&sp=r&sv=2018-03-28&sr=b&sig=LQIbomcKI8Ooz425hWtjeq6d61uEaq21UVX7YrM61N4%3D' with (format=multijson, jsonMappingReference=FlatEventMapping)
336+
```
337337

338338
# [C#](#tab/c-sharp)
339339

340-
1. Ingest data into the table `Events`.
340+
Ingest data into the `Events` table.
341341

342-
```C#
343-
var tableMapping = "FlatEventMapping";
344-
var blobPath = "https://kustosamplefiles.blob.core.windows.net/jsonsamplefiles/multilined.json?st=2018-08-31T22%3A02%3A25Z&se=2020-09-01T22%3A02%3A00Z&sp=r&sv=2018-03-28&sr=b&sig=LQIbomcKI8Ooz425hWtjeq6d61uEaq21UVX7YrM61N4%3D";
345-
var properties =
346-
new KustoQueuedIngestionProperties(database, table)
347-
{
348-
Format = DataSourceFormat.multijson,
349-
IngestionMappingReference = tableMapping
350-
};
342+
```C#
343+
var tableMapping = "FlatEventMapping";
344+
var blobPath = "https://kustosamplefiles.blob.core.windows.net/jsonsamplefiles/multilined.json?st=2018-08-31T22%3A02%3A25Z&se=2020-09-01T22%3A02%3A00Z&sp=r&sv=2018-03-28&sr=b&sig=LQIbomcKI8Ooz425hWtjeq6d61uEaq21UVX7YrM61N4%3D";
345+
var properties =
346+
new KustoQueuedIngestionProperties(database, table)
347+
{
348+
Format = DataSourceFormat.multijson,
349+
IngestionMappingReference = tableMapping
350+
};
351351

352-
ingestClient.IngestFromSingleBlob(blobPath, deleteSourceOnSuccess: false, ingestionProperties: properties);
353-
```
352+
ingestClient.IngestFromSingleBlob(blobPath, deleteSourceOnSuccess: false, ingestionProperties: properties);
353+
```
354354

355355
# [Python](#tab/python)
356356

357-
1. Ingest data into the table `Events`.
357+
Ingest data into the `Events` table.
358358

359-
```Python
360-
MAPPING = "FlatEventMapping"
361-
BLOB_PATH = 'https://kustosamplefiles.blob.core.windows.net/jsonsamplefiles/multilined.json?st=2018-08-31T22%3A02%3A25Z&se=2020-09-01T22%3A02%3A00Z&sp=r&sv=2018-03-28&sr=b&sig=LQIbomcKI8Ooz425hWtjeq6d61uEaq21UVX7YrM61N4%3D'
362-
INGESTION_PROPERTIES = IngestionProperties(database=DATABASE, table=TABLE, dataFormat=DataFormat.multijson, mappingReference=MAPPING)
363-
BLOB_DESCRIPTOR = BlobDescriptor(BLOB_PATH, FILE_SIZE)
364-
INGESTION_CLIENT.ingest_from_blob(
365-
BLOB_DESCRIPTOR, ingestion_properties=INGESTION_PROPERTIES)
366-
```
359+
```Python
360+
MAPPING = "FlatEventMapping"
361+
BLOB_PATH = 'https://kustosamplefiles.blob.core.windows.net/jsonsamplefiles/multilined.json?st=2018-08-31T22%3A02%3A25Z&se=2020-09-01T22%3A02%3A00Z&sp=r&sv=2018-03-28&sr=b&sig=LQIbomcKI8Ooz425hWtjeq6d61uEaq21UVX7YrM61N4%3D'
362+
INGESTION_PROPERTIES = IngestionProperties(database=DATABASE, table=TABLE, dataFormat=DataFormat.multijson, mappingReference=MAPPING)
363+
BLOB_DESCRIPTOR = BlobDescriptor(BLOB_PATH, FILE_SIZE)
364+
INGESTION_CLIENT.ingest_from_blob(
365+
BLOB_DESCRIPTOR, ingestion_properties=INGESTION_PROPERTIES)
366+
```
367367

368368
---
369369

@@ -423,13 +423,13 @@ Array data types are an ordered collection of values. Ingestion of a json array
423423
.alter table Events policy update @'[{"Source": "RawEvents", "Query": "EventRecordsExpand()", "IsEnabled": "True"}]'
424424
```
425425
426-
1. Ingest data into the table `RawEvents`.
426+
1. Ingest data into the `RawEvents` table.
427427
428428
```Kusto
429429
.ingest into table Events h'https://kustosamplefiles.blob.core.windows.net/jsonsamplefiles/array.json?st=2018-08-31T22%3A02%3A25Z&se=2020-09-01T22%3A02%3A00Z&sp=r&sv=2018-03-28&sr=b&sig=LQIbomcKI8Ooz425hWtjeq6d61uEaq21UVX7YrM61N4%3D' with (format=multijson, jsonMappingReference=RawEventMapping)
430430
```
431431
432-
1. Review data in table `Events`.
432+
1. Review data in the `Events` table.
433433
434434
```Kusto
435435
Events
@@ -471,7 +471,7 @@ Array data types are an ordered collection of values. Ingestion of a json array
471471
kustoClient.ExecuteControlCommand(command);
472472
```
473473
474-
1. Ingest data into the table `RawEvents`.
474+
1. Ingest data into the `RawEvents` table.
475475
476476
```C#
477477
var table = "RawEvents";
@@ -487,7 +487,7 @@ Array data types are an ordered collection of values. Ingestion of a json array
487487
ingestClient.IngestFromSingleBlob(blobPath, deleteSourceOnSuccess: false, ingestionProperties: properties);
488488
```
489489
490-
1. Review data in table `Events`.
490+
1. Review data in the `Events` table.
491491
492492
# [Python](#tab/python)
493493
@@ -521,7 +521,7 @@ Array data types are an ordered collection of values. Ingestion of a json array
521521
dataframe_from_result_table(RESPONSE.primary_results[0])
522522
```
523523
524-
1. Ingest data into the table `RawEvents`.
524+
1. Ingest data into the `RawEvents` table.
525525
526526
```Python
527527
TABLE = "RawEvents"
@@ -533,7 +533,7 @@ Array data types are an ordered collection of values. Ingestion of a json array
533533
BLOB_DESCRIPTOR, ingestion_properties=INGESTION_PROPERTIES)
534534
```
535535
536-
1. Review data in table `Events`.
536+
1. Review data in the `Events` table.
537537
538538
---
539539
@@ -577,7 +577,7 @@ Dictionary structured json contains key-value pairs. Json records undergo ingest
577577
.create table Events ingestion json mapping 'KeyValueEventMapping' '[{"column":"Time","path":"$.event[?(@.Key == 'timestamp')]"},{"column":"Device","path":"$.event[?(@.Key == 'deviceId')]"},{"column":"MessageId","path":"$.event[?(@.Key == 'messageId')]"},{"column":"Temperature","path":"$.event[?(@.Key == 'temperature')]"},{"column":"Humidity","path":"$.event[?(@.Key == 'humidity')]"}]'
578578
```
579579
580-
1. Ingest data into the table `Events`.
580+
1. Ingest data into the `Events` table.
581581
582582
```Kusto
583583
.ingest into table Events h'https://kustosamplefiles.blob.core.windows.net/jsonsamplefiles/dictionary.json?st=2018-08-31T22%3A02%3A25Z&se=2020-09-01T22%3A02%3A00Z&sp=r&sv=2018-03-28&sr=b&sig=LQIbomcKI8Ooz425hWtjeq6d61uEaq21UVX7YrM61N4%3D' with (format=multijson, jsonMappingReference=KeyValueEventMapping)
@@ -606,7 +606,7 @@ Dictionary structured json contains key-value pairs. Json records undergo ingest
606606
kustoClient.ExecuteControlCommand(command);
607607
```
608608
609-
1. Ingest data into the table `Events`.
609+
1. Ingest data into the `Events` table.
610610
611611
```C#
612612
var blobPath = "https://kustosamplefiles.blob.core.windows.net/jsonsamplefiles/dictionary.json?st=2018-08-31T22%3A02%3A25Z&se=2020-09-01T22%3A02%3A00Z&sp=r&sv=2018-03-28&sr=b&sig=LQIbomcKI8Ooz425hWtjeq6d61uEaq21UVX7YrM61N4%3D";
@@ -631,7 +631,7 @@ Dictionary structured json contains key-value pairs. Json records undergo ingest
631631
dataframe_from_result_table(RESPONSE.primary_results[0])
632632
```
633633
634-
1. Ingest data into the table `Events`.
634+
1. Ingest data into the `Events` table.
635635
636636
```Python
637637
MAPPING = "KeyValueEventMapping"

0 commit comments

Comments
 (0)