You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/data-explorer/ingest-json-formats.md
+44-44Lines changed: 44 additions & 44 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -19,13 +19,13 @@ This article shows you how to ingest json formatted data into an Azure Data Expl
19
19
20
20
## The json format
21
21
22
-
ADX supports two json file formats:
22
+
Azure Data Explorer supports two json file formats:
23
23
*`json`: Line separated json. Each line in the input data has exactly one json record.
24
24
*`multijson`: Multi-lined json. The parser ignores the line separators and reads a record from the previous position to the end of a valid json.
25
25
26
26
### Ingest and map json formatted data
27
27
28
-
Ingestion of json formatted data requires you to specify the *format*[ingestion property](/azure/kusto/management/data-ingestion/index#ingestion-properties). Ingestion of json data requires [mapping](/azure/kusto/management/mappings), which maps a json source entry to its target column. When ingesting data, use the pre-defined `jsonMappingReference` ingestion property or specify the `jsonMapping`ingestion property. This article will use the `jsonMappingReference` ingestion property which is pre-defined on the table used for ingestion. In the examples below, we'll start by ingesting json records as raw data to a single column table. Then we'll use the mapping to ingest each property to its mapped column.
28
+
Ingestion of json formatted data requires you to specify the *format*using [ingestion property](/azure/kusto/management/data-ingestion/index#ingestion-properties). Ingestion of json data requires [mapping](/azure/kusto/management/mappings), which maps a json source entry to its target column. When ingesting data, use the pre-defined `jsonMappingReference` ingestion property or specify the `jsonMapping`ingestion property. This article will use the `jsonMappingReference` ingestion property which is pre-defined on the table used for ingestion. In the examples below, we'll start by ingesting json records as raw data to a single column table. Then we'll use the mapping to ingest each property to its mapped column.
29
29
30
30
### Simple json example
31
31
@@ -63,7 +63,7 @@ Use Kusto query language to ingest data in a raw json format.
63
63
64
64
This query creates a table with a single `Event` column of a [dynamic](/azure/kusto/query/scalar-data-types/dynamic) data type.
@@ -73,13 +73,13 @@ Use Kusto query language to ingest data in a raw json format.
73
73
74
74
1. Ingest data into the `RawEvents` table.
75
75
76
-
> [!NOTE]
77
-
> This shows the `ingest` control commands executed directly to the engine endpoint. In production scenarios, ingestion is executed to the Data Management service using client libraries or data connections. Read [Ingest data using the Azure Data Explorer Python library](/azure/data-explorer/python-ingest-data) and [Ingest data using the Azure Data Explorer .NET Standard SDK](/azure/data-explorer/net-standard-ingest-data) for a walk-through regarding ingesting data with these client libraries.
78
-
79
76
```Kusto
80
77
.ingest into table RawEvents h'https://kustosamplefiles.blob.core.windows.net/jsonsamplefiles/simple.json?st=2018-08-31T22%3A02%3A25Z&se=2020-09-01T22%3A02%3A00Z&sp=r&sv=2018-03-28&sr=b&sig=LQIbomcKI8Ooz425hWtjeq6d61uEaq21UVX7YrM61N4%3D' with (format=json, jsonMappingReference=RawEventMapping)
81
78
```
82
79
80
+
> [!NOTE]
81
+
> This shows the `ingest` control commands executed directly to the engine endpoint. In production scenarios, ingestion is executed to the Data Management service using client libraries or data connections. Read [Ingest data using the Azure Data Explorer Python library](/azure/data-explorer/python-ingest-data) and [Ingest data using the Azure Data Explorer .NET Standard SDK](/azure/data-explorer/net-standard-ingest-data) for a walk-through regarding ingesting data with these client libraries.
82
+
83
83
# [C#](#tab/c-sharp)
84
84
85
85
Use C# to ingest data in raw json format.
@@ -222,7 +222,7 @@ In this example, you ingest json records data. Each json property is mapped to a
222
222
223
223
In this mapping, as defined by the table schema, the `timestamp` entries will be ingested to the column `Time` as `datetime` data types.
224
224
225
-
1. Ingest data into the table `Events`.
225
+
1. Ingest data into the `Events` table.
226
226
227
227
```Kusto
228
228
.ingest into table Events h'https://kustosamplefiles.blob.core.windows.net/jsonsamplefiles/simple.json?st=2018-08-31T22%3A02%3A25Z&se=2020-09-01T22%3A02%3A00Z&sp=r&sv=2018-03-28&sr=b&sig=LQIbomcKI8Ooz425hWtjeq6d61uEaq21UVX7YrM61N4%3D' with (format=json, jsonMappingReference=FlatEventMapping)
@@ -273,7 +273,7 @@ In this example, you ingest json records data. Each json property is mapped to a
273
273
274
274
In this mapping, as defined by the table schema, the `timestamp` entries will be ingested to the column `Time` as `datetime` data types.
275
275
276
-
1. Ingest data into the table `Events`.
276
+
1. Ingest data into the `Events` table.
277
277
278
278
```C#
279
279
var blobPath = "https://kustosamplefiles.blob.core.windows.net/jsonsamplefiles/simple.json?st=2018-08-31T22%3A02%3A25Z&se=2020-09-01T22%3A02%3A00Z&sp=r&sv=2018-03-28&sr=b&sig=LQIbomcKI8Ooz425hWtjeq6d61uEaq21UVX7YrM61N4%3D";
@@ -300,7 +300,7 @@ In this example, you ingest json records data. Each json property is mapped to a
@@ -329,41 +329,41 @@ In this example, you ingest multi-lined json records. Each json property is mapp
329
329
330
330
# [KQL](#tab/kusto-query-language)
331
331
332
-
1. Ingest data into the table `Events`.
332
+
Ingest data into the `Events` table.
333
333
334
-
```Kusto
335
-
.ingest into table Events h'https://kustosamplefiles.blob.core.windows.net/jsonsamplefiles/multilined.json?st=2018-08-31T22%3A02%3A25Z&se=2020-09-01T22%3A02%3A00Z&sp=r&sv=2018-03-28&sr=b&sig=LQIbomcKI8Ooz425hWtjeq6d61uEaq21UVX7YrM61N4%3D' with (format=multijson, jsonMappingReference=FlatEventMapping)
336
-
```
334
+
```Kusto
335
+
.ingest into table Events h'https://kustosamplefiles.blob.core.windows.net/jsonsamplefiles/multilined.json?st=2018-08-31T22%3A02%3A25Z&se=2020-09-01T22%3A02%3A00Z&sp=r&sv=2018-03-28&sr=b&sig=LQIbomcKI8Ooz425hWtjeq6d61uEaq21UVX7YrM61N4%3D' with (format=multijson, jsonMappingReference=FlatEventMapping)
336
+
```
337
337
338
338
# [C#](#tab/c-sharp)
339
339
340
-
1. Ingest data into the table `Events`.
340
+
Ingest data into the `Events` table.
341
341
342
-
```C#
343
-
var tableMapping = "FlatEventMapping";
344
-
var blobPath = "https://kustosamplefiles.blob.core.windows.net/jsonsamplefiles/multilined.json?st=2018-08-31T22%3A02%3A25Z&se=2020-09-01T22%3A02%3A00Z&sp=r&sv=2018-03-28&sr=b&sig=LQIbomcKI8Ooz425hWtjeq6d61uEaq21UVX7YrM61N4%3D";
345
-
var properties =
346
-
new KustoQueuedIngestionProperties(database, table)
.ingest into table Events h'https://kustosamplefiles.blob.core.windows.net/jsonsamplefiles/array.json?st=2018-08-31T22%3A02%3A25Z&se=2020-09-01T22%3A02%3A00Z&sp=r&sv=2018-03-28&sr=b&sig=LQIbomcKI8Ooz425hWtjeq6d61uEaq21UVX7YrM61N4%3D' with (format=multijson, jsonMappingReference=RawEventMapping)
430
430
```
431
431
432
-
1. Review data in table `Events`.
432
+
1. Review data in the `Events` table.
433
433
434
434
```Kusto
435
435
Events
@@ -471,7 +471,7 @@ Array data types are an ordered collection of values. Ingestion of a json array
471
471
kustoClient.ExecuteControlCommand(command);
472
472
```
473
473
474
-
1. Ingest data into the table `RawEvents`.
474
+
1. Ingest data into the `RawEvents` table.
475
475
476
476
```C#
477
477
var table = "RawEvents";
@@ -487,7 +487,7 @@ Array data types are an ordered collection of values. Ingestion of a json array
.ingest into table Events h'https://kustosamplefiles.blob.core.windows.net/jsonsamplefiles/dictionary.json?st=2018-08-31T22%3A02%3A25Z&se=2020-09-01T22%3A02%3A00Z&sp=r&sv=2018-03-28&sr=b&sig=LQIbomcKI8Ooz425hWtjeq6d61uEaq21UVX7YrM61N4%3D' with (format=multijson, jsonMappingReference=KeyValueEventMapping)
var blobPath = "https://kustosamplefiles.blob.core.windows.net/jsonsamplefiles/dictionary.json?st=2018-08-31T22%3A02%3A25Z&se=2020-09-01T22%3A02%3A00Z&sp=r&sv=2018-03-28&sr=b&sig=LQIbomcKI8Ooz425hWtjeq6d61uEaq21UVX7YrM61N4%3D";
0 commit comments