You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
By default, Kusto will insert incoming ingestions into a table by inferring the mapped table columns, from the payload properties. However, this mapping can be customized by creating a [JSON ingestion mapping](https://docs.microsoft.com/en-us/azure/data-explorer/kusto/management/mappings#json-mapping). The plugin can be configured to use an ingestion mapping via the `ingestion_mapping_reference` configuration key.
| tenant_id|_Required if `managed_identity_client_id`is not set_ - The tenant/domain ID of the AAD registered application. ||
54
-
| client_id|_Required if `managed_identity_client_id`is not set_ - The client ID of the AAD registered application. ||
55
-
| client_secret|_Required if `managed_identity_client_id`is not set_ - The client secret of the AAD registered application ([App Secret](https://docs.microsoft.com/en-us/azure/active-directory/develop/howto-create-service-principal-portal#option-2-create-a-new-application-secret)). |
56
-
| managed_identity_client_id|_Required if `tenant_id`, `client_id`, and `client_secret`are not set_ - The managed identity ID to authenticate with. Set to `SYSTEM` for system-assigned managed identity, or set to the MI client ID (GUID) for user-assigned managed identity. ||
57
-
| ingestion_endpoint|_Required_ - The cluster's ingestion endpoint, usually in the form `https://ingest-cluster_name.region.kusto.windows.net|
58
-
| database_name|_Required_ - The database name. ||
59
-
| table_name|_Required_ - The table name. ||
60
-
| ingestion_mapping_reference |_Optional_ - The name of a [JSON ingestion mapping](https://docs.microsoft.com/en-us/azure/data-explorer/kusto/management/mappings#json-mapping) that will be used to map the ingested payload into the table columns. ||
61
-
| log_key| Key name of the log content.|`log`|
62
-
| include_tag_key| If enabled, a tag is appended to output. The key name is used `tag_key` property. |`On`|
63
-
| tag_key| The key name of tag. If `include_tag_key` is false, This property is ignored. |`tag`|
64
-
| include_time_key| If enabled, a timestamp is appended to output. The key name is used `time_key` property. |`On`|
65
-
| time_key| The key name of time. If `include_time_key` is false, This property is ignored.|`timestamp`|
66
-
| ingestion_endpoint_connect_timeout| The connection timeout of various Kusto endpoints in seconds.|`60`|
67
-
| compression_enabled| If enabled, sends compressed HTTP payload (gzip) to Kusto.|`true`|
68
-
| ingestion_resources_refresh_interval| The ingestion resources refresh interval of Kusto endpoint in seconds.
69
-
| workers | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. |`0`|
70
-
| buffering_enabled|_Optional_ - Enable buffering into disk before ingesting into Azure Kusto. |`Off`|
71
-
| buffer_dir|_Optional_ - When buffering is `On`, specifies the location of directory where the buffered data will be stored. |`/tmp/fluent-bit/azure-kusto/`|
72
-
| upload_timeout|_Optional_ - When buffering is `On`, specifies a timeout for uploads. Fluent Bit will start ingesting buffer files which have been created more than x minutes and haven't reached `upload_file_size` limit. |`30m`|
73
-
| upload_file_size|_Optional_ - When buffering is `On`, specifies the size of files to be uploaded in MBs. |`200MB`|
74
-
| azure_kusto_buffer_key|_Optional_ - When buffering is `On`, set the Azure Kusto buffer key which must be specified when using multiple instances of Azure Kusto output plugin and buffering is enabled. |`key`|
75
-
| store_dir_limit_size|_Optional_ - When buffering is `On`, set the max size of the buffer directory. |`8GB`|
76
-
| buffer_file_delete_early |_Optional_ - When buffering is `On`, whether to delete the buffered file early after successful blob creation. |`Off`|
77
-
| unify_tag |_Optional_ - This creates a single buffer file when the buffering mode is `On`. |`On`|
78
-
| blob_uri_length |_Optional_ - Set the length of generated blob URI before ingesting to Kusto. |`64`|
79
-
| scheduler_max_retries |_Optional_ - When buffering is `On`, set the maximum number of retries for ingestion using the scheduler. |`3`|
80
-
| delete_on_max_upload_error |_Optional_ - When buffering is `On`, whether to delete the buffer file on maximum upload errors. |`Off`|
81
-
| io_timeout |_Optional_ - Configure the HTTP IO timeout for uploads. |`60s`|
82
-
83
-
### Configuration File
84
-
85
-
Get started quickly with this configuration file:
45
+
## Optional - create an ingestion mapping
46
+
47
+
By default, Kusto will insert incoming ingestion data into a table by inferring the mapped table columns, from the payload properties. However, this mapping can be customized by creating a [JSON ingestion mapping](https://docs.microsoft.com/en-us/azure/data-explorer/kusto/management/mappings#json-mapping). The plugin can be configured to use an ingestion mapping using the `ingestion_mapping_reference` configuration key.
48
+
49
+
## Configuration parameters
50
+
51
+
| Key | Description | Default |
52
+
| --- | ----------- | ----------- |
53
+
|`tenant_id`| Required if `managed_identity_client_id`isn't set. The tenant/domain ID of the AAD registered application. |_none_|
54
+
|`client_id`| Required if `managed_identity_client_id`isn't set. The client ID of the AAD registered application. |_none_|
55
+
|`client_secret`| Required if `managed_identity_client_id`isn't set. The client secret of the AAD registered application ([App Secret](https://docs.microsoft.com/en-us/azure/active-directory/develop/howto-create-service-principal-portal#option-2-create-a-new-application-secret)).|_none_|
56
+
|`managed_identity_client_id`| Required if `tenant_id`, `client_id`, and `client_secret`aren't set. The managed identity ID to authenticate with. Set to `SYSTEM` for system-assigned managed identity, or set to the MI client ID (`GUID`) for user-assigned managed identity. |_none_|
57
+
|`ingestion_endpoint`|The cluster's ingestion endpoint, usually in the form `https://ingest-cluster_name.region.kusto.windows.net`. |_none_|
58
+
|`database_name`|The database name. |_none_|
59
+
|`table_name`|The table name. |_none_|
60
+
|`ingestion_mapping_reference`|Optional. The name of a [JSON ingestion mapping](https://docs.microsoft.com/en-us/azure/data-explorer/kusto/management/mappings#json-mapping) that will be used to map the ingested payload into the table columns. |_none_|
61
+
|`log_key`| Key name of the log content. |`log`|
62
+
|`include_tag_key`| If enabled, a tag is appended to output. The key name is used `tag_key` property. |`On`|
63
+
|`tag_key`| The key name of tag. If `include_tag_key` is false, This property is ignored. |`tag`|
64
+
|`include_time_key`| If enabled, a timestamp is appended to output. The key name is used `time_key` property. |`On`|
65
+
|`time_key`| The key name of time. If `include_time_key` is false, This property is ignored. |`timestamp`|
66
+
|`ingestion_endpoint_connect_timeout`| The connection timeout of various Kusto endpoints in seconds. |`60`|
67
+
|`compression_enabled`| If enabled, sends compressed HTTP payload (gzip) to Kusto. |`true`|
68
+
|`ingestion_resources_refresh_interval`| The ingestion resources refresh interval of Kusto endpoint in seconds.|_none_|
69
+
|`workers`| The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. |`0`|
70
+
|`buffering_enabled`| Optional. Enable buffering into disk before ingesting into Azure Kusto. |`Off`|
71
+
|`buffer_dir`| Optional. When buffering is `On`, specifies the location of directory where the buffered data will be stored. |`/tmp/fluent-bit/azure-kusto/`|
72
+
|`upload_timeout`| Optional. When buffering is `On`, specifies a timeout for uploads. Fluent Bit will start ingesting buffer files which have been created more than x minutes and haven't reached `upload_file_size` limit. |`30m`|
73
+
|`upload_file_size`| Optional. When buffering is `On`, specifies the size of files to be uploaded in MBs. |`200MB`|
74
+
|`azure_kusto_buffer_key`| Optional. When buffering is `On`, set the Azure Kusto buffer key which must be specified when using multiple instances of Azure Kusto output plugin and buffering is enabled. |`key`|
75
+
|`store_dir_limit_size`| Optional. When buffering is `On`, set the max size of the buffer directory. |`8GB`|
76
+
|`buffer_file_delete_early`|Optional. When buffering is `On`, whether to delete the buffered file early after successful blob creation. |`Off`|
77
+
|`unify_tag`|Optional. This creates a single buffer file when the buffering mode is `On`. |`On`|
78
+
|`blob_uri_length`|Optional. Set the length of generated blob URI before ingesting to Kusto. |`64`|
79
+
|`scheduler_max_retries`|Optional. When buffering is `On`, set the maximum number of retries for ingestion using the scheduler. |`3`|
80
+
|`delete_on_max_upload_error`|Optional. When buffering is `On`, whether to delete the buffer file on maximum upload errors. |`Off`|
81
+
|`io_timeout`|Optional. Configure the HTTP IO timeout for uploads. |`60s`|
82
+
83
+
### Configuration file
84
+
85
+
Get started with this configuration file:
86
86
87
87
{% tabs %}
88
88
{% tab title="fluent-bit.yaml" %}
@@ -91,14 +91,14 @@ Get started quickly with this configuration file:
0 commit comments