Skip to content

Commit 2f4efde

Browse files
committed
Adding YAML exampels and sandardizing shell usage for Azure Data Explorer. Part of issue #1897.
Signed-off-by: Eric D. Schabell <[email protected]>
1 parent a3100aa commit 2f4efde

File tree

1 file changed

+53
-4
lines changed

1 file changed

+53
-4
lines changed

pipeline/outputs/azure_kusto.md

Lines changed: 53 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -38,13 +38,13 @@ Fluent-Bit ingests the event data into Kusto in a JSON format, that by default w
3838

3939
A table with the expected schema must exist in order for data to be ingested properly.
4040

41-
```kql
41+
```sql
4242
.create table FluentBit (log:dynamic, tag:string, timestamp:datetime)
4343
```
4444

4545
## Optional - Creating an Ingestion Mapping
4646

47-
By default, Kusto will insert incoming ingestions into a table by inferring the mapped table columns, from the payload properties. However, this mapping can be customized by creatng a [JSON ingestion mapping](https://docs.microsoft.com/en-us/azure/data-explorer/kusto/management/mappings#json-mapping). The plugin can be configured to use an ingestion mapping via the `ingestion_mapping_reference` configuration key.
47+
By default, Kusto will insert incoming ingestion's into a table by inferring the mapped table columns, from the payload properties. However, this mapping can be customized by creating a [JSON ingestion mapping](https://docs.microsoft.com/en-us/azure/data-explorer/kusto/management/mappings#json-mapping). The plugin can be configured to use an ingestion mapping via the `ingestion_mapping_reference` configuration key.
4848

4949
## Configuration Parameters
5050

@@ -84,10 +84,55 @@ By default, Kusto will insert incoming ingestions into a table by inferring the
8484

8585
Get started quickly with this configuration file:
8686

87+
{% tabs %}
88+
{% tab title="fluent-bit.yaml" %}
89+
90+
```yaml
91+
service:
92+
flush: 1
93+
log_level: info
94+
95+
pipeline:
96+
inputs:
97+
- name: dummy
98+
dummy: '{"name": "Fluent Bit", "year": 2020}'
99+
samples: 1
100+
tag: var.log.containers.app-default-96cbdef2340.log
101+
102+
outputs:
103+
- name: azure_kusto
104+
match: '*'
105+
tenant_id: <app_tenant_id>
106+
client_id: <app_client_id>
107+
client_secret: <app_secret>
108+
ingestion_endpoint: https://ingest-<cluster>.<region>.kusto.windows.net
109+
database_name: <database_name>
110+
table_name: <table_name>
111+
ingestion_mapping_reference: <mapping_name>
112+
ingestion_endpoint_connect_timeout: <ingestion_endpoint_connect_timeout>
113+
compression_enabled: <compression_enabled>
114+
ingestion_resources_refresh_interval: <ingestion_resources_refresh_interval>
115+
buffering_enabled: on
116+
upload_timeout: 2m
117+
upload_file_size: 125M
118+
azure_kusto_buffer_key: kusto1
119+
buffer_file_delete_early: off
120+
unify_tag: on
121+
buffer_dir: /var/log/
122+
store_dir_limit_size: 16GB
123+
blob_uri_length: 128
124+
scheduler_max_retries: 3
125+
delete_on_max_upload_error: off
126+
io_timeout: 60s
87127
```
128+
129+
{% endtab %}
130+
{% tab title="fluent-bit.conf" %}
131+
132+
```text
88133
[OUTPUT]
89-
Match *
90134
Name azure_kusto
135+
Match *
91136
Tenant_Id <app_tenant_id>
92137
Client_Id <app_client_id>
93138
Client_Secret <app_secret>
@@ -110,13 +155,17 @@ Get started quickly with this configuration file:
110155
scheduler_max_retries 3
111156
delete_on_max_upload_error Off
112157
io_timeout 60s
158+
113159
```
114160

161+
{% endtab %}
162+
{% endtabs %}
163+
115164
## Troubleshooting
116165

117166
### 403 Forbidden
118167

119168
If you get a `403 Forbidden` error response, make sure that:
120169

121170
- You provided the correct AAD registered application credentials.
122-
- You authorized the application to ingest into your database or table.
171+
- You authorized the application to ingest into your database or table.

0 commit comments

Comments
 (0)