You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+18-29Lines changed: 18 additions & 29 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,11 +8,15 @@ It is fully free and fully open source. The license is Apache 2.0, meaning you a
8
8
9
9
This plugin will enable you to process events from Logstash into an **Azure Kusto** database for later analysis.
10
10
11
+
## Requirements
12
+
13
+
- Logstash version 6+. You can find more information on how to install it by reading [this](https://www.elastic.co/guide/en/logstash/current/installing-logstash.html)
14
+
- Kusto cluster with a database. You can follow the steps outlined [here](https://docs.microsoft.com/en-us/azure/data-explorer/create-cluster-database-portal) to do this.
15
+
- AAD Application credentials with premission to ingest data into Kusto. Read this [document](https://docs.microsoft.com/en-us/azure/kusto/management/access-control/how-to-provision-aad-app) for more info.
16
+
11
17
## Installation
12
18
13
-
To make the plugin available in your Logstash environment follow these steps:
14
-
- Download the latest release from this repository
15
-
- Install the plugin from the Logstash home
19
+
To make the plugin available in your Logstash environment run the following command:
16
20
```sh
17
21
bin/logstash-plugin install logstash-output-kusto
18
22
```
@@ -38,32 +42,17 @@ output {
38
42
39
43
### Available Configuration Keys
40
44
41
-
#### path
42
-
The plugin writes events to temporary files before sending them to Kusto. These settings should include a path where these files should be written as well as some sort of time expression to note when file rotation should happen and trigger an upload to the Kusto service. The example above shows how to rotate the files every minute, check the Logstash docs for more information on time expressions.
43
-
44
-
#### ingest_url
45
-
The Kusto endpoint for ingestion related communication. You can see it on the Azure Portal.
46
-
47
-
#### app_id, app_key, app_tenant
48
-
Those are the credentials required to connect to the Kusto service. Be sure to use an application with 'ingest' privilages.
49
-
50
-
#### database
51
-
Database name where events should go to
52
-
53
-
#### table
54
-
Target table name where events should be saved
55
-
56
-
#### mapping
57
-
The mapping object name used by Kusto to map an incoming event to the right row format (what value goes into which column)
58
-
59
-
#### recovery
60
-
If this is set to true (the default), the plugin will attempt to resend pre-existing temp files it finds in the path upon startup
61
-
62
-
#### delete_temp_files
63
-
Determines if temp files will be deleted after a successful upload (default is true, set false only for debug purposes)
64
-
65
-
#### flush_interval
66
-
The time (in seconds) for flushing writes to temporary files. Defaults to 2 seconds, 0 will flush on every event. Increase this value to reduce IO calls but keep in mind that events in the buffer will be lost in case of abrupt failure.
45
+
| Parameter Name | Description | Notes |
46
+
| --- | --- | --- |
47
+
| **path** | The plugin writes events to temporary files before sending them to Kusto. This parameter should include a path where these files should be written and a time expression to note when file rotation should happen and trigger an upload to the Kusto service. The example above shows how to rotate the files every minute, check the Logstash docs for more information on time expressions. | Required.
48
+
|**ingest_url**| The Kusto endpoint for ingestion related communication. You can see it on the Azure Portal.| Required.|
49
+
|**app_id, app_key, app_tenant**| Those are the credentials required to connect to the Kusto service. Be sure to use an application with 'ingest' privilages. | Required.|
50
+
|**database**| Database name where events should go to. | Required. |
51
+
| **table** | Target table name where events should be saved | Required.
52
+
|**mapping**| A mapping is used by Kusto to map an incoming event json string to the right row format (what property goes into which column) | Required. |
53
+
|**recovery**| If this is set to true (the default), the plugin will attempt to resend pre-existing temp files it finds in the path upon startup ||
54
+
|**delete_temp_files**| Determines if temp files will be deleted after a successful upload (default is true, set false only for debug purposes)||
55
+
|**flush_interval**| The time (in seconds) for flushing writes to temporary files. Defaults to 2 seconds, 0 will flush on every event. Increase this value to reduce IO calls but keep in mind that events in the buffer will be lost in case of abrupt failure.||
0 commit comments