You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: pipeline/inputs/tail.md
+25-25Lines changed: 25 additions & 25 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,32 +8,32 @@ The plugin reads every matched file in the `Path` pattern and for every new line
8
8
9
9
The plugin supports the following configuration parameters:
10
10
11
-
| Key | Description | Default |
12
-
| :--- |:---| :--- |
13
-
| Buffer\_Chunk\_Size | Set the initial buffer size to read files data. This value is used to increase buffer size. The value must be according to the [Unit Size](../../administration/configuring-fluent-bit/unit-sizes.md) specification. | 32k |
14
-
| Buffer\_Max\_Size | Set the limit of the buffer size per monitored file. When a buffer needs to be increased \(e.g: very long lines\), this value is used to restrict how much the memory buffer can grow. If reading a file exceeds this limit, the file is removed from the monitored file list. The value must be according to the [Unit Size](../../administration/configuring-fluent-bit/unit-sizes.md) specification. | 32k |
15
-
| Path | Pattern specifying a specific log file or multiple ones through the use of common wildcards. Multiple patterns separated by commas are also allowed. ||
16
-
| Path\_Key | If enabled, it appends the name of the monitored file as part of the record. The value assigned becomes the key in the map. ||
17
-
| Exclude\_Path | Set one or multiple shell patterns separated by commas to exclude files matching certain criteria, e.g: `Exclude_Path *.gz,*.zip`||
18
-
| Offset\_Key | If enabled, Fluent Bit appends the offset of the current monitored file as part of the record. The value assigned becomes the key in the map ||
19
-
| Read\_from\_Head | For new discovered files on start \(without a database offset/position\), read the content from the head of the file, not tail. | False |
20
-
| Refresh\_Interval | The interval of refreshing the list of watched files in seconds. | 60 |
21
-
| Rotate\_Wait | Specify the number of extra time in seconds to monitor a file once is rotated in case some pending data is flushed. | 5 |
22
-
| Ignore\_Older | Ignores records which are older than this time in seconds. Supports m,h,d \(minutes, hours, days\) syntax. Default behavior is to read all records from specified files. Only available when a Parser is specified and it can parse the time of a record.||
23
-
| Skip\_Long\_Lines | When a monitored file reaches its buffer capacity due to a very long line \(Buffer\_Max\_Size\), the default behavior is to stop monitoring that file. Skip\_Long\_Lines alter that behavior and instruct Fluent Bit to skip long lines and continue processing other lines that fits into the buffer size. | Off |
24
-
| Skip\_Empty\_Lines | Skips empty lines in the log file from any further processing or output. | Off |
25
-
| DB | Specify the database file to keep track of monitored files and offsets. ||
| Buffer\_Chunk\_Size | Set the initial buffer size to read files data. This value is used to increase buffer size. The value must be according to the [Unit Size](../../administration/configuring-fluent-bit/unit-sizes.md) specification. | 32k |
14
+
| Buffer\_Max\_Size | Set the limit of the buffer size per monitored file. When a buffer needs to be increased \(e.g: very long lines\), this value is used to restrict how much the memory buffer can grow. If reading a file exceeds this limit, the file is removed from the monitored file list. The value must be according to the [Unit Size](../../administration/configuring-fluent-bit/unit-sizes.md) specification. | 32k |
15
+
| Path | Pattern specifying a specific log file or multiple ones through the use of common wildcards. Multiple patterns separated by commas are also allowed. ||
16
+
| Path\_Key | If enabled, it appends the name of the monitored file as part of the record. The value assigned becomes the key in the map. ||
17
+
| Exclude\_Path | Set one or multiple shell patterns separated by commas to exclude files matching certain criteria, e.g: `Exclude_Path *.gz,*.zip`||
18
+
| Offset\_Key | If enabled, Fluent Bit appends the offset of the current monitored file as part of the record. The value assigned becomes the key in the map ||
19
+
| Read\_from\_Head | For new discovered files on start \(without a database offset/position\), read the content from the head of the file, not tail. | False |
20
+
| Refresh\_Interval | The interval of refreshing the list of watched files in seconds. | 60 |
21
+
| Rotate\_Wait | Specify the number of extra time in seconds to monitor a file once is rotated in case some pending data is flushed. | 5 |
22
+
| Ignore\_Older | Ignores files which modification date is older than this time in seconds. Supports m,h,d \(minutes, hours, days\) syntax. ||
23
+
| Skip\_Long\_Lines | When a monitored file reaches its buffer capacity due to a very long line \(Buffer\_Max\_Size\), the default behavior is to stop monitoring that file. Skip\_Long\_Lines alter that behavior and instruct Fluent Bit to skip long lines and continue processing other lines that fits into the buffer size. | Off |
24
+
| Skip\_Empty\_Lines | Skips empty lines in the log file from any further processing or output. | Off |
25
+
| DB | Specify the database file to keep track of monitored files and offsets. ||
26
26
| DB.sync | Set a default synchronization \(I/O\) method. Values: Extra, Full, Normal, Off. This flag affects how the internal SQLite engine do synchronization to disk, for more details about each option please refer to [this section](https://www.sqlite.org/pragma.html#pragma_synchronous). Most of workload scenarios will be fine with `normal` mode, but if you really need full synchronization after every write operation you should set `full` mode. Note that `full` has a high I/O performance cost. | normal |
27
-
| DB.locking | Specify that the database will be accessed only by Fluent Bit. Enabling this feature helps to increase performance when accessing the database but it restrict any external tool to query the content. | false |
28
-
| DB.journal\_mode | sets the journal mode for databases \(WAL\). Enabling WAL provides higher performance. Note that WAL is not compatible with shared network file systems. | WAL |
29
-
| Mem\_Buf\_Limit | Set a limit of memory that Tail plugin can use when appending data to the Engine. If the limit is reach, it will be paused; when the data is flushed it resumes. ||
30
-
| Exit\_On\_Eof | When reading a file will exit as soon as it reach the end of the file. Useful for bulk load and tests | false |
31
-
| Parser | Specify the name of a parser to interpret the entry as a structured message. ||
32
-
| Key | When a message is unstructured \(no parser applied\), it's appended as a string under the key name _log_. This option allows to define an alternative name for that key. | log |
33
-
| Inotify_Watcher | Set to false to use file stat watcher instead of inotify. | true |
34
-
| Tag | Set a tag \(with regex-extract fields\) that will be placed on lines read. E.g. `kube.<namespace_name>.<pod_name>.<container_name>`. Note that "tag expansion" is supported: if the tag includes an asterisk \(\*\), that asterisk will be replaced with the absolute path of the monitored file \(also see [Workflow of Tail + Kubernetes Filter](../filters/kubernetes.md#workflow-of-tail-kubernetes-filter)\). ||
35
-
| Tag\_Regex | Set a regex to extract fields from the file name. E.g. `(?<pod_name>[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*)_(?<namespace_name>[^_]+)_(?<container_name>.+)-`||
36
-
| Static\_Batch\_Size | Set the maximum number of bytes to process per iteration for the monitored static files (files that already exists upon Fluent Bit start). | 50M |
27
+
| DB.locking | Specify that the database will be accessed only by Fluent Bit. Enabling this feature helps to increase performance when accessing the database but it restrict any external tool to query the content. | false |
28
+
| DB.journal\_mode | sets the journal mode for databases \(WAL\). Enabling WAL provides higher performance. Note that WAL is not compatible with shared network file systems. | WAL |
29
+
| Mem\_Buf\_Limit | Set a limit of memory that Tail plugin can use when appending data to the Engine. If the limit is reach, it will be paused; when the data is flushed it resumes. ||
30
+
| Exit\_On\_Eof | When reading a file will exit as soon as it reach the end of the file. Useful for bulk load and tests | false |
31
+
| Parser | Specify the name of a parser to interpret the entry as a structured message. ||
32
+
| Key | When a message is unstructured \(no parser applied\), it's appended as a string under the key name _log_. This option allows to define an alternative name for that key. | log |
33
+
| Inotify_Watcher | Set to false to use file stat watcher instead of inotify. | true |
34
+
| Tag | Set a tag \(with regex-extract fields\) that will be placed on lines read. E.g. `kube.<namespace_name>.<pod_name>.<container_name>`. Note that "tag expansion" is supported: if the tag includes an asterisk \(\*\), that asterisk will be replaced with the absolute path of the monitored file \(also see [Workflow of Tail + Kubernetes Filter](../filters/kubernetes.md#workflow-of-tail-kubernetes-filter)\). ||
35
+
| Tag\_Regex | Set a regex to extract fields from the file name. E.g. `(?<pod_name>[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*)_(?<namespace_name>[^_]+)_(?<container_name>.+)-`||
36
+
| Static\_Batch\_Size | Set the maximum number of bytes to process per iteration for the monitored static files (files that already exists upon Fluent Bit start). | 50M |
37
37
38
38
Note that if the database parameter `DB` is **not** specified, by default the plugin will start reading each target file from the beginning. This also might cause some unwanted behavior, for example when a line is bigger that `Buffer_Chunk_Size` and `Skip_Long_Lines` is not turned on, the file will be read from the beginning of each `Refresh_Interval` until the file is rotated.
0 commit comments