You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The _AWS Filter_Enriches logs with AWS Metadata. Currently the plugin adds the EC2 instance ID and availability zone to log records. To use this plugin, you must be running in EC2 and have the [instance metadata service enabled](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-service.html).
3
+
The _AWS Filter_enriches logs with AWS Metadata. The plugin adds the EC2 instance ID and availability zone to log records. To use this plugin, you must be running in EC2 and have the [instance metadata service enabled](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-service.html).
4
4
5
-
## Configuration Parameters
5
+
## Configuration parameters
6
6
7
7
The plugin supports the following configuration parameters:
8
8
9
9
| Key | Description | Default |
10
10
| :--- | :--- | :--- |
11
-
| imds\_version | Specify which version of the instance metadata service to use. Valid values are 'v1' or 'v2'. | v2 |
12
-
| az | The [availability zone](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html); for example, "us-east-1a". | true |
13
-
| ec2\_instance\_id | The EC2 instance ID. | true |
14
-
| ec2\_instance\_type | The EC2 instance type. | false |
| account\_id | The account ID for current EC2 instance. | false |
18
-
| hostname | The hostname for current EC2 instance. | false |
19
-
| vpc\_id | The VPC ID for current EC2 instance. | false |
20
-
| tags\_enabled | Specifies if should attach EC2 instance tags. EC2 instance must have the [instance-metadata-tags](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/ec2/modify-instance-metadata-options.html) option enabled (which is disabled by default). | false |
21
-
| tags\_include | Defines list of specific EC2 tag keys to inject into the logs. Tag keys must be separated by "," character. Tags which are not present in this list will be ignored. Example: `Name,tag1,tag2`. ||
22
-
| tags\_exclude | Defines list of specific EC2 tag keys not to inject into the logs. Tag keys must be separated by "," character. Tags which are not present in this list will be injected into the logs. If both `tags_include` and `tags_exclude` are specified, configuration is invalid and plugin fails. Example: `Name,tag1,tag2`||
23
-
| retry\_interval\_s |Defines minimum duration between retries for fetching EC2 instance tags. | 300 |
24
-
25
-
Note: _If you run Fluent Bit in a container, you may have to use instance metadata v1._ The plugin behaves the same regardless of which version is used.
26
-
27
-
### Command Line
11
+
|`imds_version`| Specify which version of the instance metadata service to use. Valid values are `v1` and `v2`. |`v2`|
12
+
|`az`| The [availability zone](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html), such as `us-east-1a`. |`true`|
13
+
|`ec2_instance_id`| The EC2 instance ID. |`true`|
14
+
|`ec2_instance_type`| The EC2 instance type. |`false`|
15
+
|`private_ip`| The EC2 instance private IP. |`false`|
16
+
|`ami_id`| The EC2 instance image ID. |`false`|
17
+
|`account_id`| The account ID for the current EC2 instance. |`false`|
18
+
|`hostname`| The hostname for the current EC2 instance. |`false`|
19
+
|`vpc_id`| The VPC ID for the current EC2 instance. |`false`|
20
+
|`tags_enabled`| Specifies whether to attach EC2 instance tags. The EC2 instance must have the [`instance-metadata-tags`](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/ec2/modify-instance-metadata-options.html) option enabled, which is disabled by default. |`false`|
21
+
|`tags_include`| Defines a list of specific EC2 tag keys to inject into the logs. Tag keys must be comma-separated (`,`). Tags not included in this list will be ignored. Example: `Name,tag1,tag2`. |_none_|
22
+
|`tags_exclude`| Defines a list of specific EC2 tag keys not to inject into the logs. Tag keys must be comma-separated (`,`). Tags not included in this list will be injected into the logs. If both `tags_include` and `tags_exclude` are specified, the configuration is invalid and the plugin fails. Example: `Name,tag1,tag2`|_none_|
23
+
|`retry_interval_s`|Defines minimum duration between retries for fetching EC2 instance tags. |`300`|
24
+
25
+
If you run Fluent Bit in a container, you might need to use instance metadata v1. The plugin behaves the same regardless of which version is used.
EC2 Tags are a useful feature that enables you to label and organize your EC2 instances by creating custom-defined key-value pairs. These tags are commonly utilized for resource management, cost allocation, and automation. Consequently, including them in the Fluent Bitgenerated logs is almost essential.
73
+
EC2 Tags let you label and organize your EC2 instances by creating custom-defined key-value pairs. These tags are commonly used for resource management, cost allocation, and automation. Including them in the Fluent Bit-generated logs is almost essential.
66
74
67
-
To achieve this, AWS Filter can be configured with `tags_enabled true` to enable the _tagging_ of logs with the relevant EC2 instance tags. This setup ensures that logs are appropriately tagged, making it easier to manage and analyze them based on specific criteria.
75
+
To achieve this, AWS Filter can be configured with `tags_enabled true` to enable the tagging of logs with the relevant EC2 instance tags. This setup ensures that logs are appropriately tagged, making it easier to manage and analyze them based on specific criteria.
68
76
69
77
### Requirements
70
78
71
-
To use the `tags_enabled true`functionality in Fluent Bit, the [instance-metadata-tags](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/ec2/modify-instance-metadata-options.html) option must be enabled on the EC2 instance where Fluent Bit is running. Without this option enabled, Fluent Bit will not be able to retrieve the tags associated with the EC2 instance. However, this does not mean that Fluent Bit will fail or stop working altogether. Instead, if [instance-metadata-tags](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/ec2/modify-instance-metadata-options.html) option is not enabled, Fluent Bit will continue to operate normally and capture other values, such as the EC2 instance ID or availability zone, based on its configuration.
79
+
To use the `tags_enabled true`feature in Fluent Bit, the [instance-metadata-tags](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/ec2/modify-instance-metadata-options.html) option must be enabled on the EC2 instance where Fluent Bit is running. Without this option enabled, Fluent Bit won't be able to retrieve the tags associated with the EC2 instance. However, this doesn't mean that Fluent Bit will fail or stop working altogether. Instead, if [instance-metadata-tags](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/ec2/modify-instance-metadata-options.html) option isn't enabled, Fluent Bit will continue to operate normally and capture other values, such as the EC2 instance ID or availability zone, based on its configuration.
72
80
73
81
### Example
74
82
75
-
#### tags_include
83
+
#### `tags_include`
76
84
77
-
Assume that our EC2 instance has many tags, some of which have lengthy values that are irrelevant to the logs we want to collect. Only two tags, `department` and `project`, seem to be valuable for our purpose. Here is a configuration which reflects this requirement:
85
+
Assume the EC2 instance has many tags, some of which have lengthy values that are irrelevant to the logs you want to collect. Only two tags, `department` and `project`, are valuable for your purpose. The following configuration reflects this requirement:
78
86
79
-
```
87
+
```python
80
88
[FILTER]
81
89
Name aws
82
90
Match *
83
91
tags_enabled true
84
92
tags_include department,project
85
93
```
86
94
87
-
If we run Fluent Bit, what will the logs look like? Here is an example of what the logs might contain:
88
-
```
95
+
If you run Fluent Bit logs might look like the following:
96
+
97
+
```text
89
98
{"log"=>"fluentbit is awesome", "az"=>"us-east-1a", "ec2_instance_id"=>"i-0e66fc7f9809d7168", "department"=>"it", "project"=>"fluentbit"}
90
99
```
91
100
92
-
#### tags_exclude
101
+
#### `tags_exclude`
93
102
94
-
Suppose our EC2 instance has three tags: `Name:fluent-bit-docs-example`, `project:fluentbit`, and `department:it`. In this example, we want to exclude the `department` tag since we consider it redundant. This is because all of our projects belong to the `it` department, and we do not need to waste storage space on redundant labels.
103
+
Suppose the EC2 instance has three tags: `Name:fluent-bit-docs-example`, `project:fluentbit`, and `department:it`. In this example, the `department` tag is redundant and will be excluded. All of the projects belong to the `it` department, and you don't want to waste storage space on redundant labels.
95
104
96
105
Here is an example configuration that achieves this:
97
106
98
-
```
107
+
```python
99
108
[FILTER]
100
109
Name aws
101
110
Match *
@@ -105,6 +114,6 @@ Here is an example configuration that achieves this:
105
114
106
115
The resulting logs might look like this:
107
116
108
-
```
117
+
```shell
109
118
{"log"=>"aws is awesome", "az"=>"us-east-1a", "ec2_instance_id"=>"i-0e66fc7f9809d7168", "Name"=>"fluent-bit-docs-example", "project"=>"fluentbit"}
0 commit comments