Revert "Revert "Update the filter docs with YAML configuration examples, the first half of filters in list. Fixes #1872.""

pipeline/filters/aws-metadata.md

@@ -29,12 +29,12 @@ If you run Fluent Bit in a container, you might need to use instance metadata v1
 Run Fluent Bit from the command line:
 
 ```shell
-bin/fluent-bit -c /PATH_TO_CONF_FILE/fluent-bit.conf
+$ ./fluent-bit -c /PATH_TO_CONF_FILE/fluent-bit.conf
 ```
 
 You should see results like this:
 
-```shell
+```text
 [2020/01/17 07:57:17] [ info] [engine] started (pid=32744)
 [0] dummy: [1579247838.000171227, {"message"=>"dummy", "az"=>"us-west-2c", "ec2_instance_id"=>"i-0c862eca9038f5aae", "ec2_instance_type"=>"t2.medium", "private_ip"=>"172.31.6.59", "vpc_id"=>"vpc-7ea11c06", "ami_id"=>"ami-0841edc20334f9287", "account_id"=>"YOUR_ACCOUNT_ID", "hostname"=>"ip-172-31-6-59.us-west-2.compute.internal"}]
 [0] dummy: [1601274509.970235760, {"message"=>"dummy", "az"=>"us-west-2c", "ec2_instance_id"=>"i-0c862eca9038f5aae", "ec2_instance_type"=>"t2.medium", "private_ip"=>"172.31.6.59", "vpc_id"=>"vpc-7ea11c06", "ami_id"=>"ami-0841edc20334f9287", "account_id"=>"YOUR_ACCOUNT_ID", "hostname"=>"ip-172-31-6-59.us-west-2.compute.internal"}]
@@ -44,7 +44,38 @@ You should see results like this:
 
 The following is an example of a configuration file:
 
-```python
+{% tabs %}
+{% tab title="fluent-bit.yaml" %}
+
+```yaml
+pipeline:
+    inputs:
+        - name: dummy
+          tag: dummy
+
+    filters:
+        - name: aws
+          match: '*'
+          imds_version: v1
+          az: true
+          ec2_instance_id: true
+          ec2_instance_type: true
+          private_ip: true
+          ami_id: true
+          account_id: true
+          hostname: true
+          vpc_id: true
+          tags_enabled: true
+
+    outputs:
+        - name: stdout
+          match: '*'
+```
+
+{% endtab %}
+{% tab title="fluent-bit.conf" %}
+
+```text
 [INPUT]
     Name dummy
     Tag dummy
@@ -68,6 +99,9 @@ The following is an example of a configuration file:
     Match *
 ```
 
+{% endtab %}
+{% endtabs %}
+
 ## EC2 tags
 
 EC2 Tags let you label and organize your EC2 instances by creating custom-defined key-value pairs. These tags are commonly used for resource management, cost allocation, and automation. Including them in the Fluent Bit-generated logs is almost essential.
@@ -84,14 +118,33 @@ To use the `tags_enabled true` feature in Fluent Bit, the [instance-metadata-tag
 
 Assume the EC2 instance has many tags, some of which have lengthy values that are irrelevant to the logs you want to collect. Only two tags, `department` and `project`, are valuable for your purpose. The following configuration reflects this requirement:
 
-```python
+{% tabs %}
+{% tab title="fluent-bit.yaml" %}
+
+```yaml
+pipeline:
+
+    filters:
+        - name: aws
+          match: '*'
+          tags_enabled: true
+          tags_include: department,project
+```
+
+{% endtab %}
+{% tab title="fluent-bit.conf" %}
+
+```text
 [FILTER]
     Name aws
     Match *
     tags_enabled true
     tags_include department,project
 ```
 
+{% endtab %}
+{% endtabs %}
+
 If you run Fluent Bit logs might look like the following:
 
 ```text
@@ -104,16 +157,35 @@ Suppose the EC2 instance has three tags: `Name:fluent-bit-docs-example`, `projec
 
 Here is an example configuration that achieves this:
 
-```python
+{% tabs %}
+{% tab title="fluent-bit.yaml" %}
+
+```yaml
+pipeline:
+
+    filters:
+        - name: aws
+          match: '*'
+          tags_enabled: true
+          tags_exclude: department
+```
+
+{% endtab %}
+{% tab title="fluent-bit.conf" %}
+
+```text
 [FILTER]
     Name aws
     Match *
     tags_enabled true
     tags_exclude department
 ```
 
+{% endtab %}
+{% endtabs %}
+
 The resulting logs might look like this:
 
-```shell
+```text
 {"log"=>"aws is awesome", "az"=>"us-east-1a", "ec2_instance_id"=>"i-0e66fc7f9809d7168", "Name"=>"fluent-bit-docs-example", "project"=>"fluentbit"}
-```
+```

pipeline/filters/ecs-metadata.md

@@ -35,9 +35,50 @@ The following template variables can be used for values with the `ADD` option. S
 
 ### Configuration file
 
+Below configurations assume a properly configured parsers file and 'storage.path' variable defined in the services
+section of the fluent bit configuration (not shown below).
+
 #### Example 1: Attach Task ID and cluster name to container logs
 
-```python
+{% tabs %}
+{% tab title="fluent-bit.yaml" %}
+
+```yaml
+pipeline:
+    inputs:
+        - name: tail
+          tag: ecs.*
+          path: /var/lib/docker/containers/*/*.log
+          docker_mode: on
+          docker_mode_flush: 5
+          docker_mode_parser: container_firstline
+          parser: docker
+          db: /var/fluent-bit/state/flb_container.db
+          mem_buf_limit: 50MB
+          skip_long_lines: on
+          refresh_interval: 10
+          rotate_wait: 30
+          storage.type: filesystem
+          read_from_head: off
+
+    filters:
+        - name: ecs
+          match: '*'
+          ecs_tag_prefix: ecs.var.lib.docker.containers.
+          add: 
+            - ecs_task_id $TaskID
+            - cluster $ClusterName
+
+    outputs:
+        - name: stdout
+          match: '*'
+          format: json_lines
+```
+
+{% endtab %}
+{% tab title="fluent-bit.conf" %}
+
+```text
 [INPUT]
     Name                tail
     Tag                 ecs.*
@@ -67,6 +108,9 @@ The following template variables can be used for values with the `ADD` option. S
     Format json_lines
 ```
 
+{% endtab %}
+{% endtabs %}
+
 The output log should be similar to:
 
 ```text
@@ -80,6 +124,42 @@ The output log should be similar to:
 
 #### Example 2: Attach customized resource name to container logs
 
+{% tabs %}
+{% tab title="fluent-bit.yaml" %}
+
+```yaml
+pipeline:
+    inputs:
+        - name: tail
+          tag: ecs.*
+          path: /var/lib/docker/containers/*/*.log
+          docker_mode: on
+          docker_mode_flush: 5
+          docker_mode_parser: container_firstline
+          parser: docker
+          db: /var/fluent-bit/state/flb_container.db
+          mem_buf_limit: 50MB
+          skip_long_lines: on
+          refresh_interval: 10
+          rotate_wait: 30
+          storage.type: filesystem
+          read_from_head: off
+
+    filters:
+        - name: ecs
+          match: '*'
+          ecs_tag_prefix: ecs.var.lib.docker.containers.
+          add: resource $ClusterName.$TaskDefinitionFamily.$TaskID.$ECSContainerName
+
+    outputs:
+        - name: stdout
+          match: '*'
+          format: json_lines
+```
+
+{% endtab %}
+{% tab title="fluent-bit.conf" %}
+
 ```text
 [INPUT]
     Name                tail
@@ -109,6 +189,9 @@ The output log should be similar to:
     Format json_lines
 ```
 
+{% endtab %}
+{% endtabs %}
+
 The output log would be similar to:
 
 ```text
@@ -124,9 +207,42 @@ The template variables in the value for the `resource` key are separated by dot
 
 #### Example 3: Attach cluster metadata to non-container logs
 
-This examples shows a use case for the `Cluster_Metadata_Only` option- attaching cluster metadata to ECS Agent logs.
+This examples shows a use case for the `Cluster_Metadata_Only` option attaching cluster metadata to ECS Agent logs.
+
+{% tabs %}
+{% tab title="fluent-bit.yaml" %}
+
+```yaml
+pipeline:
+    inputs:
+        - name: tail
+          tag: ecsagent.*
+          path: /var/log/ecs/*
+          db: /var/fluent-bit/state/flb_ecs.db
+          mem_buf_limit: 50MB
+          skip_long_lines: on
+          refresh_interval: 10
+          rotate_wait: 30
+          storage.type: filesystem
+         # Collect all logs on instance
+          read_from_head: on
 
-```python
+    filters:
+        - name: ecs
+          match: '*'
+          cluster_metadata_only: on
+          add: cluster $ClusterName
+
+    outputs:
+        - name: stdout
+          match: '*'
+          format: json_lines
+```
+
+{% endtab %}
+{% tab title="fluent-bit.conf" %}
+
+```text
 [INPUT]
     Name                tail
     Tag                 ecsagent.*
@@ -151,3 +267,6 @@ This examples shows a use case for the `Cluster_Metadata_Only` option- attaching
     Match *
     Format json_lines
 ```
+
+{% endtab %}
+{% endtabs %}

pipeline/filters/geoip2-filter.md

@@ -22,7 +22,33 @@ This plugin supports the following configuration parameters:
 
 The following configuration processes the incoming `remote_addr` and appends country information retrieved from the GeoLite2 database.
 
-```python
+{% tabs %}
+{% tab title="fluent-bit.yaml" %}
+
+```yaml
+pipeline:
+    inputs:
+        - name: dummy
+          dummy: {"remote_addr": "8.8.8.8"}
+
+    filters:
+        - name: gioip2
+          match: '*'
+          database: GioLite2-City.mmdb
+          lookup_key: remote_addr
+          record:
+              - country remote_addr %{country.names.en}
+              - isocode remote_addr %{country.iso_code}
+
+    outputs:
+        - name: stdout
+          match: '*'
+```
+
+{% endtab %}
+{% tab title="fluent-bit.conf" %}
+
+```text
 [INPUT]
     Name   dummy
     Dummy  {"remote_addr": "8.8.8.8"}
@@ -40,6 +66,9 @@ The following configuration processes the incoming `remote_addr` and appends cou
     Match  *
 ```
 
+{% endtab %}
+{% endtabs %}
+
 Each `Record` parameter specifies the following triplet:
 
 - `country`: The field name to be added to records.
@@ -48,6 +77,6 @@ Each `Record` parameter specifies the following triplet:
 
 By running Fluent Bit with this configuration, you will see the following output:
 
-```javascript
+```text
 {"remote_addr": "8.8.8.8", "country": "United States", "isocode": "US"}
-```
+```