Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions pipeline/filters/aws-metadata.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ If you run Fluent Bit in a container, you might need to use instance metadata v1
Run Fluent Bit from the command line:

```shell
$ ./fluent-bit -c /PATH_TO_CONF_FILE/fluent-bit.conf
fluent-bit -c /PATH_TO_CONF_FILE/fluent-bit.conf
```

You should see results like this:
Expand Down Expand Up @@ -66,7 +66,7 @@ pipeline:
hostname: true
vpc_id: true
tags_enabled: true

outputs:
- name: stdout
match: '*'
Expand Down Expand Up @@ -188,4 +188,4 @@ The resulting logs might look like this:

```text
{"log"=>"aws is awesome", "az"=>"us-east-1a", "ec2_instance_id"=>"i-0e66fc7f9809d7168", "Name"=>"fluent-bit-docs-example", "project"=>"fluentbit"}
```
```
16 changes: 8 additions & 8 deletions pipeline/filters/ecs-metadata.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,8 @@ The following template variables can be used for values with the `ADD` option. S

### Configuration file

Below configurations assume a properly configured parsers file and 'storage.path' variable defined in the services
section of the fluent bit configuration (not shown below).
The following configurations assume a properly configured parsers file and 'storage.path' variable defined in the services
section of the Fluent bit Configuration (not shown).

#### Example 1: Attach Task ID and cluster name to container logs

Expand Down Expand Up @@ -65,10 +65,10 @@ pipeline:
- name: ecs
match: '*'
ecs_tag_prefix: ecs.var.lib.docker.containers.
add:
add:
- ecs_task_id $TaskID
- cluster $ClusterName

outputs:
- name: stdout
match: '*'
Expand Down Expand Up @@ -150,7 +150,7 @@ pipeline:
match: '*'
ecs_tag_prefix: ecs.var.lib.docker.containers.
add: resource $ClusterName.$TaskDefinitionFamily.$TaskID.$ECSContainerName

outputs:
- name: stdout
match: '*'
Expand Down Expand Up @@ -207,7 +207,7 @@ The template variables in the value for the `resource` key are separated by dot

#### Example 3: Attach cluster metadata to non-container logs

This examples shows a use case for the `Cluster_Metadata_Only` option attaching cluster metadata to ECS Agent logs.
This example shows a use case for the `Cluster_Metadata_Only` option attaching cluster metadata to ECS Agent logs.

{% tabs %}
{% tab title="fluent-bit.yaml" %}
Expand All @@ -232,7 +232,7 @@ pipeline:
match: '*'
cluster_metadata_only: on
add: cluster $ClusterName

outputs:
- name: stdout
match: '*'
Expand Down Expand Up @@ -269,4 +269,4 @@ pipeline:
```

{% endtab %}
{% endtabs %}
{% endtabs %}
20 changes: 10 additions & 10 deletions pipeline/filters/record-modifier.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ pipeline:
inputs:
- name: mem
tag: mem.local

filters:
- name: record_modifier
match: '*'
Expand Down Expand Up @@ -76,7 +76,7 @@ pipeline:
You can run the filter from command line:

```shell
$ ./fluent-bit -i mem -o stdout -F record_modifier -p 'Record=hostname ${HOSTNAME}' -p 'Record=product Awesome_Tool' -m '*'
fluent-bit -i mem -o stdout -F record_modifier -p 'Record=hostname ${HOSTNAME}' -p 'Record=product Awesome_Tool' -m '*'
```

The output looks something like:
Expand All @@ -97,15 +97,15 @@ pipeline:
inputs:
- name: mem
tag: mem.local

filters:
- name: record_modifier
match: '*'
remove_key:
- Swap.total
- Swap.used
- Swap.free

outputs:
- name: stdout
match: '*'
Expand Down Expand Up @@ -137,7 +137,7 @@ pipeline:
You can also run the filter from command line.

```shell
$ ./fluent-bit -i mem -o stdout -F record_modifier -p 'Remove_key=Swap.total' -p 'Remove_key=Swap.free' -p 'Remove_key=Swap.used' -m '*'
fluent-bit -i mem -o stdout -F record_modifier -p 'Remove_key=Swap.total' -p 'Remove_key=Swap.free' -p 'Remove_key=Swap.used' -m '*'
```

The output looks something like:
Expand All @@ -158,15 +158,15 @@ pipeline:
inputs:
- name: mem
tag: mem.local

filters:
- name: record_modifier
match: '*'
Allowlist_key:
- Mem.total
- Mem.used
- Mem.free

outputs:
- name: stdout
match: '*'
Expand All @@ -186,7 +186,7 @@ pipeline:
Allowlist_key Mem.total
Allowlist_key Mem.used
Allowlist_key Mem.free

[OUTPUT]
Name stdout
Match *
Expand All @@ -198,11 +198,11 @@ pipeline:
You can also run the filter from command line:

```shell
$ ./fluent-bit -i mem -o stdout -F record_modifier -p 'Allowlist_key=Mem.total' -p 'Allowlist_key=Mem.free' -p 'Allowlist_key=Mem.used' -m '*'
fluent-bit -i mem -o stdout -F record_modifier -p 'Allowlist_key=Mem.total' -p 'Allowlist_key=Mem.free' -p 'Allowlist_key=Mem.used' -m '*'
```

The output looks something like:

```text
[0] mem.local: [1492436998.000000000, {"Mem.total"=>1016024, "Mem.used"=>716672, "Mem.free"=>295332}]
```
```
22 changes: 11 additions & 11 deletions pipeline/filters/tensorflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,33 +25,33 @@ The plugin supports the following configuration parameters:
To create a Tensorflow Lite shared library:

1. Clone the [Tensorflow repository](https://github.com/tensorflow/tensorflow).
2. Install the [Bazel](https://bazel.build/) package manager.
3. Run the following command to create the shared library:
1. Install the [Bazel](https://bazel.build/) package manager.
1. Run the following command to create the shared library:

```shell
$ ./bazel build -c opt //tensorflow/lite/c:tensorflowlite_c # see https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/c
bazel build -c opt //tensorflow/lite/c:tensorflowlite_c # see https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/c
```

The script creates the shared library
`bazel-bin/tensorflow/lite/c/libtensorflowlite_c.so`.
4. Copy the library to a location such as `/usr/lib` that can be used by Fluent Bit.
1. Copy the library to a location such as `/usr/lib` that can be used by Fluent Bit.

## Building Fluent Bit with Tensorflow filter plugin

The Tensorflow filter plugin is disabled by default. You must build Fluent Bit with the Tensorflow plugin enabled. In addition, it requires access to Tensorflow Lite header files to compile. Therefore, you must pass the address of the Tensorflow source code on your machine to the [build script](https://github.com/fluent/fluent-bit#build-from-scratch):

```shell
$ ./cmake -DFLB_FILTER_TENSORFLOW=On -DTensorflow_DIR=<AddressOfTensorflowSourceCode> ...
cmake -DFLB_FILTER_TENSORFLOW=On -DTensorflow_DIR=<AddressOfTensorflowSourceCode> ...
```

### Command line

If Tensorflow plugin initializes correctly, it reports successful creation of the interpreter, and prints a summary of model's input and output types and dimensions.
If the Tensorflow plugin initializes correctly, it reports successful creation of the interpreter, and prints a summary of model's input and output types and dimensions.

The command:

```shell
$ ./fluent-bit -i mqtt -p 'tag=mqtt.data' -F tensorflow -m '*' -p 'input_field=image' -p 'model_file=/home/user/model.tflite' -p
fluent-bit -i mqtt -p 'tag=mqtt.data' -F tensorflow -m '*' -p 'input_field=image' -p 'model_file=/home/user/model.tflite' -p
```

produces an output like:
Expand All @@ -75,20 +75,20 @@ service:
flush: 1
daemon: off
log_level: info

pipeline:
inputs:
- name: mqtt
tag: mqtt.data

filters:
- name: tensorflow
match: mqtt.data
input_field: image
model_file: /home/m/model.tflite
include_input_fields: false
normalization_value: 255

outputs:
- name: stdout
match: '*'
Expand Down Expand Up @@ -121,4 +121,4 @@ pipeline:
```

{% endtab %}
{% endtabs %}
{% endtabs %}