Skip to content

Pipeline: output: influxdb: style #1945

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jul 17, 2025
Merged

Pipeline: output: influxdb: style #1945

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
6e1a439
Pipeline: output: influxdb: style
esmerel Jul 17, 2025
529afb1
Apply suggestions from code review
esmerel Jul 17, 2025
ff89913
Apply suggestions from code review
esmerel Jul 17, 2025
3088be6
Update influxdb.md
esmerel Jul 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
210 changes: 107 additions & 103 deletions pipeline/outputs/influxdb.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,51 +1,53 @@
# InfluxDB

The **influxdb** output plugin, allows to flush your records into a [InfluxDB](https://www.influxdata.com/time-series-platform/influxdb/) time series database. The following instructions assumes that you have a fully operational InfluxDB service running in your system.
The _InfluxDB_ output plugin lets you flush your records into a [InfluxDB](https://www.influxdata.com/time-series-platform/influxdb/) time series database. The following instructions assumes that you have an operational InfluxDB service running in your system.

## Configuration Parameters
## Configuration parameters

| Key | Description | default |
This plugin supports the following parameters:

| Key | Description | Default |
| :--- | :--- | :--- |
| Host | IP address or hostname of the target InfluxDB service | 127.0.0.1 |
| Port | TCP port of the target InfluxDB service | 8086 |
| Database | InfluxDB database name where records will be inserted | fluentbit |
| Bucket | InfluxDB bucket name where records will be inserted - if specified, `database` is ignored and v2 of API is used | |
| Org | InfluxDB organization name where the bucket is \(v2 only\) | fluent |
| Sequence\_Tag | The name of the tag whose value is incremented for the consecutive simultaneous events. | \_seq |
| HTTP\_User | Optional username for HTTP Basic Authentication | |
| HTTP\_Passwd | Password for user defined in HTTP\_User | |
| HTTP\_Token | Authentication token used with InfluDB v2 - if specified, both HTTP\_User and HTTP\_Passwd are ignored | |
| HTTP\_Header | Add a HTTP header key/value pair. Multiple headers can be set | |
| Tag\_Keys | Space separated list of keys that needs to be tagged | |
| Auto\_Tags | Automatically tag keys where value is _string_. This option takes a boolean value: True/False, On/Off. | Off |
| Uri | Custom URI endpoint | |
| Add\_Integer\_Suffix | Use integer type of [influxdb's line protocol](https://docs.influxdata.com/influxdb/v1/write_protocols/line_protocol_reference/). This option takes a boolean value: `True|False`, `On|Off`. | `Off` |
| Workers | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `0` |
| `Host` | IP address or hostname of the target InfluxDB service. | `127.0.0.1` |
| `Port` | TCP port of the target InfluxDB service. | `8086` |
| `Database` | InfluxDB database name where records will be inserted | `fluentbit` |
| `Bucket` | InfluxDB bucket name where records will be inserted. If specified, `database` is ignored and v2 of API is used. | _none_ |
| `Org` | InfluxDB organization name where the bucket is (v2 only). | `fluent` |
| `Sequence_Tag` | The name of the tag whose value is incremented for the consecutive simultaneous events. | `_seq` |
| `HTTP_User` | Optional username for HTTP Basic Authentication. | _none_ |
| `HTTP_Passwd` | Password for user defined in `HTTP_User`. | _none_ |
| `HTTP_Token` | Authentication token used with InfluxDB v2. If specified, both `HTTP_User` and `HTTP_Passwd` are ignored. | _none_ |
| `HTTP_Header` | Add a HTTP header key/value pair. Multiple headers can be set. | _none_ |
| `Tag_Keys` | Space separated list of keys that needs to be tagged | _none_ |
| `Auto_Tags` | Automatically tag keys where value is `string`. This option takes a Boolean value: `True`/`False`, `On`/`Off`. | `Off` |
| `Uri` | Custom URI endpoint. | _none_ |
| `Add_Integer_Suffix` | Use integer type of [InfluxDB's line protocol](https://docs.influxdata.com/influxdb/v1/write_protocols/line_protocol_reference/). This option takes a Boolean value: `True`/`False`, `On`/`Off`. | `Off` |
| `Workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `0` |

### TLS / SSL

The InfluxDB output plugin supports TLS/SSL.
For more details about the properties available and general configuration, see [TLS/SSL](../../administration/transport-security.md).

## Getting Started
## Get started

In order to start inserting records into an InfluxDB service, you can run the plugin from the command line or through the configuration file:
To start inserting records into an InfluxDB service, you can run the plugin from the command line or through the configuration file.

### Command Line
### Command line

The **influxdb** plugin, can read the parameters from the command line in two ways, through the **-p** argument \(property\) or setting them directly through the service URI. The URI format is the following:
The InfluxDB plugin can read the parameters from the command line through the `-p` argument (property) or by setting them directly through the service URI. The URI format is the following:

```text
influxdb://host:port
```

Using the format specified, you could start Fluent Bit through:
Using the format specified, you could start Fluent Bit:

```shell
fluent-bit -i cpu -t cpu -o influxdb://127.0.0.1:8086 -m '*'
```

### Configuration File
### Configuration file

In your main configuration file append the following:

Expand All @@ -57,7 +59,7 @@ pipeline:
inputs:
- name: cpu
tag: cpu

outputs:
- name: influxdb
match: '*'
Expand Down Expand Up @@ -101,7 +103,7 @@ pipeline:
tag: apache.access
parser: apache2
path: /var/log/apache2/access.log

outputs:
- name: influxdb
match: '*'
Expand Down Expand Up @@ -137,9 +139,9 @@ pipeline:
{% endtab %}
{% endtabs %}

With **Auto\_Tags=On** in this example cause error, because every parsed field value type is _string_. Best usage of this option in metrics like record where one or more field value is not _string_ typed.
`Auto_Tags=On` in this example causes an error, because every parsed field value type is `string`. The best usage of this option in metrics like record where one or more field value isn't `string` typed.

Basic example of `Tags_List_Key` usage:
A basic example of `Tags_List_Key` usage:

{% tabs %}
{% tab title="fluent-bit.yaml" %}
Expand All @@ -150,7 +152,7 @@ pipeline:
- name: dummy
# tagged fields: level, ID, businessObjectID, status
dummy: '{"msg": "Transfer completed", "level": "info", "ID": "1234", "businessObjectID": "qwerty", "status": "OK", "tags": ["ID", "businessObjectID"]}'

outputs:
- name: influxdb
match: '*'
Expand Down Expand Up @@ -192,101 +194,103 @@ pipeline:

### Testing

Before to start Fluent Bit, make sure the target database exists on InfluxDB, using the above example, we will insert the data into a _fluentbit_ database.
Before starting Fluent Bit, make sure the target database exists on InfluxDB. Using the previous example, insert the data into a `fluentbit` database.

#### 1. Create database
1. Log into the InfluxDB console:

Log into InfluxDB console:
```shell
influx
```

```shell
$ influx
Which should return a result like:

Visit https://enterprise.influxdata.com to register for updates, InfluxDB server management, and monitoring.
Connected to http://localhost:8086 version 1.1.0
InfluxDB shell version: 1.1.0
>
```
```text
Visit https://enterprise.influxdata.com to register for updates, InfluxDB server management, and monitoring.
Connected to http://localhost:8086 version 1.1.0
InfluxDB shell version: 1.1.0
>
```

Create the database:
1. Create the database:

```text
> create database fluentbit
>
```
```text
> create database fluentbit
>
```

Check the database exists:
1. Check that the database exists:

```text
> show databases
name: databases
name
----
_internal
fluentbit

>
```
```text
> show databases
name: databases
name
----
_internal
fluentbit

#### 2. Run Fluent Bit
>
```

The following command will gather CPU metrics from the system and send the data to InfluxDB database every five seconds:
1. Run Fluent Bit:

```shell
fluent-bit -i cpu -t cpu -o influxdb -m '*'
```
The following command will gather CPU metrics from the system and send the data to InfluxDB database every five seconds:

Note that all records coming from the _cpu_ input plugin, have a tag _cpu_, this tag is used to generate the measurement in InfluxDB
```shell
fluent-bit -i cpu -t cpu -o influxdb -m '*'
```

#### 3. Query the data
All records coming from the `cpu` input plugin have a tag `cpu`. This tag is used to generate the measurement in InfluxDB.

From InfluxDB console, choose your database:
1. Query the data:

```text
> use fluentbit
Using database fluentbit
```
From InfluxDB console, choose your database:

Now query some specific fields:
```text
> use fluentbit
Using database fluentbit
```

```text
> SELECT cpu_p, system_p, user_p FROM cpu
name: cpu
time cpu_p system_p user_p
---- ----- -------- ------
1481132860000000000 2.75 0.5 2.25
1481132861000000000 2 0.5 1.5
1481132862000000000 4.75 1.5 3.25
1481132863000000000 6.75 1.25 5.5
1481132864000000000 11.25 3.75 7.5
```
1. query some specific fields:

The CPU input plugin gather more metrics per CPU core, in the above example we just selected three specific metrics. The following query will give a full result:
```text
> SELECT cpu_p, system_p, user_p FROM cpu
name: cpu
time cpu_p system_p user_p
---- ----- -------- ------
1481132860000000000 2.75 0.5 2.25
1481132861000000000 2 0.5 1.5
1481132862000000000 4.75 1.5 3.25
1481132863000000000 6.75 1.25 5.5
1481132864000000000 11.25 3.75 7.5
```

```text
> SELECT * FROM cpu
```
The CPU input plugin gathers more metrics per CPU core. This example selected three specific metrics. The following query will give a full result:

#### 4. View tags
```text
> SELECT * FROM cpu
```

Query tagged keys:
1. View tags:

```text
> SHOW TAG KEYS ON fluentbit FROM "apache.access"
name: apache.access
tagKey
------
_seq
method
path
```
Query tagged keys:

And now query _method_ key values:
```text
> SHOW TAG KEYS ON fluentbit FROM "apache.access"
name: apache.access
tagKey
------
_seq
method
path
```

```text
> SHOW TAG VALUES ON fluentbit FROM "apache.access" WITH KEY = "method"
name: apache.access
key value
--- -----
method "MATCH"
method "POST"
```
1. Query `method` key values:

```text
> SHOW TAG VALUES ON fluentbit FROM "apache.access" WITH KEY = "method"
name: apache.access
key value
--- -----
method "MATCH"
method "POST"
```