diff --git a/pipeline/outputs/influxdb.md b/pipeline/outputs/influxdb.md index 3b654417d..6ebde8910 100644 --- a/pipeline/outputs/influxdb.md +++ b/pipeline/outputs/influxdb.md @@ -1,51 +1,53 @@ # InfluxDB -The **influxdb** output plugin, allows to flush your records into a [InfluxDB](https://www.influxdata.com/time-series-platform/influxdb/) time series database. The following instructions assumes that you have a fully operational InfluxDB service running in your system. +The _InfluxDB_ output plugin lets you flush your records into a [InfluxDB](https://www.influxdata.com/time-series-platform/influxdb/) time series database. The following instructions assume that you have an operational InfluxDB service running in your system. -## Configuration Parameters +## Configuration parameters -| Key | Description | default | +This plugin supports the following parameters: + +| Key | Description | Default | | :--- | :--- | :--- | -| Host | IP address or hostname of the target InfluxDB service | 127.0.0.1 | -| Port | TCP port of the target InfluxDB service | 8086 | -| Database | InfluxDB database name where records will be inserted | fluentbit | -| Bucket | InfluxDB bucket name where records will be inserted - if specified, `database` is ignored and v2 of API is used | | -| Org | InfluxDB organization name where the bucket is \(v2 only\) | fluent | -| Sequence\_Tag | The name of the tag whose value is incremented for the consecutive simultaneous events. | \_seq | -| HTTP\_User | Optional username for HTTP Basic Authentication | | -| HTTP\_Passwd | Password for user defined in HTTP\_User | | -| HTTP\_Token | Authentication token used with InfluDB v2 - if specified, both HTTP\_User and HTTP\_Passwd are ignored | | -| HTTP\_Header | Add a HTTP header key/value pair. Multiple headers can be set | | -| Tag\_Keys | Space separated list of keys that needs to be tagged | | -| Auto\_Tags | Automatically tag keys where value is _string_. This option takes a boolean value: True/False, On/Off. | Off | -| Uri | Custom URI endpoint | | -| Add\_Integer\_Suffix | Use integer type of [influxdb's line protocol](https://docs.influxdata.com/influxdb/v1/write_protocols/line_protocol_reference/). This option takes a boolean value: `True|False`, `On|Off`. | `Off` | -| Workers | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `0` | +| `Host` | IP address or hostname of the target InfluxDB service. | `127.0.0.1` | +| `Port` | TCP port of the target InfluxDB service. | `8086` | +| `Database` | InfluxDB database name where records will be inserted. | `fluentbit` | +| `Bucket` | InfluxDB bucket name where records will be inserted. If specified, `database` is ignored and v2 of API is used. | _none_ | +| `Org` | InfluxDB organization name where the bucket is (v2 only). | `fluent` | +| `Sequence_Tag` | The name of the tag whose value is incremented for the consecutive simultaneous events. | `_seq` | +| `HTTP_User` | Optional username for HTTP Basic Authentication. | _none_ | +| `HTTP_Passwd` | Password for user defined in `HTTP_User`. | _none_ | +| `HTTP_Token` | Authentication token used with InfluxDB v2. If specified, both `HTTP_User` and `HTTP_Passwd` are ignored. | _none_ | +| `HTTP_Header` | Add a HTTP header key/value pair. Multiple headers can be set. | _none_ | +| `Tag_Keys` | Space separated list of keys that needs to be tagged. | _none_ | +| `Auto_Tags` | Automatically tag keys where value is `string`. This option takes a Boolean value: `True`/`False`, `On`/`Off`. | `Off` | +| `Uri` | Custom URI endpoint. | _none_ | +| `Add_Integer_Suffix` | Use integer type of [InfluxDB's line protocol](https://docs.influxdata.com/influxdb/v1/write_protocols/line_protocol_reference/). This option takes a Boolean value: `True`/`False`, `On`/`Off`. | `Off` | +| `Workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `0` | ### TLS / SSL The InfluxDB output plugin supports TLS/SSL. For more details about the properties available and general configuration, see [TLS/SSL](../../administration/transport-security.md). -## Getting Started +## Get started -In order to start inserting records into an InfluxDB service, you can run the plugin from the command line or through the configuration file: +To start inserting records into an InfluxDB service, you can run the plugin from the command line or through the configuration file. -### Command Line +### Command line -The **influxdb** plugin, can read the parameters from the command line in two ways, through the **-p** argument \(property\) or setting them directly through the service URI. The URI format is the following: +The InfluxDB plugin can read the parameters from the command line through the `-p` argument (property) or by setting them directly through the service URI. The URI format is the following: ```text influxdb://host:port ``` -Using the format specified, you could start Fluent Bit through: +Using the format specified, you could start Fluent Bit: ```shell fluent-bit -i cpu -t cpu -o influxdb://127.0.0.1:8086 -m '*' ``` -### Configuration File +### Configuration file In your main configuration file append the following: @@ -57,7 +59,7 @@ pipeline: inputs: - name: cpu tag: cpu - + outputs: - name: influxdb match: '*' @@ -101,7 +103,7 @@ pipeline: tag: apache.access parser: apache2 path: /var/log/apache2/access.log - + outputs: - name: influxdb match: '*' @@ -137,9 +139,9 @@ pipeline: {% endtab %} {% endtabs %} -With **Auto\_Tags=On** in this example cause error, because every parsed field value type is _string_. Best usage of this option in metrics like record where one or more field value is not _string_ typed. +`Auto_Tags=On` in this example causes an error, because every parsed field value type is `string`. The best usage of this option in metrics like record where one or more field value isn't `string` typed. -Basic example of `Tags_List_Key` usage: +A basic example of `Tags_List_Key` usage: {% tabs %} {% tab title="fluent-bit.yaml" %} @@ -150,7 +152,7 @@ pipeline: - name: dummy # tagged fields: level, ID, businessObjectID, status dummy: '{"msg": "Transfer completed", "level": "info", "ID": "1234", "businessObjectID": "qwerty", "status": "OK", "tags": ["ID", "businessObjectID"]}' - + outputs: - name: influxdb match: '*' @@ -192,101 +194,101 @@ pipeline: ### Testing -Before to start Fluent Bit, make sure the target database exists on InfluxDB, using the above example, we will insert the data into a _fluentbit_ database. +Before starting Fluent Bit, ensure the target database exists on InfluxDB. Using the previous example, insert the data into a `fluentbit` database. -#### 1. Create database +1. Log into the InfluxDB console: -Log into InfluxDB console: + ```shell + influx + ``` -```shell -$ influx + Which should return a result like: -Visit https://enterprise.influxdata.com to register for updates, InfluxDB server management, and monitoring. -Connected to http://localhost:8086 version 1.1.0 -InfluxDB shell version: 1.1.0 -> -``` + ```text + Visit https://enterprise.influxdata.com to register for updates, InfluxDB server management, and monitoring. + Connected to http://localhost:8086 version 1.1.0 + InfluxDB shell version: 1.1.0 + > + ``` -Create the database: +1. Create the database: -```text -> create database fluentbit -> -``` + ```text + > create database fluentbit + > + ``` -Check the database exists: +1. Check that the database exists: -```text -> show databases -name: databases -name ----- -_internal -fluentbit - -> -``` + ```text + > show databases + name: databases + name + ---- + _internal + fluentbit -#### 2. Run Fluent Bit + > + ``` -The following command will gather CPU metrics from the system and send the data to InfluxDB database every five seconds: +1. Run Fluent Bit: -```shell -fluent-bit -i cpu -t cpu -o influxdb -m '*' -``` + The following command will gather CPU metrics from the system and send the data to InfluxDB database every five seconds: -Note that all records coming from the _cpu_ input plugin, have a tag _cpu_, this tag is used to generate the measurement in InfluxDB + ```shell + fluent-bit -i cpu -t cpu -o influxdb -m '*' + ``` -#### 3. Query the data + All records coming from the `cpu` input plugin have a tag `cpu`. This tag is used to generate the measurement in InfluxDB. -From InfluxDB console, choose your database: +1. Query the data: -```text -> use fluentbit -Using database fluentbit -``` + From InfluxDB console, choose your database: -Now query some specific fields: + ```text + > use fluentbit + Using database fluentbit + ``` -```text -> SELECT cpu_p, system_p, user_p FROM cpu -name: cpu -time cpu_p system_p user_p ----- ----- -------- ------ -1481132860000000000 2.75 0.5 2.25 -1481132861000000000 2 0.5 1.5 -1481132862000000000 4.75 1.5 3.25 -1481132863000000000 6.75 1.25 5.5 -1481132864000000000 11.25 3.75 7.5 -``` +1. Query some specific fields: -The CPU input plugin gather more metrics per CPU core, in the above example we just selected three specific metrics. The following query will give a full result: + ```text + > SELECT cpu_p, system_p, user_p FROM cpu + name: cpu + time cpu_p system_p user_p + ---- ----- -------- ------ + 1481132860000000000 2.75 0.5 2.25 + 1481132861000000000 2 0.5 1.5 + 1481132862000000000 4.75 1.5 3.25 + 1481132863000000000 6.75 1.25 5.5 + 1481132864000000000 11.25 3.75 7.5 + ``` -```text -> SELECT * FROM cpu -``` + The CPU input plugin gathers more metrics per CPU core. This example selected three specific metrics. The following query will give a full result: -#### 4. View tags + ```text + > SELECT * FROM cpu + ``` -Query tagged keys: +1. View tags by querying tagged keys: -```text -> SHOW TAG KEYS ON fluentbit FROM "apache.access" -name: apache.access -tagKey ------- -_seq -method -path -``` + ```text + > SHOW TAG KEYS ON fluentbit FROM "apache.access" + name: apache.access + tagKey + ------ + _seq + method + path + ``` -And now query _method_ key values: +1. Query `method` key values: -```text -> SHOW TAG VALUES ON fluentbit FROM "apache.access" WITH KEY = "method" -name: apache.access -key value ---- ----- -method "MATCH" -method "POST" -``` \ No newline at end of file + ```text + > SHOW TAG VALUES ON fluentbit FROM "apache.access" WITH KEY = "method" + name: apache.access + key value + --- ----- + method "MATCH" + method "POST" + ```