Skip to content

out_influxdb: documentation for tag prefix stripping #1468

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

out_influxdb: documentation for tag prefix stripping #1468

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
cd45857
out_influxdb: add documentation and example use of strip_prefix
Oct 1, 2024
4bff80f
Add comment about bucket versus measurement naming
Oct 1, 2024
a391c5e
Merge branch 'master' into influxdb-strip-tag-prefix
esmerel Oct 22, 2025
ce52150
Apply suggestions from code review
esmerel Oct 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions pipeline/outputs/influxdb.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ This plugin supports the following parameters:
| `Bucket` | InfluxDB bucket name where records will be inserted. If specified, `database` is ignored and v2 of API is used. | _none_ |
| `Org` | InfluxDB organization name where the bucket is (v2 only). | `fluent` |
| `Sequence_Tag` | The name of the tag whose value is incremented for the consecutive simultaneous events. | `_seq` |
| `Strip_Prefix` | String to be stripped from the front of `tag` when writing InfluxDB measurement names. | _none_ |
Copy link
Contributor

@patrick-stephens patrick-stephens Nov 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if the prefix is longer than the tag, not present, etc.?

| `HTTP_User` | Optional username for HTTP Basic Authentication. | _none_ |
| `HTTP_Passwd` | Password for user defined in `HTTP_User`. | _none_ |
| `HTTP_Token` | Authentication token used with InfluxDB v2. If specified, both `HTTP_User` and `HTTP_Passwd` are ignored. | _none_ |
Expand Down Expand Up @@ -191,6 +192,43 @@ pipeline:
{% endtab %}
{% endtabs %}

### Prefix stripping

When collecting data from many inputs into many buckets, it can be helpful to remove a common prefix using `Strip_prefix`.

```python
Copy link
Contributor

@patrick-stephens patrick-stephens Nov 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should be using the recently made default approach of having both configs (YAML and legacy) in place.

[INPUT]
Name cpu
Tag cpu.one

[INPUT]
Name cpu
Tag cpu.two

[INPUT]
Name cpu
Tag gpu.one

[INPUT]
Name cpu
Tag gpu.two

[OUTPUT]
Name influxdb
Match cpu*
Bucket cpubucket
Strip_prefix cpu.

[OUTPUT]
Name influxdb
Match gpu*
Bucket gpubucket
Strip_prefix gpu.
```

This will result in the buckets `cpubucket` and `gpubucket` each containing two measurement streams named `one` and `two`. Without prefix stripping, the measurement names would be `cpu.one` and `cpu.two` (stored in `cpubucket`), and `gpu.one`, `gpu.two` (stored in `gpubucket`).


### Testing

Before starting Fluent Bit, ensure the target database exists on InfluxDB. Using the previous example, insert the data into a `fluentbit` database.
Expand Down