-
Notifications
You must be signed in to change notification settings - Fork 0
Description
Proposal:
Is it possible to enhance the InfluxDB output plugin (https://github.com/influxdata/influxdb-plugin-fluent) to allow the tag_keys and field_keys settings to accept arrays specified within the JSON structure?
Current behavior:
Currently, only static configuration is possible.
Desired behavior:
The current static enumeration significantly limits flexibility, especially when dealing with data sources where tags and fields vary dynamically. While the measurement can be set dynamically, its utility is constrained since different measurements typically have distinct fields and tags.
For instance, consider a scenario where a JSON key contains an array of tags, and this key is referenced in the configuration:
{
"name": "apiserver_request_sli_duration_seconds_bucket",
"timestamp": "2025-02-17T14:49:04Z",
"value": "88808",
"labels_keys": [
"name",
"apiserver",
"component",
"endpoint",
"group",
"instance",
"job",
"le",
"namespace",
"prometheus",
"prometheus_replica",
"resource",
"scope",
"service",
"verb",
"version",
"topic"
],
"name": "apiserver_request_sli_duration_seconds_bucket",
"apiserver": "kube-apiserver",
"component": "apiserver",
"endpoint": "https",
"group": "machineconfiguration.openshift.io",
"instance": "1.1.1.12:6443",
"job": "apiserver",
"le": "3",
"namespace": "default",
"prometheus": "openshift-monitoring/k8s",
"prometheus_replica": "prometheus-k8s-0",
"resource": "machineconfigs",
"scope": "resource",
"service": "kubernetes",
"verb": "GET",
"version": "v1",
"topic": "ch-ocp-cw01-prod-ocp-metrics"
}
In this example, the labels_keys array enumerates the desired tags. Enhancing the plugin to interpret such arrays within the JSON payload would provide greater flexibility in handling dynamic data sources.
This feature request aligns with discussions in the plugin's GitHub repository, where users have expressed the need for dynamically setting tag_keys based on incoming data. Implementing this capability would significantly enhance the plugin's versatility in diverse logging environments.
Alternatives considered:
Describe other solutions or features you considered.
Use case:
Why is this important (helps with prioritizing requests)?