Documentation should state that tags need to be defined in `data_points` as well

[jessetan]

Fields that should be sent as tags to InfluxDB (by default only host) need to be in data_points but this is not clear from the documentation:

This does not work:

influxdb {
    host => "localhost"
    send_as_tags => ["somefield"]
}

Output is logstash VALUE TIME.

This is the correct way:

influxdb {
    host => "localhost"
    send_as_tags => ["somefield"]
    data_points => {
        "somefield" => "%{[somefield]}"
    }
}

Output: logstash,somefield=SOMEFIELD VALUE TIME

Another way to do this is to set use_event_fields_for_data_points to true (default is false), which will automatically use somefield (and all other non-excluded fields) to InfluxDB

[mikelaws]

Great point! It's definitely not clear in the documentation that any field in the send_as_tags list must also be sent to InfluxDB, either explicitly in data_points or implicitly through use_event_fields_for_data_points. From my experience, if the tag field is NOT present as a data point, this silently fails, which makes the problem a bit harder to diagnose.

From a usability perspective, would it make any sense to have the plugin automatically include fields listed in send_as_tags in data_points as well, or would that introduce more confusion or problems down the road?

[udpno]

I burnt myself on this one as well, would have been nice with some documentation (I guess the answer is to write it myself... perhaps I can find some time soon)

[jessetan]

@mikelaws implicitly assuming send_as_tags means inclusion in data_points sounds ok to me, but rather than have a long discussion about possible dangers of implicit assumptions, I've filed a PR for a simple documentation change that will at least clarify this.
Comments welcome.

[ydp]

ALSO I found an error in Documents data_points, on this page: https://www.elastic.co/guide/en/logstash/current/plugins-outputs-influxdb.html#plugins-outputs-influxdb-data_points
it writes:

Hash of key/value pairs representing data points to send to the named database Example: {'column1' => 'value1', 'column2' => 'value2'}

BUT, actually, hash value should be like below:

match => {
"field1" => "value1"
"field2" => "value2"
...
}

[ydp]

also cannot set Number as tags

[berghauz]

It's some sort of bug, coz tags deleted if not set implicitly in data_points, coz, again, pipeline do things in strange manner:

1. removing all excluded fields
2. adding tags (witch should be a fileds before it deleted)
3. addind @measurement

I have try to commit fix #51

In my case i have result like this:

> show series
key
IPPOOL,bulkstats=true,ggsn=true,host=msk1-ggsn01,name=mtt-1,startaddr=10.8.0.0,state=G,vpnname=Gi
IPPOOL,bulkstats=true,ggsn=true,host=msk1-ggsn01,name=mtt-2,startaddr=10.8.128.0,state=G,vpnname=Gi
IPPOOL,bulkstats=true,ggsn=true,host=msk1-ggsn01,name=test,startaddr=10.7.9.0,state=G,vpnname=Gi

> show field keys
name: IPPOOL
------------
fieldKey        fieldType
free            float
groupname-ipv4free  float
groupname-ipv4hold  float
groupname-ipv4rele  float
groupname-ipv4used  float
groupname-ipv6free  float
groupname-ipv6hold  float
groupname-ipv6rels  float
groupname-ipv6used  float
hold            float
priority        float
release         float
used            float
vpnid           float

no useless extra field, especially of String type.

[cl0udgeek]

I'm having an issue with data points as well...I can't get my "commit_time" to insert as the "time" column....my output looks like this:

influxdb { host => "10.204.95.88" db => "metrics" use_event_fields_for_data_points => "true" retention_policy => "one_week" data_points => { "time" => "%{commit_time}" } measurement => "offsets" }

I've also tried this too data_points => { "time" => "%{[commit_time]}" } but that didn't work either...