Skip to content

How to get the data in the array by fluent? #23

@rljie

Description

@rljie

Framework
filebeat --> kafka --> fluentd --> elasticsearch

Describe the problem
filebeat comes with an array of tags, but fluentd doesn't filter out!
The filter in logstash is

mutate {
     add_field =>  {
           "city" => "%{[tags][0]}"
           "ip" => "%{[tags][1]}"
     }
}

filebeat output configuration

output.kafka:
   hosts: ["120.0.0.1:9092"]
   topic: test
   required_acks: 1
tags: ["xinyi", "19.18.14.168"]
Fluentd Filter Configuration
<filter **>
   @type record_transformer
   enable_ruby true
   auto_typecast true
   <record>
      city %{[tags][0]}
      ip %{[tags][1]}
   </record>
</filter>

result
The city and IP fields inserted into es are %{[tags][0]} and %{[tags][1}
So, filtering didn't work, looking forward to answer, thanks

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions