-
Notifications
You must be signed in to change notification settings - Fork 12
Open
Description
Framework
filebeat --> kafka --> fluentd --> elasticsearch
Describe the problem
filebeat comes with an array of tags, but fluentd doesn't filter out!
The filter in logstash is
mutate {
add_field => {
"city" => "%{[tags][0]}"
"ip" => "%{[tags][1]}"
}
}
filebeat output configuration
output.kafka:
hosts: ["120.0.0.1:9092"]
topic: test
required_acks: 1
tags: ["xinyi", "19.18.14.168"]
Fluentd Filter Configuration
<filter **>
@type record_transformer
enable_ruby true
auto_typecast true
<record>
city %{[tags][0]}
ip %{[tags][1]}
</record>
</filter>
result
The city and IP fields inserted into es are %{[tags][0]} and %{[tags][1}
So, filtering didn't work, looking forward to answer, thanks
Metadata
Metadata
Assignees
Labels
No labels