-
I'm trying to parse and extract keys from a JSON object <source>
@type cloudwatch_logs
@id star_dev_postgresql
tag star_dev_postgresql
fetch_interval 60
log_group_name /aws/rds/cluster/star-dev/postgresql
log_stream_name star-dev
use_log_stream_name_prefix true
include_metadata true
add_log_group_name true
log_group_name_key log_group
use_log_group_name_prefix false
format none
<storage>
@type local
</storage>
<web_identity_credentials>
role_arn "#{ENV['AWS_ROLE_ARN']}"
role_session_name fluentd-cw-star-dev-rds
web_identity_token_file "#{ENV['AWS_WEB_IDENTITY_TOKEN_FILE']}"
</web_identity_credentials>
</source>
<filter start_dev_postgresql>
@type parser
@id parse_cw_meta
key_name metadata
reserve_data true
remove_key_name_field true
# emit_invalid_record_to_error false
<parse>
@type json
</parse>
</filter>
The error I'm getting:
I'm not sure what kind of pattern is not matched? "metadata"=>{"log_stream_name"=>"star-dev-2.1", "log_group_name"=>"/aws/rds/cluster/star-dev/postgresql"} Can't find answer in docs: How do you extract fields from simple JSON objects without crazy regex/grok manipulations? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
"metadata"=>{"log_stream_name"=>"star-dev-2.1", ...
==== So you have to parse nothing, because the message is already parsed! |
Beta Was this translation helpful? Give feedback.
metadata
is not JSON, but a Ruby object. You can tell that bya pair of key and value is separated by
=>
, not:
.So you have to parse nothing, because the message is already parsed!