The Fluentd parser is not working for label @FLUENT_LOG

[vkamlesh]

I am trying to push fluentd's pods logs to logz.io. Basically, I have installed the fluent-plugin-throttle plugin which push logs whenever any container reaches the log limit. These log lines show in fluentd's logs. The plugin pushes logs in Json but Fluentd add "2022-10-19 16:57:22 +0000 [info]:" prefix in each log lines. I want to capture online the following log lines, the rest of the lines can be discarded.

2022-10-19 15:31:43 +0000 [warn]: #0 rate exceeded group_key=["handler-func-http"] rate_s=117 period_s=60 limit=600 rate_limit_s=10 reset_rate_s=10
2022-10-19 18:48:33 +0000 [info]: #0 [filter_kube_metadata] stats - namespace_cache_size: 13, pod_cache_size: 16, namespace_cache_host_updates: 65, namespace_watch_failures: 4, pod_cache_host_updates: 80, pod_watch_failures: 4, namespace_cache_api_updates: 7, pod_cache_api_updates: 7, id_cache_miss: 7
2022-10-19 18:47:28 +0000 [info]: #0 Timeout flush: kubernetes.var.log.containers.id-provider-d69b85bfb-api_id-provider-829f4e600965fd54875af5d48f535904c7.log:default```



I have tried following the configs.

Type 1:



```<label @FLUENT_LOG>
      <match fluent.*>
         @type stdout
        </match>
         <filter fluent.*>
           @type parser
           key_name log
           reserve_data true
           remove_key_name_field true
             <parse>
              @type regexp
              expression /^(?<time>[^ ]* [^ ]*) (?:(?:[^ ]*) (?:[^ ]*) (?:[^ ]*)) (?:rate[^ ]*) (?<rate_status>(exceeded|back down)) (?:(group_key=))(?<group_key>[^ ]*) (?<message>.*)$/
             </parse>
        </filter>
    </label>

Type 2:

  <match buffer.*.log>
     @type file
      path /var/log/fluentd
  </match>
  <filter buffer.*.log >
    @type grep
    <regexp>
      key message
      pattern /rate/
     </regexp>
    <exclude>
      key message
      pattern /filter_kube_metadata/
    </exclude>
   </filter>
 </label>```
 
 Type 3:
 
 ``` <label @FLUENT_LOG>
   <match buffer.*.log>
     @type file
     path /var/log/fluentd
   </match>
   <filter buffer.*.log >
      @type parser
      key_name message
      reserve_data true
      remove_key_name_field true
      hash_value_field "parsed_nested"
      <parse>
        @type json
      </parse>
   </filter>
</label>```

[fujimotos]

The order of configuration blocks is significant in Fluentd. So if you write:

<match fluent.*>
 @type stdout
</match>
<filter fluent.*>
 @type parser
 ...
</filter>

Fluentd will apply <match> block before <filter>. This is why Fluentd
didn't work as expected in your case.

The solution is simple. You just need to reorder the filter/match blocks as
follows:

<filter fluent.*>
 @type parser
 ...
</filter>
<match fluent.*>
 @type stdout
</match>