fluentD not parisng logs without a timestap

[milank78git]

This is my current fluentd-1.9.2configuration that parses JSON and standard logs with a timestamp:

<source>
    @id fluentd-containers.log
    @type tail
    path /var/log/containers/*.log
    pos_file /var/log/containers.log.pos
    tag raw.test.*
    read_from_head true
    <parse>
      @type multi_format
      <pattern>
        format json
        time_key time
        time_format %Y-%m-%dT%H:%M:%S.%NZ
      </pattern>
      <pattern>
        format /^(?<time>.+) (?<stream>stdout|stderr) [^ ]* (?<log>.*)$/
        time_format %Y-%m-%dT%H:%M:%S.%N%:z
      </pattern>
    </parse>
  </source>

  <match raw.test.**>
    @id raw.test
    @type detect_exceptions
    remove_tag_prefix raw
    message log
    stream stream
    multiline_flush_interval 5
    max_bytes 500000
    max_lines 1000
  </match>

  <filter **>
    @id filter_concat
    @type concat
    key message
    multiline_end_regexp /\n$/
    separator ""
  </filter>

  <filter test.**>
    @id filter_kubernetes_metadata
    @type kubernetes_metadata
  </filter>

  <filter test.**>
    @id filter_test
    @type grep
    <regexp>
      key log
      pattern ^(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}(?:\.\d{3})?)\s+(ERROR|INFO|WARN|DEBUG|TRACE)\s?\s.*
    </regexp>
  </filter>


  <filter test.**>
    @id filter_severity
    @type parser
    key_name log
    reserve_data true
    <parse>
      @type regexp
      expression ^(?<timestamp>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}(?:.\d{3})?)\s+(?<severity>ERROR|INFO|WARN|DEBUG|TRACE)\s+(?:--- +\[\s*(?<process>[^]]*)\] +(?<class>\S*) +: +(?<syslog_message>.*)|traceId=(?<traceId>[^\s]*)\sspanId=(?<spanId>[^\s]*)\ssampled=(?<sampled>[^\s]*)\s\[(?<class2>[^]]+)\]\s\((?<process2>[^)]+)\)\s(?<syslog_message2>.*))
    </parse>
  </filter>

  <filter test.**>
    @type record_transformer
    enable_ruby
    <record>
      process ${record["process2"] != nil ? record["process2"] : record["process"]}
      class ${record["class2"] != nil ? record["class2"] : record["class"]}
      syslog_message ${record["syslog_message2"] != nil ? record["syslog_message2"] : record["syslog_message"]}
    </record>
    remove_keys process2,class2,syslog_message2
  </filter>

  <filter test.**>
    @id filter_parser
    @type parser
    key_name log
    reserve_data true
    remove_key_name_field true
    <parse>
      @type multi_format
      <pattern>
        format json
      </pattern>
      <pattern>
        format none
      </pattern>
    </parse>
  </filter>

  <match test.**>
    @id elasticsearch
    @type elasticsearch_dynamic

Can you please advise me on setting up the filter that would not ignore multiline logs without a timestamp?

[ERROR] testAll{TestCase}[41]  Time elapsed: 0.059 s  <<< ERROR!
javax.ws.rs.NotFoundException: HTTP 404 Not Found
[ERROR] Test.testAll:225->restClientPut:292 » NotFound HTTP 404 Not Foun...
java.lang.RuntimeException: java.net.SocketTimeoutException: Connect timed out
[INFO] Results:

I've tried to create a filter with regex where add level and message

    <filter test.**>
      @id filter_log_level
      @type parser
      key_name log
      reserve_data true
      <parse>
        @type regexp
        expression /(?<log_level>ERROR|INFO|WARN|DEBUG|TRACE)(?<class_exp>[^]]+)\:(?<exception>[^]]+)/
      </parse>
    </filter>

Please advise me How add configuration for messages

[fujimotos]

Use fluent-plugin-multi-format-parser and defines fallback parser
for those logs.

https://github.com/repeatedly/fluent-plugin-multi-format-parser

[milank78git]

I'm sorry, but I don't understand. I use @type multi_format in first source