Filebeat High memory usage

[guogl1988]

Hey, We Use Filebeat Version 3.17.5

The Filebeat collects logs and sends to Kafka；

The config is
filebeat.inputs:

• type: log
  path: [cd /xxxxx]
  fields:
  ip: "xxx.xxx.xxx.xx"
  system: "xxxx"
  logType: 'link'
  logSubType: -1
  fields_under_root: true
  include_lines: [ERROR]
  exclude_lines:[INFO]
  monitorining.enabled: true
  output.kafka.enabled: true
  output.kafka.hosts: ['xxx.xx.xx.xx:9092']
  output.kafka.topic: '%{[system]}-%{[logType]}%{[logSubType]}'
  output.kafka.compression: gzip
  output.kafka.max_message_types: 100000
  output.kafka.required_acks: 1
  logging.to_files: true
  close_inactive: 5m
  close_timeout: 2m
  clean_inactive: 12h
  close_older: 1m
  close_renamed: true
  close_removed: true
  tail_files: true
  max_procs: 1

Filebeat's normally memory usage is around 400MB

However, there was a unexcpected surge in memory consumption one day.

the Filebeat's memery used about:
15:25:55 436M
15:26:26 776M
15:26:58 10.6G
15:27:29 12.7G
15:28:01 15.9G
15:28:33 19.2G
....
15:38:20 55.2G
then the Filebeat oom killed

To begin with, the amount of logs is staying below 1 GB
and there are no abnormal log messages (including multiline log entries) in the logs.

Since no logs were printed and the issue cannot be reproduced at the moment, please assist in analyzing it.

[botelastic]

This issue doesn't have a Team:<team> label.