fluentd still buffers the old chunks even with below configuration where the usecase does not need any buffering

[shilpa87-khushi]

Hello,

We are using below configuration in fluentd with kafka output plugin
Usecase demands no buffering as in absence of any events to kafka consumer/server it should be detected as anomoly(or connection issue) and further actions will be set up based on this
currently with below configuration when kafka is down for even long time we are still getting data of previous days (not all of them but some in between events) once kafka is up .

Please help in configuring the right configuration for the above usecase where we do not need any persistancy .
requirement is no persistance, no buffering , flush immediately as u get the data and do not retry / even if u retry it should be very quick and discard the event when failed .

<match hbk>
@type copy
<store ignore_error>
<buffer>
  chunk_limit_size 205
  total_limit_size 205
  flush_interval 1s
  retry_max_times 0
  retry_forever false
  max_send_retries 0
  overflow_action drop_oldest_chunk
  @log_level error
</buffer>
@type kafka2
get_kafka_client_log false
@log_level debug
topic_key topic
default_topic 199wobuffer
max_send_limit_bytes 1000000
#ssl config
#ssl_ca_cert /etc/fluent/certs/fm_kafka_ca_cert.pem
#ssl_client_cert /etc/fluent/certs/fm_kafka_client_cert.pem
#ssl_client_cert_key /etc/fluent/certs/fm_kafka_client_cert_key.pem
#ssl_client_cert_key_password
#ssl_verify_hostname true
#ssl config end
brokers ["xxx.xxx.xxx.xxx:9092"]
<format>
    @type json
</format>
use_event_time false
required_acks 1
request_timeout 30s
reconnect_on_error true
</store>
</match>

[Watson1978]

How about using out_kafka or out_rdkafka instead of out_kafka2?
Since the plugins do not appear to require settings in the buffer section.