Vector events in disk buffer not preserved after pod restart

[viktordebulat]

When restarting pod with disk buffer persistence enabled vector buffer doesn't try to send already got events from kafka but starts from 0 with current consumer, so events are lost. How this issue could be resolved or I missed something?

Use 0.31.0 chart version (the same issue was on 0.26.0 version too, so bump didn't help)

Persistence is enabled:

          persistence:
            enabled: true
            storageClassName: premium-rwo
            size: 10Gi

Typical sink config.

            sinks:  
              clickhouse:
                type: clickhouse
                healthcheck: true
                acknowledgements:
                  enabled: true
                inputs: ["kafka"]
                endpoint: _lorem-ipsum_
                database: _lorem-ipsum_
                table: _lorem-ipsum_
                skip_unknown_fields: true
                auth:
                  strategy: basic
                  user: vector
                  password: _lorem-ipsum_
                buffer:
                  type: disk
                  max_size: 5368709120
                  when_full: block
                batch:
                  max_bytes: 536870912
                  timeout_secs: 5
                request:
                  timeout_secs: 120

P.S. Using Aggregator role with 1 replica

[jszwedko]

Responded over here: vectordotdev/helm-charts#375. Closing this one to keep the discussion focused.