KAFKA LOG recovery

[sreejesh-radhakrishnan-db]

Hi

We run our KAFKA on GKE with GCP Persistent Disk as store.

I am doing a kubectl snapshot to get backup of PD (all broker and ZK on daily basis)

Is there any document which states how I can restore KAFKA (broker and ZK) to a given snapshot in the backup?

[scholzj]

AFAIK. there is no kubectl snapshot command. So you are probably using some extension? There is no guide and there is no real guarantee that snapshots of running brokers will work out of the box with a distributed system like Kafka. But I guess you should be able to recreate the PVs and VCs from the backup and then follow this guide: https://strimzi.io/docs/operators/latest/full/configuring.html#cluster-recovery-str

[sreejesh-radhakrishnan-db]

thank you, and really appreciate you highlight the risk . I meant to say k8s snapshot ( but typed kubectl snapshot) sorry.

I will be doing this function on GKE (https://cloud.google.com/kubernetes-engine/docs/how-to/persistent-volumes/volume-snapshots)

I will follow the document link you shared to recovery,

[scholzj]

The problem with snapshots like this is that every disk is snapshotted at a slightly different time with slightly different time. I'm sure it will work in most cases. But I did not find anyone who would be able to confirm that this will work in 100% of cases. So, I do not want to discourage you from using it ... and I do not have any better ideas for backup 🤷 . So yeah, just point out the risks.