Kafka with PersistentClaimStorage

[teimyBr]

Hi,

we are Running Strimzi Kafka with 0.26.1 in Production with Storage PersistentClaimStorage. (On Premise and Vshere under it)

We have adding a second Storage (PersistentClaimStorage) under each Kafka Node. Because Our Storage Class doesn‘t Support volume resizing. And the Storage Class with This Option has a lot of Bug with Vshere.

It Looks like the data is always under Storage0 Kafka 0 and dont get to Storage1 Kafka 0. So whats Happen when storage0 ist Full get the new load then to the Storage1 or the Kafka Cluster will Break because volumes are full?
If yes is there any way to get the data from storage0 to Storage1 ?

[scholzj]

So, just to make sure I got you right -> you use the JBOD storage and you sucessfully added the second disk (i.e. if you ece into the broker pod and do df -h you can see it)? But you are wondering how to move the data to the new disk. Correct?

Kafka does not do this automatically -> you have to do this your self. In 0.26.1, you will need to use the kafka-reassign-partitions.sh tool. (check the section Partition reassignment between JBOD volumes in https://strimzi.io/docs/operators/latest/full/configuring.html#con-partition-reassignment-str)

If you upgrade to 0.28, you would be able to also use the Intra-broker disk balancing using Cruise Cotnrol (see https://strimzi.io/docs/operators/latest/full/configuring.html#con-rebalance-str). But that is new in 0.28.0.

[teimyBr]

Thanks for the answer i will check this out.

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: sinapse-standard
parameters:
  diskformat: thin
provisioner: kubernetes.io/vsphere-volume
reclaimPolicy: Delete
volumeBindingMode: Immediate

Im using this storage class

[teimyBr]

And have to of them under one broker

[teimyBr]

Sry yes this JBOD.

[scholzj]

Right, so if it is JBOD, that is great, you can follow the suggestions mentioned above.

[teimyBr]

Yes we test this in dev and this works :D Thanks