Volume Node Affinity Conflict

[brunooliveiramac]

I'm facing an issue when using persistent volumes in google cloud Kubernetes.
Basically, I do have a cluster with a Kafka node, and this node has 3 zones:

• us-central1-f
• us-central1-b
• us-central1-c

I do have node affinity so my strmzi Kafka clusters are scheduled on the Kafka node.
The problem is, one of my volumes is being created by strimzi in a different node, other than Kafka kubernetes node, and my kafka cluster is not being able to schedule, complaining about a volume affinity conflict.

[scholzj]

What is the volumeBindingMode in your Storage class? Is it WaitForFirstConsumer?

[brunooliveiramac]

Hey scholzj
Its immediate

You mean that we should use a WaitForFirstConsumer, so the binding and provisioning of a PV to a PVC will be delayed until a pod using the PVC is created. This way it would ensure that the PVC/PV is on the correct node?

[scholzj]

Immediate means that when the PVC is created, the storage class creates the PV and binds it. It will use whatever logic it has to pick up the node / zone. At that point, it does not know what will the pod constraints be when it is actually used. So I think you should try to move to WaitForFirstConsumer -> which creates the PV and binds it only when the Pod tries to mount it. At that point, it can see the constraints of the Pod and should be able to adjust how the PV is created to them.

So I guess you should either change the Storage Class - or assuming you have it as Immediate for some reason, you can create another Storage CLass with the WaitForFirstConsumer mode which you will use for Kafka / ZooKeeper. You can specify which storage class will be used in the storage section in the Kafka CR.

[brunooliveiramac]

It worked, thanks a lot scholzj