Pods distribution among nodes

[GrSof]

Hello,

I have a Kubernetes cluster that consists on 1 master node and 2 workers. When I deploy a Kafka cluster with 3 zookeeper replicas and 3 kafka replicas, I have noticed that the zookeeper pods are properly spread among all nodes, but, that's not the case for the kafka pods, they are all in the same node. Why is that? Are pods automatically distributed? Is there a logic behind it?

For example:

• Worker1: zookeeper1, zookeeper2,
• Worker2: zookeeper3, kafka1, kafka2, kafka3
  And, if I redeploy the cluster using the same .yaml file the distribution changes to:
• Worker1: zookeeper1, zookeeper2, kafka3
• Worker2: zookeeper3, kafka1, kafka2,

Doing the same in a cluster with 3 workers I get something like this:

• Worker1: zookeeper1, kafka3
• Worker2: zookeeper2, kafka1, kafka2,
• Worker3: zookeeper3
  Any clue about why are the zookeeper pods well spread among all nodes and not the kafka ones?

Thank you in advance

[scholzj]

By default, Kubernetes schedules the pods as it fits. So it can be affected by many aspects => available capacity, storage, etc. If you want, the Kafka custom resource allows you to configure affinity or topology spread constraints to better tell Kubernetes what your desired state is and control the scheduling through it.

[GrSof]

Perfect, thank you very much for the early reply :)

[GrSof]

I forgot to mention that all workers are exactly the same and that there was no other pods in the cluster. How is it possible that the default configuration leaves one node empty?

On the other hand, with respect to the Kafka custom resource. Since we are talking about Kafka replicas and all pods come from the same configuration, I guess I would need to use the topologyKey variable like this?

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
spec:
kafka:
# ...
template:
pod:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: strimzi.io/name
operator: In
values:
- CLUSTER-NAME-kafka
topologyKey: "kubernetes.io/hostname"

Thank you

[scholzj]

I forgot to mention that all workers are exactly the same and that there was no other pods in the cluster. How is it possible that the default configuration leaves one node empty?

As I said, it pretty much depends on the Kubernetes scheduler. I do not have any deep understanding of why would it do something like that.

On the other hand, with respect to the Kafka custom resource. Since we are talking about Kafka replicas and all pods come from the same configuration, I guess I would need to use the topologyKey variable like this?

Yeah, I think this should work, yes.