DNS alias records handling during scale out (Azure)

[opsocomusr]

As per the documentation, w need to add annotations and host for every broker (per-broker and +1 for bootstrap) for the ingress type listener.

How does this work in an autoscaling (rebalancing) scenario using cruise control? Will Strimzi or cruise control add the DNS alias record in the Azure cloud provider when the brokers scale out? Right now, I am adding the DNS record manually for every additional broker that gets added to the cluster.

What is the impact of not having the DNS record for the new brokers that are scaled out?

[scholzj]

I think there are several things ...

• In general, I do not expect anything to autoscale the Kafka cluster. For the most common Kafka workloads, autoscaling the cluster makes little sense. So when it is needed because of onboarding new projects or growing traffic, most people just scale it manually. So you have an opportunity to configure the address for the new broker.
• Cruise Control / rebalancing will not add any additional brokers. It just reshuffles the partition replicas between the broker or tells you that you need to scale-up. So this is not a problem.
• One of the common ways how to manage the DNS names is to use External DNS and service / ingress annotations which instruct it to manage the DNS records. It supports most common DNS providers.
• At last Kubernetes Nginx Ingress AFAIK always requires you to specify the address. So we just pass it to the user. This works better for example with OpenShift Routes where the hostname is assigned automatically. In theory we can have some way to generate the names if not set, but nobody requested it so far and I'm not sure if we use some pattern such as <service>-<namespace>.<domain> if it works for everyone.
• Load Balancer and node port listeners do not require to specify anything when adding brokers ... the addresses come from nodes or from the load balancer. So you can consider that as an alternative.