Is using split DNS the only way to enable connectivity to Strimzi Kafka both from inside and outside the Kubernetes cluster?

[sovereignstack]

Out of the box, Strimzi Kafka works without connectivity issues from inside the k8s cluster. I have tried some examples of accessing from it outside the cluster but it didn't work so far. Besides, I ultimately want to use it from inside the cluster as well as from outside and this use case is not well supported by Kafka (hope to be proven wrong). Since Strimzi just builds on Kafka, I thought of asking before investing more time in trial and error.

This is my understanding of how client establishes connection with Kafka:

1. Client connects to Boot Strap Kafka server.
2. Bootstrap server knows about the other Kafka nodes in the cluster and relays that information to the client.
3. If a advertized listener is configured, those endpoints are relayed to the client instead of the endpoints Kafka brokers use internally.
4. Client directly connects to any of the endpoints returned by Kafka broker.

Constraint: Incoming traffic on Public IP is blocked by firewall except from a few whitelisted IPs. If I try to connect to Kafka using public IP from within the cluster, it won't work, I have use ClusterIP from inside the cluster.

Now,

1. If I don't configure advertised listeners, clients can't connect from another server as Bootstrap server will return cluster IPs
2. If I configure Public IP of the server in advertised listeners, clients inside the cluster will try to access the public IP which would fail due to firewall restrictions. Even if firewall was to be removed, it is not good that a the packet has go outside the cluster and back in to access a resource within the cluster.

The only way I could make it work is by using split DNS - i.e use a domain in advertised listeners. The domain should resolve to cluster IPs from inside the cluster and Public IPs from outside the cluster.

I want someone to confirm that using split DNS is my only option so that I don't waste time trying to make it work without using split DNS

The reason I don't want to use split DNS is it requires a domain name and it becomes difficult to obtain in corporate setting. hosts file entry is an option but work computers are often locked down so making custom host entries without admin rights may not be possible.

[sreejesh123]

By default if authorised host is not mentioned bootstrap returns ip address. My insert By default bootstrap route when customers call they return the broker IPs. We use GKE and use both internal and external connections . We use authorised host to return a set of hosts so that we can white list those in DNS of our certificates.

we setup 9093 port to use NODEIP as internal load balancer, this is accessible or visible from outside the cluster and as long as you have fw rules set anyone can connect.
You can setup external LB as well for external connections.
regarding internal Setup service which Kafka has in our case 9092 has automatic visibility within cluster and we use servicename.namespace.Svc to access it.

[scholzj]

Have you read the documentation? The part about the various listener types? There are several types of external listeners using load balancers, node ports or ingress for example which you can use to expose your Kafka cluster to the outside of your Kubernetes cluster.