|
| 1 | += Overview |
| 2 | + |
| 3 | +So. You've deployed something, and now you want let people call into it. |
| 4 | +Just grab the IP address, port, maybe punch a hole through the firewall if you're feeling particularly adventurous, and you're off to the races... right? |
| 5 | + |
| 6 | +I'm afraid it's not quite so simple when we're running in Kubernetes, because having more machines makes everything harder. |
| 7 | + |
| 8 | +When exposing a product there are a few things that we need to understand about the product and customer environment. |
| 9 | + |
| 10 | +This page will contain some guidance for when to expect each option to apply, but it is still up to you, dear implementer, to verify how your product in question works. |
| 11 | + |
| 12 | +[#responsibility] |
| 13 | +== Whose Policy Is It Anyway? |
| 14 | + |
| 15 | +One of the core principles of the Listener operator is to separate _product knowledge_ from _cluster policy_. |
| 16 | + |
| 17 | +We (Stackable) know what networking requirements the applications have (xref:#routing[routing], xref:#address-stability[stability expectations], and so on). |
| 18 | +But there's also a lot that we _don't_. |
| 19 | +Who needs to xref:#access-scope[access the service]? |
| 20 | +Does the cluster provide a xref:#address-stability[load balancer]? |
| 21 | +How much does using that load balancer cost? |
| 22 | + |
| 23 | +The Listener Operator tries to separate these concepts, by letting administrators define cluster policies using xref:listenerclass.adoc[ListenerClasses], and then letting each application _apply_ one (or more) of those policies via xref:listener.adoc[]. |
| 24 | + |
| 25 | +[#access-scope] |
| 26 | +== Access control/scope |
| 27 | + |
| 28 | +Not all services should be accessible from the public internet. |
| 29 | +Of course, xref:#authentication[authentication] is still also very important, but a useful first step is limiting who is able to access the service at all in the first place. |
| 30 | + |
| 31 | +With the Listener operator, this kind of policy is generally defined by the xref:listenerclass.adoc[] and/or regular Kubernetes mechanisms such as https://kubernetes.io/docs/concepts/services-networking/network-policies/[NetworkPolicy]. |
| 32 | +Application administrators _must_ always be free to apply any ListenerClass of their choosing. |
| 33 | + |
| 34 | +[#routing] |
| 35 | +== Request routing |
| 36 | + |
| 37 | +We'll usually want to support running more than one replica of a given service.. which means that we need some way to direct clients to the right server. |
| 38 | + |
| 39 | +=== Server-side request routing |
| 40 | + |
| 41 | +Sometimes, clients should just connect to _any_ instance of the service, and the service itself is responsible for Doing The Right Thing(tm). |
| 42 | + |
| 43 | +This is common for stateless front-ends, or for services that handle a more complicated consensus protocol internally (like ZooKeeper). |
| 44 | + |
| 45 | +Kubernetes traditionally handles this through deploying a common https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.31/#service-v1-core[Service] across the entire replica set. |
| 46 | + |
| 47 | +The Listener operator supports this by manually deploying a common xref:listener.adoc[] object, and then xref:volume.adoc#shared-exposure[mounting] it into each replica. |
| 48 | + |
| 49 | +NOTE: However, these listeners _may_ still have more than one address (for example: when using xref:listenerclass.adoc#servicetype-nodeport[node-bound listeners]). Clients are responsible for picking a random target address from the options given. |
| 50 | + |
| 51 | +=== Client-side request routing |
| 52 | + |
| 53 | +Sometimes, the client needs to connect to a specific instance of a service. |
| 54 | + |
| 55 | +For example, a HDFS client connects to _all_ available NameNodes to find the current primary instance, then queries _that_ one for which DataNode has the file that it is looking for. |
| 56 | + |
| 57 | +Kubernetes doesn't really handle this natively, you would need to deploy a separate Service for each |
| 58 | + |
| 59 | +The Listener operator supports this by automatically creating xref:listener.adoc[Listeners] corresponding to each xref:volume.adoc[] when xref:volume.adoc#individual-pod-exposure[requested]. |
| 60 | + |
| 61 | +[#address-stability] |
| 62 | +== Address stability |
| 63 | + |
| 64 | +We want to avoid clients needing to update their connection configuration just because the service got redeployed. |
| 65 | + |
| 66 | +The Listener operator binds the lifetime of an address to the lifetime of the xref:volume.adoc[Listener Volume]. |
| 67 | +As long as the PersistentVolume exists, the xref:listener.adoc[] is expected to keep the same address. |
| 68 | +If the PersistentVolume(Claim) is deleted (and recreated), then the address may{empty}footnote:[But isn't always.] be changed. |
| 69 | +Long-lived bindings can be created through `StatefulSet.spec.volumeClaimTemplates`, which creates "permanent" PersistentVolumes, which are not cleaned up automatically and must be deleted manually by an administrator once they are no longer used. |
| 70 | +Short-lived bindings should be created through `Pod.spec.volumes.ephemeral`, which creates a PersistentVolume that will automatically be deleted once the Pod no longer exists. |
| 71 | + |
| 72 | +WARNING: Ephemeral CSI volumes (configured via `Pod.spec.volumes.csi`) are a different thing entirely, and are not supported by the Listener operator. Ephemeral PersistentVolumeClaims (`.volumes.ephemeral`) are still "Persistent" from CSI's point of view. |
| 73 | + |
| 74 | +The listener operator provides a few tools for dealing with this: |
| 75 | +load balancers (xref:listenerclass.adoc#servicetype-loadbalancer[external] and xref:listenerclass.adoc#servicetype-clusterip[in-cluster]) and xref:volume.adoc#pinning[pinning]. |
| 76 | + |
| 77 | +Load balancers provide a stable shared address, but (external) load balancers aren't available in all clusters (Kubernetes provides https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer[a standard API], but the actual implementation is up to the cloud provider or an external project like https://metallb.io/[MetalLB]). |
| 78 | +Going through an (external) load balancer also tends to add an extra traffic hop, slowing down access and incurring extra costs (especially in cloud environments, which will generally charge extra for traffic that passes through load balancers). |
| 79 | + |
| 80 | +:fn-nodeport-lb: footnote:[Unless Kubernetes is configured to also https://kubernetes.io/docs/reference/networking/virtual-ips/#external-traffic-policy[balance NodePort traffic]. However, the Stackable Data Platform generally avoids this feature, since it increases the blast radius of unavailable Nodes.] |
| 81 | + |
| 82 | +xref:listenerclass.adoc#servicetype-nodeport[NodePort] services avoid the additional hop, but require users to direct the traffic to correct Node hosting the service.{fn-nodeport-lb} |
| 83 | +Normally directing traffic to individual Nodes is wildly impractical, because Kubernetes is free to schedule a given Pod to a new Node every time it is recreated. |
| 84 | +The Listener operator works around this by xref:volume.adoc#pinning[pinning] Pods to specific Nodes if required to provide a stable address. |
| 85 | +However, this _does_ come at the caveat of preventing Kubernetes from scheduling Pods that are pinned to Nodes that are no longer available (or that no longer exist). |
| 86 | + |
| 87 | +[#authentication] |
| 88 | +== Authentication (TLS/Kerberos) |
| 89 | + |
| 90 | +Services often need to authenticate their identity, so that clients can be sure that their traffic isn't intercepted by an impostor. Additionally, services usually want to authenticate who their clients are! |
| 91 | + |
| 92 | +This isn't covered by the Listener operator itself, but the xref:secret-operator:index.adoc[] can be used to provision TLS and Kerberos credentials that xref:secret-operator:scope.adoc#listener-volume[correspond] to xref:listener.adoc[] addresses. |
0 commit comments