You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/modules/listener-operator/pages/index.adoc
+1Lines changed: 1 addition & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,4 +1,5 @@
1
1
= Stackable Listener Operator
2
+
:description: The Stackable Listener Operator for Kubernetes provisions network listeners according to the cluster policy, and injects connection parameters into Pods.
:description: The Listener exposes Pods based on ListenerClass rules, provides address info via Ingress_addresses, supports PVC mounting, and enables sticky scheduling.
2
3
3
-
A `Listener` object exposes a set of ``Pod``s according to the rules of a xref:listenerclass.adoc[], but it also adds a couple of other
4
-
features that are useful for the Stackable platform at large.
4
+
A Listener object exposes a set of Pods according to the rules of a xref:listenerclass.adoc[], but it also adds a couple of other features that are useful for the Stackable data platform at large.
5
5
6
-
== `ListenerClass`
6
+
== ListenerClass
7
7
8
-
The exact rules of pod exposure are dictated by the specified xref:listenerclass.adoc[], which allow a single `Listener` definition to be reused in different clusters, regardless of the Kubernetes distribution or cloud provider.
8
+
The exact rules of pod exposure are dictated by the specified xref:listenerclass.adoc[], which allow a single Listener definition to be reused in different clusters, regardless of the Kubernetes distribution or cloud provider.
9
9
10
10
== Address API
11
11
12
-
A `Listener` writes back all addresses that it can be reached on to `Listener.status.ingress_addresses`, which can then be used to generate discovery information. Contrary to Kubernetes' `Service`, this is done regardless of the type of service, and transparently also contains information about remapped ports.
12
+
A Listener writes back all addresses that it can be reached on to `Listener.status.ingress_addresses`, which can then be used to generate discovery information.
13
+
Contrary to Kubernetes' Service, this is done regardless of the type of service, and transparently also contains information about remapped ports.
13
14
14
15
== Address volume projection
15
16
16
-
`Listener` objects can be mounted into a `Pod` as a `PersistentVolumeClaim`, which contains information about how the `Pod` should request that external clients refer to it.
17
+
Listener objects can be mounted into a Pod as a PersistentVolumeClaim (PVC), which contains information about how the Pod should request that external clients refer to it.
17
18
18
19
For example, if the volume is mounted to `/stackable/listener`, the primary address can be read from `/stackable/listener/default-address/address`, and the public `http` port number can be read from `/stackable/listener/default-address/ports/http`.
19
20
20
21
== Per-replica listeners
21
22
22
-
A `Listener` PVC can also specify a xref:listenerclass.adoc[] rather than a `Listener`, in which case a `Listener` object is created
23
-
automatically. These PVCs can automatically be created for each replica using either ``StatefulSet``'s `volumeClaimTemplates` (for long-lived listeners that will
24
-
be kept across replica restarts and upgrades) or ``Pod``'s `volumes[].ephemeral` (for temporary listeners that are deleted when their corresponding `Pod` is deleted).
23
+
A Listener PVC can also specify a xref:listenerclass.adoc[] rather than a Listener, in which case a Listener object is created automatically.
24
+
These PVCs can automatically be created for each replica using either StatefulSet's `volumeClaimTemplates` (for long-lived listeners that will be kept across replica restarts and upgrades) or Pod's `volumes[].ephemeral` (for temporary listeners that are deleted when their corresponding Pod is deleted).
25
25
26
26
== Sticky scheduling
27
27
28
-
When mounting a `Listener` PVC, it will be made "sticky" to that node if the xref:listenerclass.adoc[] uses a strategy that depends on the node
29
-
that the workload is running on.
28
+
When mounting a Listener PVC, it will be made "sticky" to that node if the xref:listenerclass.adoc[] uses a strategy that depends on the node that the workload is running on.
30
29
31
-
Keep in mind that this will only work correctly when using long-lived PVCs (such as via ``StatefulSet``'s `volumeClaimTemplates`). Ephemeral PVCs
32
-
will be "reset" for every pod that is created, even if they refer to a long-lived `Listener` object.
30
+
Keep in mind that this will only work correctly when using long-lived PVCs (such as via StatefulSet's `volumeClaimTemplates`).
31
+
Ephemeral PVCs will be "reset" for every pod that is created, even if they refer to a long-lived Listener object.
Copy file name to clipboardExpand all lines: docs/modules/listener-operator/pages/listenerclass.adoc
+35-15Lines changed: 35 additions & 15 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,8 @@
1
1
= ListenerClass
2
+
:description: The ListenerClass defines listener types and exposure rules for Kubernetes Pods, supporting various service types like ClusterIP, NodePort, and LoadBalancer.
2
3
3
-
A ListenerClass defines a category of listeners. For example, this could be "VPC-internal service", "internet-accessible service", or "K8s-internal service".
4
+
A ListenerClass defines a category of listeners.
5
+
For example, this could be "VPC-internal service", "internet-accessible service", or "K8s-internal service".
4
6
The ListenerClass then defines how this intent is realized in a given cluster.
5
7
6
8
For example, a Google Kubernetes Engine (GKE) cluster might want to expose all internet-facing services using a managed load balancer, since GKE nodes are
@@ -11,14 +13,17 @@ relatively short-lived and don't have stable addresses:
11
13
include::example$listenerclass-public-gke.yaml[]
12
14
----
13
15
14
-
On the other hand, an on-premise cluster might not have dedicated load balancer infrastructure at all, but instead use "pet" Nodes which may be expected to live for years. This might lead administrators of such systems to prefer exposing node ports directly instead:
16
+
On the other hand, an on-premise cluster might not have dedicated load balancer infrastructure at all, but instead use "pet" Nodes which may be expected to live for years.
17
+
This might lead administrators of such systems to prefer exposing node ports directly instead:
Finally, it can be desirable to add additional annotations to a Service. For example, a user might want to only expose some services inside a given cloud vendor VPC. How exactly this is accomplished depends on the cloud provider in question, but for GKE this requires the annotation `networking.gke.io/load-balancer-type`:
24
+
Finally, it can be desirable to add additional annotations to a Service.
25
+
For example, a user might want to only expose some services inside a given cloud vendor VPC.
26
+
How exactly this is accomplished depends on the cloud provider in question, but for GKE this requires the annotation `networking.gke.io/load-balancer-type`:
The service type is defined by `ListenerClass.spec.serviceType`. The following service types are currently supported by the Stackable Listener Operator:
35
+
The service type is defined by `ListenerClass.spec.serviceType`.
36
+
The following service types are currently supported by the Stackable Listener Operator:
31
37
32
38
[#servicetype-clusterip]
33
39
=== `ClusterIP`
34
40
35
-
The Listener can be accessed from inside the Kubernetes cluster. The Listener addresses will direct clients to the cluster-internal address.
41
+
The Listener can be accessed from inside the Kubernetes cluster.
42
+
The Listener addresses will direct clients to the cluster-internal address.
36
43
37
44
[#servicetype-nodeport]
38
45
=== `NodePort`
39
46
40
-
The Listener can be accessed from outside the Kubernetes cluster. This may include the internet, if the Nodes have public IP addresses. The Listener address will direct clients to connect to a randomly assigned port on the Nodes running the Pods.
47
+
The Listener can be accessed from outside the Kubernetes cluster.
48
+
This may include the internet, if the Nodes have public IP addresses.
49
+
The Listener address will direct clients to connect to a randomly assigned port on the Nodes running the Pods.
41
50
42
-
Additionally, Pods bound to `NodePort` listeners will be xref:volume.adoc#pinning[pinned] to a specific Node. If this is undesirable, consider using xref:#servicetype-loadbalancer[] instead.
51
+
Additionally, Pods bound to `NodePort` listeners will be xref:volume.adoc#pinning[pinned] to a specific Node.
52
+
If this is undesirable, consider using xref:#servicetype-loadbalancer[] instead.
43
53
44
54
[#servicetype-loadbalancer]
45
55
=== `LoadBalancer`
46
56
47
-
The Listener can be accessed from outside the Kubernetes cluster. This may include the internet, depending on the configuration of the Kubernetes cloud controller manager. A dedicated address will be allocated for the Listener.
57
+
The Listener can be accessed from outside the Kubernetes cluster.
58
+
This may include the internet, depending on the configuration of the Kubernetes cloud controller manager.
59
+
A dedicated address will be allocated for the Listener.
48
60
49
-
Compared to xref:#servicetype-nodeport[], this service type allows Pods to be moved freely between Nodes. However, it requires https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer[a cloud controller manager that supports load balancers]. Additionally, many cloud providers charge for load-balanced traffic.
61
+
Compared to xref:#servicetype-nodeport[], this service type allows Pods to be moved freely between Nodes.
62
+
However, it requires https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer[a cloud controller manager that supports load balancers].
63
+
Additionally, many cloud providers charge for load-balanced traffic.
50
64
51
65
== Default ListenerClasses
52
66
@@ -58,13 +72,16 @@ The Stackable Data Platform assumes the existence of a few predefined ListenerCl
58
72
59
73
=== Presets
60
74
61
-
To help users get started, the Stackable Listener Operator ships different ListenerClass _presets_ for different environments. These are configured using the `preset` Helm value.
75
+
To help users get started, the Stackable Listener Operator ships different ListenerClass _presets_ for different environments.
76
+
These are configured using the `preset` Helm value.
62
77
63
78
[#preset-stable-nodes]
64
79
==== `stable-nodes`
65
80
66
-
The `stable-nodes` preset installs ListenerClasses appropriate for Kubernetes clusters that use long-lived "pet nodes". This does _not_ require any particular networking setup, but makes pods that require
67
-
stable addresses "sticky" to the Kubernetes Node that they were scheduled to. In addition, downstream operators may generate configurations that refer to particular nodes by name.
81
+
The `stable-nodes` preset installs ListenerClasses appropriate for Kubernetes clusters that use long-lived "pet nodes".
82
+
This does _not_ require any particular networking setup, but makes pods that require
83
+
stable addresses "sticky" to the Kubernetes Node that they were scheduled to.
84
+
In addition, downstream operators may generate configurations that refer to particular nodes by name.
68
85
69
86
The following ListenerClasses are installed:
70
87
@@ -75,12 +92,15 @@ The following ListenerClasses are installed:
75
92
[#preset-ephemeral-nodes]
76
93
==== `ephemeral-nodes`
77
94
78
-
The `ephemeral-nodes` preset installs ListenerClasses appropriate for Kubernetes clusters that use short-lived "cattle nodes". This makes them appropriate for managed cloud environments, but requires that
95
+
The `ephemeral-nodes` preset installs ListenerClasses appropriate for Kubernetes clusters that use short-lived "cattle nodes".
96
+
This makes them appropriate for managed cloud environments, but requires that
79
97
a LoadBalancer controller is present in the cluster.
80
98
81
-
Managed cloud environments should generally already provide an integrated LoadBalancer controller. For on-premise environments, an external implementation such as https://docs.tigera.io/calico/latest/networking/configuring/advertise-service-ips[Calico] or https://metallb.org/[MetalLB] can be used.
99
+
Managed cloud environments should generally already provide an integrated LoadBalancer controller.
100
+
For on-premise environments, an external implementation such as https://docs.tigera.io/calico/latest/networking/configuring/advertise-service-ips[Calico] or https://metallb.org/[MetalLB] can be used.
82
101
83
-
NOTE: K3s' built-in https://docs.k3s.io/networking#service-load-balancer[ServiceLB] (Klipper) is _not_ recommended, because it doesn't allow multiple Services to bind the same Port. If you use ServiceLB, use the xref:#preset-stable-nodes[] preset instead.
102
+
NOTE: K3s' built-in https://docs.k3s.io/networking#service-load-balancer[ServiceLB] (Klipper) is _not_ recommended, because it doesn't allow multiple Services to bind the same Port.
103
+
If you use ServiceLB, use the xref:#preset-stable-nodes[] preset instead.
:description: The Listener Operator requires root container privileges to create a Unix domain socket and write exposed address information to pod volumes.
2
3
3
4
== Container privileges
4
5
5
-
The Listener Operator runs as a set of root containers. This is needed for two reasons:
6
+
The Listener Operator runs as a set of root containers.
7
+
This is needed for two reasons:
6
8
7
-
1. We need to run as root to have permission to create the Unix domain socket hosting the Container Storage interface (CSI)
8
-
driver. The Kubelet communicates with the CSI driver over this socket.
9
-
2. We need to run as root to have permission to write information about externally exposed addresses into the pods' volume paths, as directed
10
-
by the CSI.
9
+
1. The root user is needed for the permission to create the Unix domain socket hosting the Container Storage interface (CSI) driver.
10
+
The Kubelet communicates with the CSI driver over this socket.
11
+
2. The root user is needed for the permission to write information about externally exposed addresses into the pods' volume paths, as directed by the CSI.
:description: The Listener Operator uses CSI PersistentVolumes to stabilize network addresses, inject pod metadata, and expose individual Pods with pinning.
2
3
3
-
4
-
The Listener Operator acts as a CSI `PersistentVolume`, which helps it to stabilize network addresses, inject pod metadata and expose individual pods.
4
+
The Listener Operator acts as a CSI PersistentVolume, which helps it to stabilize network addresses, inject pod metadata and expose individual Pods.
5
5
6
6
[#pinning]
7
7
== Stable addresses
8
8
9
-
Some xref:listenerclass.adoc[] strategies, such as `NodePort`, tie the public address to the Kubernetes node that the `Pod` is running on. When this address must be configured statically in clients
10
-
(such as for HDFS NameNodes), then Kubernetes' default "floating" scheduling either requires all clients to be reconfigured every time something moves, or for all clients to proxy their traffic through
11
-
a single static node, which then becomes a single point of failure (along with the node that the workload is running on).
9
+
Some xref:listenerclass.adoc[] strategies, such as `NodePort`, tie the public address to the Kubernetes node that the Pod is running on.
10
+
When this address must be configured statically in clients (such as for HDFS NameNodes), then Kubernetes' default "floating" scheduling either requires all clients to be reconfigured every time something moves, or for all clients to proxy their traffic through a single static node, which then becomes a single point of failure (along with the node that the workload is running on).
12
11
13
-
Mounting listeners into Pods as `PersistentVolume` allows the Listener Operator to pin these workloads to one node. Note that this only happens for xref:listenerclass.adoc[]es that actually benefit
14
-
from pinning.
12
+
Mounting listeners into Pods as PersistentVolume allows the Listener Operator to pin these workloads to one node.
13
+
Note that this only happens for xref:listenerclass.adoc[]es that actually benefit from pinning.
15
14
16
15
== Pod metadata injection
17
16
18
-
Some services (such as Kafka) need to know their external address, so that they can advertise it to their own replica discovery mechanism. xref:listener.adoc[] volumes contain a file tree that exposes
19
-
this information:
17
+
Some services (such as Kafka) need to know their external address, so that they can advertize it to their own replica discovery mechanism.
18
+
xref:listener.adoc[] volumes contain a file tree that exposes this information:
20
19
21
20
[square]
22
21
* `default-address/`- A symlink to `addresses/{primary address}`
@@ -31,19 +30,25 @@ this information:
31
30
32
31
== Individual pod exposure
33
32
34
-
Sometimes each replica must be exposed individually, for example because clients need to access data on a specific shard. `PersistentVolumeClaim` templates can be used to provision this automatically.
33
+
Sometimes each replica must be exposed individually, for example because clients need to access data on a specific shard.
34
+
PersistentVolumeClaim templates can be used to provision this automatically.
35
35
36
-
=== `StatefulSet` `volumeClaimTemplates`
36
+
=== StatefulSet `volumeClaimTemplates`
37
37
38
-
The `volumeClaimTemplates` allow volumes to be provisioned for each `StatefulSet` replica. These volumes are _persistent_, and will not be deleted when the `Pod` or `StatefulSet` is. This makes them useful for provisioning addresses that must be hard-coded into client configuration.
38
+
The `volumeClaimTemplates` allow volumes to be provisioned for each StatefulSet replica.
39
+
These volumes are _persistent_, and will not be deleted when the Pod or StatefulSet is.
40
+
This makes them useful for provisioning addresses that must be hard-coded into client configuration.
39
41
40
42
=== Pod-scoped ephemeral volumes
41
43
42
-
`Pod.spec.volumes[].ephemeral` allows volumes to be provisioned for each `Pod`. These volumes are tied to the lifetime of the `Pod` and will be deleted along with it. This makes them useful for provisioning temporary addresses that will be discovered out of band (such as for HDFS DataNodes).
44
+
`Pod.spec.volumes[].ephemeral` allows volumes to be provisioned for each Pod.
45
+
These volumes are tied to the lifetime of the Pod and will be deleted along with it.
46
+
This makes them useful for provisioning temporary addresses that will be discovered out of band (such as for HDFS DataNodes).
43
47
44
48
== Reference
45
49
46
-
All configuration must be specified as `annotations` on the `PersistentVolumeClaim`. The following attributes are currently supported:
50
+
All configuration must be specified as `annotations` on the PersistentVolumeClaim.
51
+
The following attributes are currently supported:
47
52
48
53
=== `listeners.stackable.tech/listener-name`
49
54
@@ -55,5 +60,6 @@ Provisions metadata about an existing xref:listener.adoc[] that was created manu
55
60
56
61
*Required*: If `listeners.stackable.tech/listener-name` is not specified
57
62
58
-
Provisions a new xref:listener.adoc[] using the specified xref:listenerclass.adoc[]. The created xref:listener.adoc[] will expose
59
-
all of the ``Pod``'s ports.
63
+
Provisions a new xref:listener.adoc[] using the specified xref:listenerclass.adoc[].
0 commit comments