Skip to content

Commit a2326a4

Browse files
committed
add blog for feature LoadBalancerIPMode
1 parent 0719f77 commit a2326a4

File tree

1 file changed

+110
-0
lines changed

1 file changed

+110
-0
lines changed
Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
---
2+
layout: blog
3+
title: "Kubernetes 1.29: New (alpha) Feature, Load Balancer IP Mode for Services"
4+
date: 2023-12-18
5+
slug: kubernetes-1-29-feature-loadbalancer-ip-mode-alpha
6+
---
7+
8+
**Author:** [Aohan Yang](https://github.com/RyanAoh)
9+
10+
This blog introduces a new alpha feature in Kubernetes 1.29.
11+
It provides a configurable approach to define how Service implementations,
12+
exemplified in this blog by kube-proxy,
13+
handle traffic from pods to the Service, within the cluster.
14+
15+
## Background
16+
17+
In older Kubernetes releases, the kube-proxy would intercept traffic that was destined for the IP
18+
address associated with a Service of `type: LoadBalancer`. This happened whatever mode you used
19+
for `kube-proxy`.
20+
The interception implemented the expected behavior (traffic eventually reaching the expected
21+
endpoints behind the Service). The mechanism to make that work depended on the mode for kube-proxy;
22+
on Linux, kube-proxy in iptables mode would redirecting packets directly to the endpoint; in ipvs mode,
23+
kube-proxy would configure the load balancer's IP address to one interface on the node.
24+
The motivation for implementing that interception was for two reasons:
25+
26+
1. **Traffic path optimization:** Efficiently redirecting pod traffic - when a container in a pod sends an outbound
27+
packet that is destined for the load balancer's IP address -
28+
directly to the backend service by bypassing the load balancer.
29+
30+
2. **Handling load balancer packets:** Some load balancers send packets with the destination IP set to
31+
the load balancer's IP address. As a result, these packets need to be routed directly to the correct backend (which
32+
might not be local to that node), in order to avoid loops.
33+
34+
## Problems
35+
36+
However, there are several problems with the aforementioned behavior:
37+
38+
1. **[Source IP](https://github.com/kubernetes/kubernetes/issues/79783):**
39+
Some cloud providers use the load balancer's IP as the source IP when
40+
transmitting packets to the node. In the ipvs mode of kube-proxy,
41+
there is a problem that health checks from the load balancer never return. This occurs because the reply packets
42+
would be forward to the local interface `kube-ipvs0`(where the load balancer's IP is bound to)
43+
and be subsequently ignored.
44+
45+
2. **[Feature loss at load balancer level](https://github.com/kubernetes/kubernetes/issues/66607):**
46+
Certain cloud providers offer features(such as TLS termination, proxy protocol, etc.) at the
47+
load balancer level.
48+
Bypassing the load balancer results in the loss of these features when the packet reaches the service
49+
(leading to protocol errors).
50+
51+
52+
Even with the new alpha behaviour disabled (the default), there is a
53+
[workaround](https://github.com/kubernetes/kubernetes/issues/66607#issuecomment-474513060)
54+
that involves setting `.status.loadBalancer.ingress.hostname` for the Service, in order
55+
to bypass kube-proxy binding.
56+
But this is just a makeshift solution.
57+
58+
## Solution
59+
60+
In summary, providing an option for cloud providers to disable the current behavior would be highly beneficial.
61+
62+
To address this, Kubernetes v1.29 introduces a new (alpha) `.status.loadBalancer.ingress.ipMode`
63+
field for a Service.
64+
This field specifies how the load balancer IP behaves and can be specified only when
65+
the `.status.loadBalancer.ingress.ip` field is also specified.
66+
67+
Two values are possible for `.status.loadBalancer.ingress.ipMode`: `"VIP"` and `"Proxy"`.
68+
The default value is "VIP", meaning that traffic delivered to the node
69+
with the destination set to the load balancer's IP and port will be redirected to the backend service by kube-proxy.
70+
This preserves the existing behavior of kube-proxy.
71+
The "Proxy" value is intended to prevent kube-proxy from binding the load balancer's IP address
72+
to the node in both ipvs and iptables modes.
73+
Consequently, traffic is sent directly to the load balancer and then forwarded to the destination node.
74+
The destination setting for forwarded packets varies depending on how the cloud provider's load balancer delivers traffic:
75+
76+
- If the traffic is delivered to the node then DNATed to the pod, the destination would be set to the node's IP and node port;
77+
- If the traffic is delivered directly to the pod, the destination would be set to the pod's IP and port.
78+
79+
## Usage
80+
81+
Here are the necessary steps to enable this feature:
82+
83+
- Download the [latest Kubernetes project](https://kubernetes.io/releases/download/) (version `v1.29.0` or later).
84+
- Enable the feature gate with the command line flag `--feature-gates=LoadBalancerIPMode=true`
85+
on kube-proxy, kube-apiserver, and cloud-controller-manager.
86+
- For Services with `type: LoadBalancer`, set `ipMode` to the appropriate value.
87+
This step is likely handled by your chosen cloud-controller-manager during the `EnsureLoadBalancer` process.
88+
89+
## More information
90+
91+
- Read [Specifying IPMode of load balancer status](/docs/concepts/services-networking/service/#load-balancer-ip-mode).
92+
- Read [KEP-1860](https://kep.k8s.io/1860) - [Make Kubernetes aware of the LoadBalancer behaviour](https://github.com/kubernetes/enhancements/tree/b103a6b0992439f996be4314caf3bf7b75652366/keps/sig-network/1860-kube-proxy-IP-node-binding#kep-1860-make-kubernetes-aware-of-the-loadbalancer-behaviour) _(sic)_.
93+
94+
## Getting involved
95+
96+
Reach us on [Slack](https://slack.k8s.io/)[#sig-network](https://kubernetes.slack.com/messages/sig-network),
97+
or through the [mailing list](https://groups.google.com/forum/#!forum/kubernetes-sig-network).
98+
99+
## Acknowledgments
100+
101+
Huge thanks to [@Sh4d1](https://github.com/Sh4d1) for the original KEP and initial implementation code.
102+
I took over midway and completed the work. Similarly, immense gratitude to other contributors
103+
who have assisted in the design, implementation, and review of this feature (alphabetical order):
104+
105+
- [@aojea](https://github.com/aojea)
106+
- [@danwinship](https://github.com/danwinship)
107+
- [@sftim](https://github.com/sftim)
108+
- [@tengqm](https://github.com/tengqm)
109+
- [@thockin](https://github.com/thockin)
110+
- [@wojtek-t](https://github.com/wojtek-t)

0 commit comments

Comments
 (0)