Skip to content

Commit 03359a5

Browse files
v-regandownerv-regandowner
authored
Merge pull request #215783 from phealy/pahealy/aks-ipvs
kube-proxy configuration documentation
2 parents 0f40265 + f17a35e commit 03359a5

File tree

2 files changed

+134
-0
lines changed

2 files changed

+134
-0
lines changed

‎articles/aks/TOC.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -378,6 +378,8 @@
378378
href: internal-lb.md
379379
- name: Use a Standard Load Balancer
380380
href: load-balancer-standard.md
381+
- name: Use kube-proxy configuration (IPVS)
382+
href: configure-kube-proxy.md
381383
- name: Use a static IP address and DNS label
382384
href: static-ip.md
383385
- name: Use an HTTP proxy

‎articles/aks/configure-kube-proxy.md‎

Lines changed: 132 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,132 @@
1+
---
2+
title: Configure kube-proxy (iptables/IPVS) (preview)
3+
titleSuffix: Azure Kubernetes Service
4+
description: Learn how to configure kube-proxy to utilize different load balancing configurations with Azure Kubernetes Service (AKS).
5+
services: container-service
6+
ms.topic: article
7+
ms.date: 10/25/2022
8+
ms.author: pahealy
9+
author: phealy
10+
11+
#Customer intent: As a cluster operator, I want to utilize a different kube-proxy configuration.
12+
---
13+
14+
# Configure `kube-proxy` in Azure Kubernetes Service (AKS) (preview)
15+
16+
`kube-proxy` is a component of Kubernetes that handles routing traffic for services within the cluster. There are two backends available for Layer 3/4 load balancing in upstream `kube-proxy` - iptables and IPVS.
17+
18+
- iptables is the default backend utilized in the majority of Kubernetes clusters. It is simple and well supported, but is not as efficient or intelligent as IPVS.
19+
- IPVS utilizes the Linux Virtual Server, a layer 3/4 load balancer built into the Linux kernel. IPVS provides a number of advantages over the default iptables configuration, including state awareness, connection tracking, and more intelligent load balancing.
20+
21+
The AKS managed `kube-proxy` DaemonSet can also be disabled entirely if that is desired to support [bring-your-own CNI][aks-byo-cni].
22+
23+
[!INCLUDE [preview features callout](./includes/preview/preview-callout.md)]
24+
25+
## Prerequisites
26+
27+
* Azure CLI with aks-preview extension 0.5.105 or later.
28+
* If using ARM or the REST API, the AKS API version must be 2022-08-02-preview or later.
29+
30+
### Install the aks-preview CLI extension
31+
32+
```azurecli-interactive
33+
# Install the aks-preview extension
34+
az extension add --name aks-preview
35+
36+
# Update the extension to make sure you have the latest version installed
37+
az extension update --name aks-preview
38+
```
39+
40+
### Register the `KubeProxyConfigurationPreview` preview feature
41+
42+
To create an AKS cluster with custom `kube-proxy` configuration, you must enable the `KubeProxyConfigurationPreview` feature flag on your subscription.
43+
44+
Register the `KubeProxyConfigurationPreview` feature flag by using the `az feature register` command, as shown in the following example:
45+
46+
```azurecli-interactive
47+
az feature register --namespace "Microsoft.ContainerService" --name "KubeProxyConfigurationPreview"
48+
```
49+
50+
It takes a few minutes for the status to show *Registered*. Verify the registration status by using the `az feature list` command:
51+
52+
```azurecli-interactive
53+
az feature list -o table --query "[?contains(name, 'Microsoft.ContainerService/KubeProxyConfigurationPreview')].{Name:name,State:properties.state}"
54+
```
55+
56+
When the feature has been registered, refresh the registration of the *Microsoft.ContainerService* resource provider by using the `az provider register` command:
57+
58+
```azurecli-interactive
59+
az provider register --namespace Microsoft.ContainerService
60+
```
61+
62+
## Configurable options
63+
64+
The full `kube-proxy` configuration structure can be found in the [AKS Cluster Schema][aks-schema-kubeproxyconfig].
65+
66+
- `enabled` - whether or not to deploy the `kube-proxy` DaemonSet. Defaults to true.
67+
- `mode` - can be set to `IPTABLES` or `IPVS`. Defaults to `IPTABLES`.
68+
- `ipvsConfig` - if `mode` is `IPVS`, this object contains IPVS-specific configuration properties.
69+
- `scheduler` - which connection scheduler to utilize. Supported values:
70+
- `LeastConnections` - sends connections to the backend pod with the fewest connections
71+
- `RoundRobin` - distributes connections evenly between backend pods
72+
- `tcpFinTimeoutSeconds` - the value used for timeout after a FIN has been received in a TCP session
73+
- `tcpTimeoutSeconds` - the value used for timeout length for idle TCP sessions
74+
- `udpTimeoutSeconds` - the value used for timeout length for idle UDP sessions
75+
76+
> [!NOTE]
77+
> IPVS load balancing operates in each node independently and is still only aware of connections flowing through the local node. This means that while `LeastConnections` results in more even load under higher number of connections, when low numbers of connections (# connects < 2 * node count) occur traffic may still be relatively unbalanced.
78+
79+
## Utilize `kube-proxy` configuration in a new or existing AKS cluster using Azure CLI
80+
81+
`kube-proxy` configuration is a cluster-wide setting. No action is needed to update your services.
82+
83+
>[!WARNING]
84+
> Changing the kube-proxy configuration may cause a slight interruption in cluster service traffic flow.
85+
86+
To begin, create a JSON configuration file with the desired settings:
87+
88+
### Create a configuration file
89+
90+
```json
91+
{
92+
"enabled": true,
93+
"mode": "IPVS",
94+
"ipvsConfig": {
95+
"scheduler": "LeastConnection",
96+
"TCPTimeoutSeconds": 900,
97+
"TCPFINTimeoutSeconds": 120,
98+
"UDPTimeoutSeconds": 300
99+
}
100+
}
101+
```
102+
103+
### Deploy a new cluster
104+
105+
Deploy your cluster using `az aks create` and pass in the configuration file:
106+
107+
```bash
108+
az aks create -g <resourceGroup> -n <clusterName> --kube-proxy-config kube-proxy.json
109+
```
110+
111+
### Update an existing cluster
112+
113+
Configure your cluster using `az aks update` and pass in the configuration file:
114+
115+
```bash
116+
az aks update -g <resourceGroup> -n <clusterName> --kube-proxy-config kube-proxy.json
117+
```
118+
119+
## Next steps
120+
121+
Learn more about utilizing the Standard Load Balancer for inbound traffic at the [AKS Standard Load Balancer documentation][load-balancer-standard.md].
122+
123+
Learn more about using Internal Load Balancer for Inbound traffic at the [AKS Internal Load Balancer documentation](internal-lb.md).
124+
125+
Learn more about Kubernetes services at the [Kubernetes services documentation][kubernetes-services].
126+
127+
<!-- LINKS - External -->
128+
[kubernetes-services]: https://kubernetes.io/docs/concepts/services-networking/service/
129+
[aks-schema-kubeproxyconfig]: /azure/templates/microsoft.containerservice/managedclusters?pivots=deployment-language-bicep#containerservicenetworkprofilekubeproxyconfig
130+
131+
<!-- LINKS - Internal -->
132+
[aks-byo-cni]: use-byo-cni.md

0 commit comments

Comments
 (0)