Skip to content

Commit 6435c6f

Browse files
committed
Minor updates based on feedback
1 parent c7284e0 commit 6435c6f

File tree

1 file changed

+15
-10
lines changed

1 file changed

+15
-10
lines changed

articles/aks/troubleshoot-udp-packet-drops.md

Lines changed: 15 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
title: Diagnose and solve UDP packet drops in Azure Kubernetes Service (AKS)
33
description: Learn how to diagnose and solve UDP packet drops in Azure Kubernetes Service (AKS).
44
ms.topic: how-to
5-
ms.date: 05/08/2024
5+
ms.date: 05/09/2024
66
author: schaffererin
77
ms.author: schaffererin
88
ms.service: azure-kubernetes-service
@@ -37,6 +37,8 @@ To allow the buffer to grow to serve more traffic, you need to update the maximu
3737
3838
## Diagnose the issue
3939

40+
### Check current UDP buffer settings
41+
4042
1. Get a list of your nodes using the `kubectl get nodes` command and pick a node you want to check the buffer settings for.
4143

4244
```bash
@@ -52,19 +54,22 @@ To allow the buffer to grow to serve more traffic, you need to update the maximu
5254
3. Get the value of the `net.core.rmem_max` and `net.core.rmem_default` variables using the following `sysctl` command:
5355

5456
```bash
55-
sysctl net.core.rmem_max net.core.rmem_default net.core.wmem_max net.core.wmem_default
57+
sysctl net.core.rmem_max net.core.rmem_default
5658
```
5759

58-
4. Check if your buffer is too small for your application and dropping packets by simulating realistic network traffic on your pods.
59-
5. Check the UDP file using the following `cat` command:
60+
### Measure incoming UDP traffic
61+
62+
To check if your buffer is too small for your application and is dropping packets, start by simulating realistic network traffic on your pods and setting up a debug pod to monitor the incoming traffic. Then, you can use the following commands to measure the incoming UDP traffic:
63+
64+
1. Check the UDP file using the following `cat` command:
6065

6166
```bash
6267
cat /proc/net/udp
6368
```
6469

6570
This file shows you the statistics of the current open connections under the `rx_queue` column. It doesn't show historical data.
6671
67-
6. Check the snmp file using the following `cat` command:
72+
2. Check the snmp file using the following `cat` command:
6873
6974
```bash
7075
cat /proc/net/snmp
@@ -82,14 +87,14 @@ If you notice an increase beyond your buffer size in the `rx_queue` or an uptick
8287
> [!IMPORTANT]
8388
> Before you proceed, it's important to understand the impact of changing the buffer size. The buffer size tells the system kernel to reserve a certain amount of memory for the socket. More sockets and larger buffers can lead to increased memory reserved for the sockets and less memory available for other resources on the nodes. This can lead to resource starvation if not configured properly.
8489

85-
You can change buffer size values on a node pool level during the node pool creation process. The steps in this section show you how to configure a Linux OS and apply the changes to all nodes in the node pool. You can't add this setting to an existing node pool.
90+
You can change buffer size values on a node pool level during the node pool creation process. The steps in this section show you how to configure your Linux OS and apply the changes to all nodes in the node pool. You can't add this setting to an existing node pool.
8691
8792
1. Create a `linuxosconfig.json` file with the following content. You can modify the values based on your application's requirements and node SKU. The minimum value is *212992 bytes*, and the maximum value is *134217728 bytes*.
8893

8994
```json
9095
{
9196
"sysctls": {
92-
"netCoreRmemMax": 8000000
97+
"netCoreRmemMax": 2000000
9398
}
9499
}
95100
```
@@ -100,7 +105,7 @@ You can change buffer size values on a node pool level during the node pool crea
100105
az aks nodepool add --resource-group $RESOURCE_GROUP --cluster-name $CLUSTER_NAME --name $NODE_POOL_NAME --linux-os-config ./linuxosconfig.json
101106
```
102107
103-
This command sets the maximum UDP buffer size to `8 MB` for each socket on the node. You can adjust the value in the `linuxosconfig.json` file to meet your application's requirements.
108+
This command sets the maximum UDP buffer size to `2 MB` for each socket on the node. You can adjust the value in the `linuxosconfig.json` file to meet your application's requirements.
104109

105110
## Validate the changes
106111

@@ -118,10 +123,10 @@ Once you apply the new values, you can access your VM to ensure the new values a
118123
kubectl debug <node-name> -it --image=ubuntu --share-processes -- bash
119124
```
120125

121-
3. Get the value of the `net.core.rmem_max` and `net.core.wmem_max` variables using the following `sysctl` command:
126+
3. Get the value of the `net.core.rmem_max` variable using the following `sysctl` command:
122127

123128
```bash
124-
sysctl net.core.rmem_max net.core.wmem_max
129+
sysctl net.core.rmem_max
125130
```
126131

127132
## Next steps

0 commit comments

Comments
 (0)