Skip to content

Commit edbdd11

Browse files
committed
Add network optimizations section
1 parent 0e5a607 commit edbdd11

File tree

1 file changed

+71
-1
lines changed

1 file changed

+71
-1
lines changed

articles/virtual-network/virtual-network-optimize-network-bandwidth.md

Lines changed: 71 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -89,9 +89,79 @@ sudo reboot
8989

9090
Most modern distributions should have significant improvements with kernels newer than 4.19+. Check the current kernel version to make sure that you're running a newer kernel.
9191

92+
### Optimizing network performance in Linux VMs on Azure
93+
94+
Azure Linux VMs often experience network performance issues, particularly when transferring large files (1GB to 50GB) between regions, such as West Europe and West US. These issues are caused by suboptimal kernel configurations, network buffer settings, and default congestion control algorithms, which result in delayed packets, limited throughput, and inefficient resource usage.
95+
96+
To enhance network performance, consider implementing the following optimizations that have been proven effective in a number of situations on Azure:
97+
98+
- **Network buffer settings**: Adjust kernel parameters to maximize read and write memory buffers. Add these configurations to `/etc/sysctl.d/99-azure-network-buffers.conf`:
99+
100+
```plaintext
101+
/etc/sysctl.d/99-azure-network-buffers.conf
102+
net.core.rmem_max = 2147483647
103+
net.core.wmem_max = 2147483647
104+
net.ipv4.tcp_rmem = 4096 67108864 1073741824
105+
net.ipv4.tcp_wmem = 4096 67108864 1073741824
106+
```
107+
108+
- **Congestion control**: Enabling BBR congestion control can often result in better throughput. Add this configuration to `/etc/sysctl.d/99-azure-congestion-control.conf`:
109+
110+
- Ensure the BBR module is loaded by adding it to `/etc/modules-load.d/99-azure-tcp-bbr.conf`:
111+
112+
```plaintext
113+
/etc/modules-load.d/99-azure-tcp-bbr.conf
114+
tcp_bbr
115+
```
116+
117+
```plaintext
118+
/etc/sysctl.d/99-azure-congestion-control.conf
119+
net.ipv4.tcp_congestion_control = bbr
120+
```
121+
122+
- **Queue discipline (qdisc)**: Packet processing in Azure is generally improved by setting the default qdisc to `fq`. Add this configuration to `/etc/sysctl.d/99-azure-qdisc.conf`:
123+
124+
```plaintext
125+
/etc/sysctl.d/99-azure-qdisc.conf
126+
net.core.default_qdisc = fq
127+
```
128+
129+
- Create a udev rule in `/etc/udev/rules.d/99-azure-qdisc.rules` to ensure the qdisc is applied to network interfaces:
130+
131+
```plaintext
132+
/etc/udev/rules.d/99-azure-qdisc.rules
133+
ACTION=="add|change", SUBSYSTEM=="net", KERNEL=="enP*", PROGRAM="/sbin/tc qdisc replace dev \$env{INTERFACE} root noqueue"
134+
ACTION=="add|change", SUBSYSTEM=="net", KERNEL=="eth*", PROGRAM="/sbin/tc qdisc replace dev \$env{INTERFACE} root fq“
135+
```
136+
137+
- **IRQ scheduling**: Depending on your workload, you may wish to restrict the irqbalance service from scheduling IRQs on certain nodes. Update `/etc/default/irqbalance` with the following configuration:
138+
139+
```plaintext
140+
/etc/default/irqbalance
141+
IRQBALANCE_BANNED_CPULIST=0-2
142+
```
143+
144+
- **udev rules**: Add rules to optimize queue length and manage device flags efficiently. Create the following rule in `/etc/udev/rules.d/99-azure-txqueue-len.rules`:
145+
146+
```plaintext
147+
/etc/udev/rules.d/99-azure-txqueue-len.rules
148+
SUBSYSTEM=="net", ACTION=="add|change", KERNEL=="eth*", ATTR{tx_queue_len}="10000“
149+
```
150+
151+
#### For Packets delayed twice
152+
153+
When it comes to Linux performance networking we use SR-IOV with Mellanox drivers (mlx4 or mlx5), something specific to Azure is that this creates two interfaces a synthetic and a virtual interface. [Learn More](/azure/virtual-network/accelerated-networking-how-it-works).
154+
155+
156+
#### Additional Notes
157+
158+
System administrators can implement these solutions by editing configuration files such as `/etc/sysctl.d/`, `/etc/modules-load.d/`, and `/etc/udev/rules.d/`. Ensure that kernel driver updates and systemd configurations are reviewed for potential regressions.
159+
160+
For further details on specific configurations and troubleshooting, refer to Azure documentation on networking performance.
161+
92162
## Related content
93163

94164
- Deploy VMs close to each other for low latency with [proximity placement groups](/azure/virtual-machines/co-location).
95165
- See the optimized result with [Bandwidth/Throughput testing](virtual-network-bandwidth-testing.md) for your scenario.
96166
- Read about how [bandwidth is allocated to virtual machines](virtual-machine-network-throughput.md).
97-
- Learn more with [Azure Virtual Network frequently asked questions](virtual-networks-faq.md).
167+
- Learn more with [Azure Virtual Network frequently asked questions](virtual-networks-faq.md).

0 commit comments

Comments
 (0)