RTL8125B performance degradation on kernel 6.17.x (Ubuntu 25.10)

[willaraujo]

Package: linux
Version: 6.17.0-12-generic
Ubuntu Release: 25.10 (Questing)

=== SUMMARY ===
Realtek RTL8125B 2.5GbE NIC experiences progressive performance degradation
on Ubuntu 25.10 with kernel 6.17.x series. Network throughput degrades from
2300+ Mbps to 800-1100 Mbps over 5-7 days of uptime without any configuration
changes or network load issues.

=== EXPECTED BEHAVIOR ===

• Sustained 2.3+ Gbps throughput on 2.5G connection
• Stable performance over extended uptime (weeks/months)
• Proper hardware offload features enabled by default

=== ACTUAL BEHAVIOR ===

• Initial speed after reboot: 2276 Mbps down / 2061 Mbps up
• After 13 hours: 266 Mbps down (78% degradation)
• After 7 days: 1100 Mbps down (52% degradation)
• Reboot temporarily restores full speed

=== IMPACT ===

• Users on multi-gigabit connections see severe bandwidth loss
• Realtek RTL8125 is common in consumer motherboards (ASRock, ASUS, MSI, Gigabyte)
• Affects both r8125 DKMS driver and in-kernel r8169 driver
• Issue NOT present on kernel 6.8.x (Ubuntu 24.04 LTS)

=== ROOT CAUSE ANALYSIS ===
Testing revealed THREE issues introduced in kernel 6.17.x:

1. GRO (Generic Receive Offload) disabled by default

   • Critical performance feature for high-speed NICs
   • Was enabled by default in kernel 6.8.x
   • Disabling causes 50% reduction in packet processing efficiency

2. Network buffer defaults too conservative

   • net.core.rmem_max: 2.6 MB (was 134 MB in 6.8.x)
   • net.core.netdev_max_backlog: 1000 (was 5000 in 6.8.x)
   • Causes packet drops under sustained 2.5G load

3. Progressive RX packet loss

   • 1.6 million rx_mac_missed packets over 7 days
   • Single-queue NIC overwhelmed without proper buffering
   • TCP retransmissions degrade effective throughput

=== WORKAROUND (CONFIRMED WORKING) ===
Enable GRO and increase network buffers:

1. Enable GRO:
   ethtool -K enp4s0 gro on

2. Increase buffers:
   sysctl -w net.core.rmem_default=26214400
   sysctl -w net.core.rmem_max=134217728
   sysctl -w net.core.wmem_default=26214400
   sysctl -w net.core.wmem_max=134217728
   sysctl -w net.core.netdev_max_backlog=5000
   sysctl -w net.ipv4.tcp_rmem="4096 87380 67108864"
   sysctl -w net.ipv4.tcp_wmem="4096 65536 67108864"

3. Make permanent:
   Create /etc/sysctl.d/99-network-performance.conf with above values
   Create systemd service to enable GRO at boot

Result: Sustained 2.2-2.3 Gbps over 40+ hours with minimal degradation

=== REPRODUCTION STEPS ===

1. Fresh install Ubuntu 25.10 with kernel 6.17.0-8 or 6.17.0-12
2. Connect system with Realtek RTL8125B NIC to 2.5G+ internet
3. Test speed immediately after boot (should be 2.3+ Gbps)
4. Run hourly speed tests for 7 days
5. Observe progressive degradation to ~1100 Mbps or worse

=== HARDWARE DETAILS ===
Motherboard: ASRock B760 Pro RS/D4 WiFi
CPU: Intel i5-13400 (16 cores)
RAM: 32GB
NIC: Realtek RTL8125B rev 05 (PCI ID: 0000:04:00.0)

• Device ID: 8125
• Vendor: Realtek Semiconductor Co., Ltd.
• Subsystem: ASRock Incorporation

Driver: r8125 version 9.016.01-NAPI (DKMS package: r8125-dkms)

Network: Spectrum 2.5 Gbps service via fiber ONT

=== KERNEL MESSAGES (relevant) ===
[ 0.682019] r8125: loading out-of-tree module taints kernel.
[ 0.688025] r8125 Ethernet controller driver 9.016.01-NAPI loaded
[ 1.235922] r8125 0000:04:00.0 enp4s0: renamed from eth0
[ 22.338777] r8125: enp4s0: link up

No errors or warnings in dmesg related to NIC.

=== DETAILED PERFORMANCE DATA ===
Before workaround applied (kernel defaults):
Thu 21:18 (55min uptime): 1226 Mbps down / 750 Mbps up
Fri 01:18 (4h56m uptime): 452 Mbps down / 923 Mbps up
Fri 07:19 (10h57m uptime): 337 Mbps down / 655 Mbps up
Fri 10:20 (13h57m uptime): 266 Mbps down / 927 Mbps up

After workaround applied:
Fri 10:24 (13h57m uptime): 1370 Mbps down / 727 Mbps up (immediate 5x improvement)
Fri 14:20 (17h58m uptime): 1099 Mbps down / 629 Mbps up
Sat 02:23 (1d6h uptime): 1401 Mbps down / 473 Mbps up
Sat 11:24 (1d15h uptime): 1327 Mbps down / 528 Mbps up

RX packet statistics:

• rx_mac_missed: 365,814 after 1.5 days
• rx_mac_missed: 1,606,047 after 7 days
• Packet loss rate: 0.075% (acceptable but indicates buffer exhaustion)

=== COMPARISON WITH KERNEL 6.8.x ===
Ubuntu 24.04 LTS with kernel 6.8.x does NOT exhibit this issue:

• GRO enabled by default
• Appropriate buffer sizes
• Stable 2.3+ Gbps performance over weeks

Regression introduced between kernel 6.8.x → 6.17.x

=== AFFECTED USERS ===
This issue affects:

• All Ubuntu 25.10 users with Realtek RTL8125/RTL8125B NICs
• Users with multi-gigabit (2.5G, 5G, 10G) internet connections
• Potentially affects Debian testing/unstable with kernel 6.17.x

Common motherboards with RTL8125B:

• ASRock B760/Z790 series
• ASUS ROG/TUF gaming boards
• MSI MAG/MPG series
• Gigabyte AORUS boards

=== PROPOSED FIX ===

1. Re-enable GRO by default for Realtek 8125 NICs in kernel
2. Restore network buffer defaults to kernel 6.8.x values
3. Update r8125 DKMS driver module parameters if needed

Alternative: Backport fixes from kernel 6.12+ LTS if issue already resolved there

=== ADDITIONAL INFORMATION ===
BIOS: Version 9.01 (latest from ASRock)
Ubuntu installation date: April 22, 2025
System type: Desktop/server (always-on)
Kernel: 6.17.0-12-generic #12-Ubuntu SMP PREEMPT_DYNAMIC

PCI device info:
04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 05)
Subsystem: ASRock Incorporation RTL8125 2.5GbE Controller
Flags: bus master, fast devsel, latency 0, IRQ 135, IOMMU group 13
I/O ports at 3000 [size=256]
Memory at 84500000 (64-bit, non-prefetchable) [size=64K]
Memory at 84510000 (64-bit, non-prefetchable) [size=16K]
Capabilities: [40] Power Management version 3
Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
Capabilities: [70] Express Endpoint, MSI 01
Capabilities: [b0] MSI-X: Enable+ Count=32 Masked-
Kernel driver in use: r8125
Kernel modules: r8125

ethtool output:

• Speed: 2500Mb/s
• Duplex: Full
• Auto-negotiation: on
• Link detected: yes
• Queues: RX:1 TX:1 (hardware limitation)
• Ring buffer: RX:1024 TX:1024 (max supported by hardware)

=== RELATED BUGS ===
Possibly related to:

• Installation on Debian 13 (Trixie) #85
• Kernel regression reports in 6.17.x series

=== TAGS ===
regression kernel-6.17 realtek rtl8125 network-performance high-speed-networking

[willaraujo]

UPDATE: Workaround Does Not Fully Resolve Issue

The workaround (enabling GRO + increasing buffers) significantly improves performance but does not eliminate degradation.

Performance With Workaround Applied:

• Fresh boot: 2276 Mbps download
• After 1 day: 1200-1400 Mbps
• After 7 days: 1100 Mbps (52% degradation still occurs)

Comparison:
WITHOUT workaround:

• Fresh boot: 2276 Mbps
• After 13 hours: 266 Mbps (88% loss)

WITH workaround:

• Fresh boot: 2276 Mbps
• After 7 days: 1100 Mbps (52% loss)

RX Packet Misses Continue Accumulating:

• 365,814 after 1.5 days
• 1,606,047 after 7 days
• Rate: ~230,000 packets/day

Analysis:
The workaround makes the problem less severe but doesn't eliminate it. This suggests:

1. Driver memory leak or resource exhaustion over extended uptime
2. Single-queue limitation under sustained multi-gig load (1 RX/1 TX queue can't keep up)
3. Additional kernel 6.17.x regressions beyond GRO/buffer defaults

Important Note:
This behavior did not occur on Ubuntu 24.04 LTS (kernel 6.8.x). The same hardware sustained full 2.3+ Gbps indefinitely without degradation.

The workaround is a mitigation, not a fix. Reboot remains the only way to restore full speed.