RPi5 internal Wi-Fi: WPA2-Enterprise (802.1X) fails at PTK installation with hostapd

[Vig122001]

On Raspberry Pi 5, using the internal Wi-Fi (cyw43 driver), WPA2-Enterprise authentication with FreeRADIUS fails after successful EAP authentication. The client never finishes the 4-way handshake because the PTK key installation fails in the driver. WPA2-PSK mode works fine.

This makes it impossible to deploy a production WPA2-Enterprise AP on the RPi5 internal Wi-Fi.

Environment

• Hardware: Raspberry Pi 5
• Wi-Fi interface: internal (wlan0, cyw43xx chipset)
• OS/Distro: Raspberry Pi OS Bookworm (64-bit)
• Kernel version: Linux codescandev.vmeasure.systems 6.6.62+rpt-rpi-2712 libilclient.a in hardffp not hardfp #1 SMP PREEMPT Debian 1:6.6.62-1+rpt1 (2024-11-25) aarch64 GNU/Linux
• Firmware version: Copyright (c) 2012 Broadcom
• version 3c4fc886 (release) (embedded)
• hostapd version: hostapd v2.10

Steps to Reproduce

• Install FreeRADIUS and hostapd.
• Configure WPA2-Enterprise AP with ieee8021x=1, auth_server_addr, and RADIUS settings.
• Connect a laptop client (tested with ).
• Observe authentication succeeds at RADIUS, but handshake fails.

Expected Behavior

• Client should successfully associate and complete 4-way handshake. PTK keys should install.

Actual Behavior

• RADIUS authentication succeeds (radtest also confirms working).
• 4-way handshake fails during PTK installation.
• Client disconnects immediately after authentication.

Logs

dmesg

[58904.370699] brcmfmac: brcmf_set_channel: set chanspec 0xd02e fail, reason -52
[58906.063029] brcmfmac: brcmf_set_channel: set chanspec 0xd090 fail, reason -52

hostapd log snippet

nl80211: New station 2c:cf:67:40:f8:bc
nl80211: Assoc Req IEs - hexdump(len=152): 00 0b 45 41 50 5f 50 45 41 50 5f 41 50 01 08 8c 12 98 24 b0 48 60 6c 21 02 03 14 24 06 24 04 34 04 64 0b 30 14 01 00 00 0f ac 04 01 00 00 0f ac 04 01 00 00 0f ac 01 80 00 2d 1a 63 00 17 ff 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 7f 08 00 00 08 00 00 00 00 40 bf 0c 32 50 80 0f fe ff 00 00 fe ff 00 00 dd 1e 00 90 4c 33 63 00 17 ff 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 dd 09 00 10 18 02 00 00 10 00 00
wlan0: Event ASSOC (0) received
wlan0: STA 2c:cf:67:40:f8:bc IEEE 802.11: associated
STA included RSN IE in (Re)AssocReq
  New STA
ap_sta_add: register ap_handle_timer timeout for 2c:cf:67:40:f8:bc (300 seconds - ap_max_inactivity)
nl80211: Set STA flags - ifname=wlan0 addr=2c:cf:67:40:f8:bc total_flags=0x60 flags_or=0x0 flags_and=0xfffffff1 authorized=0
wlan0: STA 2c:cf:67:40:f8:bc WPA: event 1 notification
wpa_driver_nl80211_set_key: ifindex=6 (wlan0) alg=0 addr=0x555652cb8e10 key_idx=0 set_tx=1 seq_len=0 key_len=0 key_flag=0x20
nl80211: DEL_KEY
   addr=2c:cf:67:40:f8:bc
   pairwise key
nl80211: set_key failed; err=-22 Invalid argument
RSN: PTK removal from the driver failed
IEEE 802.1X: Ignore STA - 802.1X not enabled or forced for WPS
wlan0: STA 2c:cf:67:40:f8:bc WPA: start authentication
WPA: 2c:cf:67:40:f8:bc WPA_PTK entering state INITIALIZE
wpa_driver_nl80211_set_key: ifindex=6 (wlan0) alg=0 addr=0x555652cb8e10 key_idx=0 set_tx=1 seq_len=0 key_len=0 key_flag=0x20
nl80211: DEL_KEY
   addr=2c:cf:67:40:f8:bc
   pairwise key
nl80211: set_key failed; err=-22 Invalid argument
RSN: PTK removal from the driver failed
WPA: 2c:cf:67:40:f8:bc WPA_PTK_GROUP entering state IDLE
WPA: 2c:cf:67:40:f8:bc WPA_PTK entering state AUTHENTICATION
WPA: 2c:cf:67:40:f8:bc WPA_PTK entering state AUTHENTICATION2
WPA: Re-initialize GMK/Counter on first station
Get randomness: len=32 entropy=1
GMK - hexdump(len=32): [REMOVED]
Get randomness: len=32 entropy=0
Key Counter - hexdump(len=32): [REMOVED]
Get randomness: len=16 entropy=0
GTK - hexdump(len=16): [REMOVED]
wpa_driver_nl80211_set_key: ifindex=6 (wlan0) alg=3 addr=0x55564fff4b78 key_idx=1 set_tx=1 seq_len=0 key_len=16 key_flag=0x1a
nl80211: NEW_KEY
nl80211: KEY_DATA - hexdump(len=16): [REMOVED]
   broadcast key
nl80211: NL80211_CMD_SET_KEY - default key
Get randomness: len=32 entropy=0
WPA: Assign ANonce - hexdump(len=32): 8b b3 45 a3 d5 cf fd d9 a6 54 01 9c da 52 85 89 70 6b 37 ef 28 6f 96 8e da 0f 97 11 0d a5 d3 56
WPA: 2c:cf:67:40:f8:bc WPA_PTK entering state INITPMK
AUTH_GET_MSK: Key is null, eapol_sm: (nil)
WPA: Could not get PMK, get_msk: 0x55564fed2b10
wlan0: STA 2c:cf:67:40:f8:bc WPA: WPA_PTK: sm->Disconnect
WPA: 2c:cf:67:40:f8:bc WPA_PTK entering state DISCONNECT
wpa_sta_disconnect STA 2c:cf:67:40:f8:bc (reason 2)
hostapd_wpa_auth_disconnect: WPA authenticator requests disconnect: STA 2c:cf:67:40:f8:bc reason 2
wlan0: ap_sta_disconnect addr 2c:cf:67:40:f8:bc reason=2
nl80211: sta_remove -> DEL_STATION wlan0 2c:cf:67:40:f8:bc --> 0 (Success)
nl80211: Set STA flags - ifname=wlan0 addr=2c:cf:67:40:f8:bc total_flags=0x0 flags_or=0x0 flags_and=0xfffffff1 authorized=0
wlan0: STA 2c:cf:67:40:f8:bc WPA: event 3 notification
wpa_driver_nl80211_set_key: ifindex=6 (wlan0) alg=0 addr=0x555652cb8e10 key_idx=0 set_tx=1 seq_len=0 key_len=0 key_flag=0x20
nl80211: DEL_KEY
   addr=2c:cf:67:40:f8:bc
   pairwise key
nl80211: set_key failed; err=-22 Invalid argument
RSN: PTK removal from the driver failed
wlan0: ap_sta_disconnect: reschedule ap_handle_timer timeout for 2c:cf:67:40:f8:bc (5 seconds - AP_MAX_INACTIVITY_AFTER_DEAUTH)

Notes

WPA2-PSK works fine on the same hardware.
WPA2-Enterprise works fine with external USB Wi-Fi adapters (rtl8812bu).
This seems specific to the brcmfmac driver/firmware on RPi5.

Request

Please advise whether this is a known limitation of the RPi5 internal Wi-Fi driver/firmware, or if there is a workaround or patch in progress. Happy to provide more detailed logs or run test builds if needed.