The slabinfo collector produces duplicate metrics on a host with Mellanox cards #3506
Description
The slabinfo collector in node_exporter 1.10.2 produces constant errors about duplicate metric names and values on a Linux host with Mellanox ConnectX-6 Ethernet and Infiniband cards. There are indeed slabs with duplicate names in the /proc/slabinfo file.
The error:
time=2025-12-17T06:08:16.966Z level=ERROR source=http.go:175 msg="error gathering metrics: 50 error(s) occurred:\n* [from Gatherer #2[] collected metric \"node_slabinfo_active_objects\" { label:{name:\"slab\" value:\"mlx5_fs_ftes\"} gauge:{value:44}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_objects\" { label:{name:\"slab\" value:\"mlx5_fs_ftes\"} gauge:{value:44}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_object_size_bytes\" { label:{name:\"slab\" value:\"mlx5_fs_ftes\"} gauge:{value:736}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_objects_per_slab\" { label:{name:\"slab\" value:\"mlx5_fs_ftes\"} gauge:{value:44}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_pages_per_slab\" { label:{name:\"slab\" value:\"mlx5_fs_ftes\"} gauge:{value:8}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_active_objects\" { label:{name:\"slab\" value:\"mlx5_fs_fgs\"} gauge:{value:42}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_objects\" { label:{name:\"slab\" value:\"mlx5_fs_fgs\"} gauge:{value:42}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_object_size_bytes\" { label:{name:\"slab\" value:\"mlx5_fs_fgs\"} gauge:{value:776}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_objects_per_slab\" { label:{name:\"slab\" value:\"mlx5_fs_fgs\"} gauge:{value:42}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_pages_per_slab\" { label:{name:\"slab\" value:\"mlx5_fs_fgs\"} gauge:{value:8}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_active_objects\" { label:{name:\"slab\" value:\"mlx5_fs_ftes\"} gauge:{value:44}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_objects\" { label:{name:\"slab\" value:\"mlx5_fs_ftes\"} gauge:{value:44}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_object_size_bytes\" { label:{name:\"slab\" value:\"mlx5_fs_ftes\"} gauge:{value:736}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_objects_per_slab\" { label:{name:\"slab\" value:\"mlx5_fs_ftes\"} gauge:{value:44}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_pages_per_slab\" { label:{name:\"slab\" value:\"mlx5_fs_ftes\"} gauge:{value:8}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_active_objects\" { label:{name:\"slab\" value:\"mlx5_fs_fgs\"} gauge:{value:42}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_objects\" { label:{name:\"slab\" value:\"mlx5_fs_fgs\"} gauge:{value:42}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_object_size_bytes\" { label:{name:\"slab\" value:\"mlx5_fs_fgs\"} gauge:{value:776}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_objects_per_slab\" { label:{name:\"slab\" value:\"mlx5_fs_fgs\"} gauge:{value:42}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_pages_per_slab\" { label:{name:\"slab\" value:\"mlx5_fs_fgs\"} gauge:{value:8}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_active_objects\" { label:{name:\"slab\" value:\"mlx5_fs_ftes\"} gauge:{value:44}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_objects\" { label:{name:\"slab\" value:\"mlx5_fs_ftes\"} gauge:{value:44}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_object_size_bytes\" { label:{name:\"slab\" value:\"mlx5_fs_ftes\"} gauge:{value:736}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_objects_per_slab\" { label:{name:\"slab\" value:\"mlx5_fs_ftes\"} gauge:{value:44}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_pages_per_slab\" { label:{name:\"slab\" value:\"mlx5_fs_ftes\"} gauge:{value:8}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_active_objects\" { label:{name:\"slab\" value:\"mlx5_fs_fgs\"} gauge:{value:42}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_objects\" { label:{name:\"slab\" value:\"mlx5_fs_fgs\"} gauge:{value:42}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_object_size_bytes\" { label:{name:\"slab\" value:\"mlx5_fs_fgs\"} gauge:{value:776}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_objects_per_slab\" { label:{name:\"slab\" value:\"mlx5_fs_fgs\"} gauge:{value:42}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_pages_per_slab\" { label:{name:\"slab\" value:\"mlx5_fs_fgs\"} gauge:{value:8}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_active_objects\" { label:{name:\"slab\" value:\"mlx5_fs_ftes\"} gauge:{value:396}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_objects\" { label:{name:\"slab\" value:\"mlx5_fs_ftes\"} gauge:{value:396}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_object_size_bytes\" { label:{name:\"slab\" value:\"mlx5_fs_ftes\"} gauge:{value:736}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_objects_per_slab\" { label:{name:\"slab\" value:\"mlx5_fs_ftes\"} gauge:{value:44}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_pages_per_slab\" { label:{name:\"slab\" value:\"mlx5_fs_ftes\"} gauge:{value:8}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_active_objects\" { label:{name:\"slab\" value:\"mlx5_fs_fgs\"} gauge:{value:42}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_objects\" { label:{name:\"slab\" value:\"mlx5_fs_fgs\"} gauge:{value:42}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_object_size_bytes\" { label:{name:\"slab\" value:\"mlx5_fs_fgs\"} gauge:{value:776}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_objects_per_slab\" { label:{name:\"slab\" value:\"mlx5_fs_fgs\"} gauge:{value:42}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_pages_per_slab\" { label:{name:\"slab\" value:\"mlx5_fs_fgs\"} gauge:{value:8}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_active_objects\" { label:{name:\"slab\" value:\"mlx5_fs_ftes\"} gauge:{value:352}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_objects\" { label:{name:\"slab\" value:\"mlx5_fs_ftes\"} gauge:{value:352}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_object_size_bytes\" { label:{name:\"slab\" value:\"mlx5_fs_ftes\"} gauge:{value:736}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_objects_per_slab\" { label:{name:\"slab\" value:\"mlx5_fs_ftes\"} gauge:{value:44}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_pages_per_slab\" { label:{name:\"slab\" value:\"mlx5_fs_ftes\"} gauge:{value:8}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_active_objects\" { label:{name:\"slab\" value:\"mlx5_fs_fgs\"} gauge:{value:84}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_objects\" { label:{name:\"slab\" value:\"mlx5_fs_fgs\"} gauge:{value:84}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_object_size_bytes\" { label:{name:\"slab\" value:\"mlx5_fs_fgs\"} gauge:{value:776}} was collected before with the same name and label values\n* [from Gatherer #2[] collected metric \"node_slabinfo_objects_per_slab\" { label:{name:\"slab\" value:\"mlx5_fs_fgs\"} gauge:{value:42}} was collected before with the same name and label values\n* [from Gatherer #2] collected metric \"node_slabinfo_pages_per_slab\" { label:{name:\"slab\" value:\"mlx5_fs_fgs\"} gauge:{value:8}} was collected before with the same name and label values"
Some details about the system in question:
# cat /etc/os-release
NAME="AlmaLinux"
VERSION="9.4 (Seafoam Ocelot)"
ID="almalinux"
ID_LIKE="rhel centos fedora"
VERSION_ID="9.4"
PLATFORM_ID="platform:el9"
PRETTY_NAME="AlmaLinux 9.4 (Seafoam Ocelot)"
ANSI_COLOR="0;34"
LOGO="fedora-logo-icon"
CPE_NAME="cpe:/o:almalinux:almalinux:9::baseos"
HOME_URL="https://almalinux.org/"
DOCUMENTATION_URL="https://wiki.almalinux.org/"
BUG_REPORT_URL="https://bugs.almalinux.org/"
ALMALINUX_MANTISBT_PROJECT="AlmaLinux-9"
ALMALINUX_MANTISBT_PROJECT_VERSION="9.4"
REDHAT_SUPPORT_PRODUCT="AlmaLinux"
REDHAT_SUPPORT_PRODUCT_VERSION="9.4"
SUPPORT_END=2032-06-01
# uname -r
5.14.0-427.42.1.el9_4.x86_64
# lspci | grep -i mellanox
0000:19:00.0 Ethernet controller: Mellanox Technologies MT2894 Family [ConnectX-6 Lx]
0000:19:00.1 Ethernet controller: Mellanox Technologies MT2894 Family [ConnectX-6 Lx]
0000:41:00.0 Infiniband controller: Mellanox Technologies MT28908 Family [ConnectX-6]
0000:41:00.1 Infiniband controller: Mellanox Technologies MT28908 Family [ConnectX-6]
0000:9b:00.0 Infiniband controller: Mellanox Technologies MT28908 Family [ConnectX-6]
0000:9b:00.1 Infiniband controller: Mellanox Technologies MT28908 Family [ConnectX-6]
# cat /proc/slabinfo | grep mlx
mlx5_fs_ftes 44 44 736 44 8 : tunables 0 0 0 : slabdata 1 1 0
mlx5_fs_fgs 42 42 776 42 8 : tunables 0 0 0 : slabdata 1 1 0
mlx5_fs_ftes 44 44 736 44 8 : tunables 0 0 0 : slabdata 1 1 0
mlx5_fs_fgs 42 42 776 42 8 : tunables 0 0 0 : slabdata 1 1 0
mlx5_fs_ftes 44 44 736 44 8 : tunables 0 0 0 : slabdata 1 1 0
mlx5_fs_fgs 42 42 776 42 8 : tunables 0 0 0 : slabdata 1 1 0
mlx5_fs_ftes 44 44 736 44 8 : tunables 0 0 0 : slabdata 1 1 0
mlx5_fs_fgs 42 42 776 42 8 : tunables 0 0 0 : slabdata 1 1 0
mlx5_fs_ftes 396 396 736 44 8 : tunables 0 0 0 : slabdata 9 9 0
mlx5_fs_fgs 42 42 776 42 8 : tunables 0 0 0 : slabdata 1 1 0
mlx5_fs_ftes 352 352 736 44 8 : tunables 0 0 0 : slabdata 8 8 0
mlx5_fs_fgs 84 84 776 42 8 : tunables 0 0 0 : slabdata 2 2 0