RDMA not detected on Intel(R) E810-DA2

[cmontemuino]

I have multiple servers in a K8S cluster with the following profiles:

1. Lenovo SR630 V2 - 2x ThinkSystem Mellanox ConnectX-6 Lx 10/25GbE SFP28 2-port OCP Ethernet Adapter
2. Lenovo SR670 V2 - 2x Intel(R) E810-DA2 10/25GbE SFP28 2-Port OCP Ethernet Adapter

I get label rdma.capable: true for the former, but nodes from second profile do not get the label.

According to this, the NICs from second profile are RDMA capable.

If I'm not mistaken, RDMA detection is done here:
https://github.com/kubernetes-sigs/node-feature-discovery/blob/master/source/custom/static_features.go

These Intel cards have device ID 159b:

$ lspci -nn | grep -i ethernet
4b:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller E810-XXV for SFP [8086:159b] (rev 02)
4b:00.1 Ethernet controller [0200]: Intel Corporation Ethernet Controller E810-XXV for SFP [8086:159b] (rev 02)

System recognizes both NICs as RDMA Channel Adapters (ca):

$ rdma dev show
0: irdma0: node_type ca fw 1.69 node_guid b696:91ff:fef9:6f5c sys_image_guid b696:91ff:fef9:6f5c
1: irdma1: node_type ca fw 1.69 node_guid b696:91ff:fef9:6f5d sys_image_guid b696:91ff:fef9:6f5d

$ rdma link show
link irdma0/1 state ACTIVE physical_state LINK_UP netdev ens27f0
link irdma1/1 state ACTIVE physical_state LINK_UP netdev ens27f1

I wonder if a check against "159b" is missing, or am I missing something?

[marquiz]

Thanks @cmontemuino for reaching out. Your analysis is correct: rdma detection is done in the static_features.go BUT it is missing the rule(s) to detect other than Mellanox NICs. A quick resolution for your case/device would be to add a rule matching vendor (8086) & device id (159b). However, it would be nice to add detection for other intel rdma-capable nics, too, but that would require some investigation. I'm not familiar with or have access to the devices.

We could start by adding this one device that we know of and then add other device ids later.

@ozhuraki would you mind picking this up?

[marquiz]

@cmontemuino as a quick fix you can create a NodeFeatureRule to create the missing label.

Something like this (not tested)

apiVersion: nfd.kubernetes.io/v1alpha1
kind: NodeFeatureRule
metadata:
  name: intel-rdma
spec:
  rules:
    - name: "intel-rdma"
      labels:
        feature.node.kubernetes.io/rdma.capable: "true"
      matchFeatures:
        - feature: pci.device
          matchExpressions:
            vendor: { op: In, value: ["8086"] }
            device: { op: In, value: ["159b"] }

[cmontemuino]

Thanks @marquiz! I'll use the NodeFeatureRule in the meantime.
Regarding code implementation, I can also give it a try, or I can test it in our hardware.