24.11 dpdk + master分支编译dperf bond网卡测试异常

[Laplacence]

#bond_client.conf with different NIC
mode            client

tx_burst                    128
launch_num                  10
cpu                         112

payload_size                1
duration                    60s

cps                         10000


#port           pci                                  addr         gateway
#               802.ad policy3+4
port            bond4:2(0000:81:00.0,0000:25:00.0)  10.11.100.115  10.11.100.1 78:dd:33:7a:f3:52

client          10.11.100.115	    1

#               addr_start          num
server          10.11.2.118         1
#               port_start          num
listen          50000               1

#server
mode            server
tx_burst        128
cpu             96

duration        10m
payload_size    1
#keepalive       5s

#numa0
#port           pci             addr         gateway
port            0000:a1:00.0    10.11.2.118    10.11.2.1 78:dd:33:7a:f3:52


#               addr_start      num
client          10.11.100.115   1

#               addr_start      num
server          10.11.2.118     1

#               port_start      num
listen          50000           1

# ../dpdk/usertools/dpdk-devbind.py -s

Network devices using kernel driver
===================================
0000:01:00.0 'MT27800 Family [ConnectX-5] 1017' numa_node=3 if=enp1s0f0np0 drv=mlx5_core unused=vfio-pci 
0000:01:00.1 'MT27800 Family [ConnectX-5] 1017' numa_node=3 if=enp1s0f1np1 drv=mlx5_core unused=vfio-pci 
0000:25:00.0 'MT27800 Family [ConnectX-5] 1017' numa_node=2 if=enp37s0f0np0 drv=mlx5_core unused=vfio-pci 
0000:25:00.1 'MT27800 Family [ConnectX-5] 1017' numa_node=2 if=enp37s0f1np1 drv=mlx5_core unused=vfio-pci 
0000:61:00.0 'Ethernet Controller 10G X550T 1563' numa_node=0 if=enp97s0f0 drv=ixgbe unused=vfio-pci *Active*
0000:61:00.1 'Ethernet Controller 10G X550T 1563' numa_node=0 if=enp97s0f1 drv=ixgbe unused=vfio-pci 
0000:81:00.0 'MT27800 Family [ConnectX-5] 1017' numa_node=7 if=enp129s0f0np0 drv=mlx5_core unused=vfio-pci 
0000:81:00.1 'MT27800 Family [ConnectX-5] 1017' numa_node=7 if=enp129s0f1np1 drv=mlx5_core unused=vfio-pci 
0000:a1:00.0 'MT27800 Family [ConnectX-5] 1017' numa_node=6 if=enp161s0f0np0 drv=mlx5_core unused=vfio-pci 
0000:a1:00.1 'MT27800 Family [ConnectX-5] 1017' numa_node=6 if=enp161s0f1np1 drv=mlx5_core unused=vfio-pci 

# ip a | grep Northbond
2: enp37s0f0np0: <BROADCAST,MULTICAST,SLAVE> mtu 1500 qdisc mq master Northbond state DOWN group default qlen 1000
4: enp37s0f1np1: <BROADCAST,MULTICAST,SLAVE> mtu 1500 qdisc mq master Northbond state DOWN group default qlen 1000
6: enp1s0f0np0: <BROADCAST,MULTICAST,SLAVE> mtu 9000 qdisc mq master Northbond state DOWN group default qlen 1000
7: enp1s0f1np1: <BROADCAST,MULTICAST,SLAVE> mtu 9000 qdisc mq master Northbond state DOWN group default qlen 1000
8: enp161s0f0np0: <BROADCAST,MULTICAST,SLAVE> mtu 9000 qdisc mq master Northbond state DOWN group default qlen 1000
9: enp161s0f1np1: <BROADCAST,MULTICAST,SLAVE> mtu 9000 qdisc mq master Northbond state DOWN group default qlen 1000
10: enp129s0f0np0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master Northbond state UP group default qlen 1000
11: enp129s0f1np1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master Northbond state UP group default qlen 1000
12: Northbond: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 9000 qdisc noqueue state UP group default qlen 1000
    inet 10.11.100.115/24 brd 10.11.100.255 scope global Northbond

# ./build/dperf -c test/bond/bond_client.conf 
slaves not in same socket
 port init fail
dpdk init fail

根据以上数据看，暂时已知

• 跨不同卡测试失败，无法启动。
• 跨numa测试失败，无法启动。
• 同卡双口测试可以启动进程，但仍有rstRx计数。
• 三端口测试失败，无法启动。

PS: 其他测试场景的client端conf

#two port with same NIC
mode            client

tx_burst                    128
launch_num                  10
cpu                         112

payload_size                1
duration                    60s

cps                         10000
# cc                          30m
# keepalive                   60s


#port           pci                                  addr         gateway
#               802.ad policy3+4
port            bond4:2(0000:81:00.0,0000:81:00.1)  10.11.100.115  10.11.100.1 78:dd:33:7a:f3:52

client          10.11.100.115	    1

#               addr_start          num
server          10.11.2.118         1
#               port_start          num
listen          50000               1

# three port with two NICs
mode            client

tx_burst                    128
launch_num                  10
cpu                         112

payload_size                1
duration                    60s

cps                         10000
# cc                          30m
# keepalive                   60s


#port           pci                                  addr         gateway
#               802.ad policy3+4
port            bond4:2(0000:81:00.0,0000:81:00.1,0000:25:00.0)  10.11.100.115  10.11.100.1 78:dd:33:7a:f3:52

client          10.11.100.115	    1

#               addr_start          num
server          10.11.2.118         1
#               port_start          num
listen          50000               1

[pengjianzhang]

1. 你之前用的是dperf 什么版本，则个版本测试是否正常？
2. 这几个问题麻烦详细描述一下

• 跨不同卡测试失败，无法启动。
• 跨numa测试失败，无法启动。
• 同卡双口测试可以启动进程，但仍有rstRx计数。
• 三端口测试失败，无法启动。

最好预约腾讯会议沟通一下，pengjianzhang@gmail.com

[Laplacence]

也会出现一些

[pengjianzhang]

cps设置为1K呢？
可能是源IP太少，连接耗尽了

[Laplacence]

#server result
Total Numbers:
pktRx   193,183            pktTx    193,576            bitsRx   93,796,008         bitsTx  88,383,560         dropTx  0
tcpRx   193,044            tcpTx    193,550            udpRx    0                  udpTx   0
arpRx   85                 arpTx    2                  icmpRx   0                  icmpTx  0
tosRx   0                  otherRx  54                 badRx    0
synRx   65,880             synTx    66,412             finRx    63,569             finTx   63,569             rstRx   0          rstTx 24
synRt   2,760              finRt    0                  ackRt    0                  pushRt  0
tcpDrop 2                  udpDrop  0                  ackDup   0
skOpen  63,652             skClose  63,652             skCon    0                  skErr   83
tcpReq  63,569             tcpRsp   63,569
ierrors 0                  oerrors  0                  imissed  0

#client result
Total Numbers:
pktRx   191,065            pktTx    240,200            bitsRx   92,747,640         bitsTx  112,328,200        dropTx  0                 
tcpRx   190,952            tcpTx    236,737            udpRx    1                  udpTx   0                 
arpRx   16                 arpTx    2                  icmpRx   0                  icmpTx  0                 
tosRx   0                  otherRx  96                 badRx    0                 
synRx   63,797             synTx    109,487            finRx    63,569             finTx   63,569             rstRx   0          rstTx 17        
synRt   43,487             finRt    0                  ackRt    0                  pushRt  90                
tcpDrop 159                udpDrop  1                  ackDup   0                 
skOpen  66,000             skClose  66,000             skCon    0                  skErr   2,431              rtt(us) 477.7     
tcpReq  63,681             tcpRsp   63,569            
ierrors 0                  oerrors  0                  imissed  0

cps1000的结果，两边的统计数据不一致是正常的吗？

[pengjianzhang]

有 skErr 就是不正常

[pengjianzhang]

试试把 src/port.c 227行注释掉看看

225 } else if (port->socket != socket) {
226 printf("slaves not in same socket\n ");
227 // return -1;
228 }
229 }