msquic performance test failed to meet expectations

[Nacho-Neko]

Why can't I reach the speed limit when using msquic for throughput testing?
server bandwidth is 1G client bandwidth is 1G RTT is 6ms.

command :
root@VM-32-13-ubuntu:/home/ubuntu/msquic/artifacts/bin/linux/x64_Debug_quictls# ./secnetperf -target:IP -exec:maxtput -up:10s -ptput:1 -streams:10 -ptput:1 -sendbuf:1 -io:xdp -cc:bbr -iosize:131072 -ptput:1 -pconn:1
Started!

Connection Statistics:
RTT 6270 us
MinRTT 5654 us
EcnCapable 0
SendTotalPackets 83739
SendSuspectedLostPackets 41748
SendSpuriousLostPackets 5063
SendCongestionCount 18116
SendEcnCongestionCount 0
RecvTotalPackets 27398
RecvReorderedPackets 147
RecvDroppedPackets 0
RecvDuplicatePackets 0
RecvDecryptionFailures 0
Connection TLS Info:
TlsProtocolVersion 0x3000
CipherAlgorithm 0x6610
CipherStrength 256
Hash 0x800d
HashStrength 0
KeyExchangeAlgorithm 0
KeyExchangeStrength 0
CipherSuite 0x1302
TlsGroup 0
Result: Upload 33137 kbps.
App Main returning status 0

Both Qperf and iperf3 work properly and can achieve 1G bandwidth.

[Nacho-Neko]

-tcp:1 option
it returned to normal, and the speed successfully reached 1G bandwidth.

[Nacho-Neko]

[guhetier]

SendTotalPackets 83739
SendSuspectedLostPackets 41748
SendSpuriousLostPackets 5063

uint64_t SendSpuriousLostPackets;       // Actual lost is (SuspectedLostPackets - SpuriousLostPackets)

This shows close to 50% loss rate, you should probably start by investigating this.
Otherwise, https://github.com/microsoft/msquic/blob/main/docs/TroubleshootingGuide.md#trouble-shooting-a-performance-issue may contain some useful information.

[Nacho-Neko]

Yes, I tested the case without XDP, and there was no performance difference. XDP works perfectly.

[guhetier]

So, both with and without XDP, you are seeing poor performance?
Can you please collect logs using the instructions from the doc? I can try to have a look although I don't know when I'll have the cycles.

The first step would be to understand why we are seeing so many losses, so if you get logs, you can look through them for anything related to losses / packet drops.

[Nacho-Neko]

it seems to be related to the msquic inability to negotiate the MTU.
I found that qperf can handle automatic negotiation MTU.

[Nacho-Neko]

Should I submit an issue to follow up on this?

[guhetier]

MsQuic should run an MTU discovery.
If you can capture logs of your repro with MsQuic as well as the MTU used by qperf, please open an issue and we can have a look.