Performance Tuning

[semigodking]

hi author, do you have any recommended configuration for performance? With the same environment, the download speed to easily reach 20MB. But, with kcptun-libev, no matter how the parameters are tuned, the maximum speed is ~6MB and is not stable. Yes, the CPU utilization decreased a lot.
I tried the following settings:

• Make sure mtu is same as kcptun
• Increase snd/rcv buffer in both sides
• Increase udp buffer sizes
• Reduce interval
• Enable/Disable resend
• Enable/Disable NC

thank you!

[semigodking]

BTW, no encryption is enabled.

[hexian000]

hi author, do you have any recommended configuration for performance? With the same environment, the download speed to easily reach 20MB. But, with kcptun-libev, no matter how the parameters are tuned, the maximum speed is ~6MB and is not stable. Yes, the CPU utilization decreased a lot. I tried the following settings:

* Make sure mtu is same as kcptun

* Increase snd/rcv buffer in both sides

* Increase udp buffer sizes

* Reduce interval

* Enable/Disable resend

* Enable/Disable NC

thank you!

Hi @semigodking, kcptun-libev runs at 1Gbps in my benchmarks. I would like to try to reproduce this issue, can you share more details about the parameters you've tried and the network latency?

[semigodking]

Thanks for quick response. Here is what I can share:

{
    "listen": "127.0.0.1:6000",
    "kcp_connect": "123.123.123.123:6000",
    "kcp": {
        "mtu": 1440,
        "sndwnd": 768,
        "rcvwnd": 10240,
        "nodelay": 1,
        "interval": 100,
        "resend": 0,
        "nc": 1
    },
    "udp": {
        "sndbuf": 10485760,
        "rcvbuf": 10485760
    }
}

The server side config is almost the same with sndwnd & rcvwnd reversed.
The latency is about 220ms. Packet drop rate is lower than 1%.

[hexian000]

Thanks for quick response. Here is what I can share:

{
    "listen": "127.0.0.1:6000",
    "kcp_connect": "123.123.123.123:6000",
    "kcp": {
        "mtu": 1440,
        "sndwnd": 768,
        "rcvwnd": 10240,
        "nodelay": 1,
        "interval": 100,
        "resend": 0,
        "nc": 1
    },
    "udp": {
        "sndbuf": 10485760,
        "rcvbuf": 10485760
    }
}

The server side config is almost the same with sndwnd & rcvwnd reversed. The latency is about 220ms. Packet drop rate is lower than 1%.

Hi @semigodking , are you using a real network or some emulator?
Some network emulator may have a limited queue and drop more packets than expected under heavy load.
kcptun-libev sends more and ticks less. Therefore the packets are more likely to overflow the queue.

Please follow the instructions in observability section in README.

And find this line from the curl output:

[total] tcp: 301 MiB, 15.9 GiB; kcp: 17.8 GiB, 664 MiB; pkt: 18.4 GiB, 724 MiB

I believe by comparing the pkt: values from both peer, we can find out the real packet loss rate.

[semigodking]

After changing qdisc of interface from fq to fq_codel. It now has similar performance as kcptun! Great! Thank you!

[hexian000]

After changing qdisc of interface from fq to fq_codel. It now has similar performance as kcptun! Great! Thank you!

Could you kindly provide information about the problem that was found? It would greatly assist us to have this knowledge.

[semigodking]

I just noticed this difference when I was trying to measure transfer speed of my VM. Two VMs has different behavior. When comparing output from ip link I noticed difference of qdisc. One VM has qdisc set to fq, the transfer speed is not stable and is bouncing. This behavior is similar to kcptun-libev when I did tests.
Another VM has qdisc set to fq_codel, the transfer speed is very stable at almost maximum speed.
So, after changing qdisc to fq_codel, kcptun-libev could reach maximum speed.

The same difference can also be observed if I do tests with iperf3. With fq, if the target bandwidth is higher than actual bandwidth available, iperf3 reports high lost rate. But, with fq_codel, even the target bandwidth is much higher than actual bandwidth available, iperf3 only transfer at speed of available bandwidth, almost 0 drop rate reported.