Download speed is very slow

[cecchisandrone]

I'm trying to download a 7MB file and it takes more than 10 minutes. Are there any specific settings to set for large files?

tftp> get /test/smart/63.58.11.10.rom
Received 7399773 bytes in 655.7 seconds

[hanazuki]

Hi. I usually use s3tftpd to download firmware to my network equipment but have never been troubled with download speed. (My small switch downloads a ~7MB image in ~10s over 1GbE LAN). So I have no clear idea about the degraded performance you are experiencing and have prepared no such parameter for s3tftpd to tune TFTP communication.

I assume you use the prebuilt container image on Docker since #1. When I tested the --single-port option for containers, I observed small performance degradation when the option is enabled, but I suppose this is ignorable.

Also, I found that deploying s3tftpd on Docker moderately impacts on transfer speed. I think this is due to the network overhead of the container infrastructure (proxying, address translation, and firewall). As TFTP uses UDP, it is significantly affected by network latency. However, when I tested the image on Docker for Mac, I could have downloaded a 10MB image under a minute. So this may not be the only reason.

It is always recommend to deploy your TFTP server as close as possible to your clients and use stable connection to reduce network latency and speed up the transfer. But you might know this...

Another option to increase TFTP transfer speed is to employ the blocksize extension (Client support is limited, though). I will try to add a switch to accept the extension.

[hanazuki]

v0.4.0 includes --blocksize and --anticipate option. I expect that tuning the options can improve download performance. (Please refer to the manpage for details). Your feedback will be, as always, appreciated.

[cecchisandrone]

Thank you @hanazuki for your fast feedback. I'll try this immediately!

[cecchisandrone]

I noticed a slightly improvement with blocksize but speed is always very low. I don't know if it depends on the clients. I tried macOS and Linux, it's the same.

[hanazuki]

What kind of Docker server setup are you using? Is it slow even when the client is on the same machine as the Docker server?

[cecchisandrone]

I tried on the same machine and it is fast. I'm running it with docker-compose

Containers: 15
 Running: 11
 Paused: 0
 Stopped: 4
Images: 109
Server Version: 18.06.1-ce
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 468a545b9edcd5932818eb9de8e72413e616e86e
runc version: 69663f0bd4b60df09991c08812a60108003fa340
init version: fec3683
Security Options:
 apparmor
 seccomp
  Profile: default
Kernel Version: 4.15.0-23-generic
Operating System: Ubuntu 18.04.1 LTS
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 15.3GiB
Name: cicd
ID: S4XW:GQRI:4GAT:HMH5:344H:ZLQO:TNF7:6CTE:LZ2O:6GW3:UAH4:SSFO
Docker Root Dir: /mnt/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

WARNING: No swap limit support
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled

[hanazuki]

Then it seems that the problem comes from the network between server and client.
How long is the latency between them -- that is, ping from the client machine to the machine with the Docker server.

[cecchisandrone]

Bandwidth is ok, I have other server running there with no problem. Ping with 40ms

[hanazuki]

Wow! 40 ms latency is quite large for TFTP. (tsize / blksize) * RTT is the theoretical limit of transfer duration without windowing. In your case, it is (7MB / 512) * 40 ms ~ 9.1 minutes, where 512 is the default block size.

If the client supports block size extension, --blocksize 1500 would work on Ethernet connection with 2~3 times improvement in throughput.
You can use busybox tftp client to check the extension works: busybox tftp -b 1500 -g PATH SERVER PORT.
s3tftpd outputs debug log like stats {::1 path 44010 false octet map[blksize:1500 tsize:7548471] 32.28755713s 6 0} that indicates blksize of 1500 is negotiated.

In theory, windowing can help a lot this case. I would try --anticipate 16 or 32... but right now I don't have a handy environment with large latency to experiment at my end.

[cecchisandrone]

So that’s normal, due to how the protocol is implemented. Unfortunately I need this to make firmware updates across the internet and client is not modifiable to set block size. However thanks a lot for the sound explanation!

…

On 3 Jul 2020, 15:49 +0200, Kasumi Hanazuki ***@***.***>, wrote: Wow! 40 ms latency is quite large for TFTP. (tsize / blksize) * RTT is the theoretical limit of transfer duration without windowing. In your case, it is (7MB / 512) * 40 ms ~ 9.1 minutes, where 512 is the default block size. If the client supports block size extension, --blocksize 1500 would work on Ethernet connection with 2~3 times improvement in throughput. You can use busybox tftp client to check the extension works: busybox tftp -b 1500 -g PATH SERVER PORT. s3tftpd outputs debug log like stats {::1 path 44010 false octet map[blksize:1500 tsize:7548471] 32.28755713s 6 0} that indicates blksize of 1500 is negotiated. In theory, windowing can help a lot this case. I would try --anticipate 16 or 32... but right now I don't have a handy environment with large latency to experiment at my end. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[hanazuki]

While block size needs to be negotiated, sender anticipation does not require special client support. Though, it does not conform to any RFC.

[cecchisandrone]

This is also good for reference https://gtacknowledge.extremenetworks.com/articles/Solution/Downloading-a-file-over-TFTP-is-significantly-slow