Is CLI slow?

[orhun]

I'm experiencing some slow uploads sometimes. Is it due to my internet & remote server or is this really slow?

Can we tweak buffering somehow?

@tessus have you experienced anything like this?

[tessus]

Nope, not that I am aware of. I can give you access to my instance and you can run a few perf tests if you want.

However, I currently have an issue with my httpd coredumping at times due to some weird glibc issue. I use httpd as a reverse proxy for rustypaste. The issue is unrelated to rustypaste or reverse-proxying, but I mention it because it could happen that at one point I'd have to restart httpd. ;-)

[tessus]

Well, this might be a problem. My upstream is 50 MBit, while my server has 1Gbit up and down.
Even with scp the transfer varies a lot. My server is located in Europe and I have the feeling that sometimes there are perf issues over the ocean. I believe it's my provider here in Canada.

Thus I don't know how much I would trust results when initiating the transfer from here. I can run tests the next time I am in Europe.

Maybe I can setup a channel the other way around, when I have a bit more time.

[orhun]

Sounds good, in the meantime I can test and share my results here probably.

[tessus]

For the first time I've noticed a stalling during the transfer of a 20MB file. It never finished.

At around 10MB the speed went from super fast to 0 B/s.

11.80 MiB/19.09 MiB (0 B/s)

@orhun unfortunately I don't have time to run other tests at the moment. But this is certainly concerning.

P.S.: Additionally I have noticed that some things are only evaluated after the file has been uploaded. I rememeber I also had an issue in the past where the filesize limit was ignored and the entire file was uploaded. Afterwards I got the error message that the file was too big.

In the following session the 1.3MB file is uploaded and then I get the error message and obviously the file is not available on the server. Thus it was uploaded for nothing.

$ rpaste -e 7 debuglog.txt.gz
Upload error: `time unit needed, for example 7sec or 7ms (status code: 500)`

I am not very familiar with streams and multipart request processing in Rust, but I suspect the issue is somewhere in that area.

[orhun]

I see, thanks for sharing your findings. I think this is rather server-related than CLI. The upload code is semi-badly written, it is improved over time but at first I hacked it together very quickly. e.g. there were issues like orhun/rustypaste#1

We probably need a proper benchmark testing and a detailed report to find out what's going on probably. In the meantime reviewing the upload code might be useful to spot some issues.

[tessus]

I haven't tried to use a curl command, so I don't know whether the cli code had anything to do with the slowdown and ultimately the stalling into oblivion. ;-)

But I remember code in the server (the middleware) that had to drain the stream due to issues with the actix component. I have never used actix before rustypaste so I can't speak to that, but maybe newer versions have fixed the issue. I believe there are now better solutions to limit the upload size than the middleware code. Or maybe rocket is a better way to handle requests? Idk.

There was also #316 which could indicate that curl requests are fine.