asynchronous task blocking user requests

[lijingrs]

Summary

In the Axum framework, if an asynchronous task takes a long time to execute, it directly blocks user requests and causes timeouts. What is the best optimization solution for this? I tried using a queue, but the optimization improvement is limited—even after dequeuing, the asynchronous task still takes too long and ends up blocking user requests.

axum version

0.8.4

[mladedav]

That really depends on what exactly you're doing. If you're blocking the Tokio thread in your handlers, the solution is to use asynchronous alternatives instead or to use spawn_blocking.

If you're not blocking the thread, timeouts like you're describing shouldn't happen.

If you can minimize an example, we can take a closer look.

[lijingrs]

Here is the code I've extracted. Thank you for your guidance. If there are any omissions or areas where you'd like more detailed information, I can provide additional details.

https://github.com/lijingrs/block_demo

[mladedav]

That's still a lot of code. Could you try to minimize it further? Or if you do, the problem goes away?

[lijingrs]

After minimizing, it indeed solves the above problem. Let me try gradually increasing the complexity according to the business logic.

[lijingrs]

The conclusions drawn from a week of stress testing：

1. Tokio's asynchronous operations are not unlimited; when concurrency is high and individual tasks take significant time to complete, it can lead to blocking, making it suitable only for handling short and fast requests.
2. CPU-intensive tasks may not be well-suited for the Axum framework.
3. Services involving CPU computations can perform up to 10 times worse in containers compared to bare-metal environments.

thanks.

[lijingrs]

If I replace it with a CPU that has more cores and higher frequency, the performance will be better and it can handle 100 concurrent requests without timeouts.
But if I manually adjust the work thread parameters, it doesn't have much effect.
Thanks for the assistance!

[lijingrs]

container: k8s docker base img: debian:bookworm-slim