Tuning for low latency on constrained environments (rinha de backend)

[faermanj]

Hello fellows,

The backend dev community in Brazil is holding a competition, rinha de backend, where the goal is to build a simple transaction processing backend with the lowest possible latency.

I started a repository with Quarkus and GraalVM native here:
https://github.com/CaravanaCloud/rinha-de-backend-2024-q1-impl/tree/main/rinhadebackend

And here is the best time so far (we'll keep it up to date):
https://caravanacloud.github.io/rinha-de-backend-2024-q1-impl/

If you'd have any suggestions on how to improve latency on quarkus/graalvm + postgresql + nginx, I'd be happy to discuss and test it out.

Thank you all in advance for the great comunity, learning a lot here!

[faermanj]

Here's an update of current times.
There's a bit of portuguese there, sorry, but it's understandable (credit / debit / extract [balance] ) :)

[faermanj]

[franz1981]

Hi!

Do you have any CPU vs network in/out metrics for the application? this can help to understand where the tuning should be.

We do have some optimized quarkus stacks at https://github.com/TechEmpower/FrameworkBenchmarks/tree/d576b81870ed924c9bd2fc61a9bdc53a4a7a0614/frameworks/Java/quarkus which introduce interesting variations of quarkus.
In particular, https://github.com/TechEmpower/FrameworkBenchmarks/tree/d576b81870ed924c9bd2fc61a9bdc53a4a7a0614/frameworks/Java/quarkus/reactive-routes-pgclient: it is using reactive routes and high level of pipelining of pgclient, which can be a double edge sword, given that each db request is serialized, so...based on how fast the DB (and operations on it) is it could improve or not the application throughput.
In the same repository we're avoiding Jackson (which uses a form of reflection) to further speedup performance, but still using reactive-routes, because at that time, it was better performing than resteasy-reactive (this is true when you have very few routes, actually...). This last statement is no longer true in main so, need to wait till the next release and you can just use resteasy-reactive and get better performance than reactive-routes.
Additionally at https://github.com/TechEmpower/FrameworkBenchmarks/blob/d576b81870ed924c9bd2fc61a9bdc53a4a7a0614/frameworks/Java/quarkus/reactive-routes-pgclient/src/main/resources/application.properties#L11 you can fully disable anything related context propagation (which include wrappers around smallrye interceptors AFAIK) and https://github.com/TechEmpower/FrameworkBenchmarks/blob/d576b81870ed924c9bd2fc61a9bdc53a4a7a0614/frameworks/Java/quarkus/run_quarkus.sh#L13 you have a serie of sys property which could be used to reduce the cost of many part of the infrastructure.

If the most of the cost of the application lie in the framework overhead, these should help to reduce it, but you need to profile and/or gather CPU usage metrics to understand it.

[faermanj]

Thanks a lot, looking into all that <3

[faermanj]

Is there load-balancer that can leverage metrics exposed by quarkus to perform better?
Or any other technique, like we used to have with AJP.
I've tested nginx, haproxy and envoy, not much of a difference on round-robin.