Efficient live data - defer?

[oscarthorn]

Hi!

First of all, thanks for the great library!

I have an embedded system that generates data for a client (browser) to consume. uWebsockets is running in one thread and the other threads are doing the processing and generating the data. I'm trying to figure out the lowest latency way to send this data, while also not wasting resources.

There is around 100-200 "messages" per second.

This simplest way is to just do defer for every message but that's a lot of locks/unlocks, possibly with contention if data is generated on different threads. But maybe this is fine, I'm mostly concerned with how this might affect the server.

Another way could be to batch the data and then do defer, though that adds some latency, but the data is not evenly spaced any way so it should not be to much.

I could have a timer in the uWebsockets loop and check some queue very often but then if you check to often it's a waste and not often enough introduces latency.

I'm mostly after if there is some clever solution I have not thought about? And at what point, in general, does calling defer become an issue for the performance of the server?

[uNetworkingAB]

Batch the data a little bit and then use defer. Defer is used in many prod apps and even if it is not ideal, it still works.

A timer is probably not a super bad idea either - even if you set it at 5ms a modern CPU that has to wake up and do nothing is still going to use practically 0% CPU-time overall because 5ms is an eternity. But this of course will drain a battery if that's something you care about. But yeah a timer is probably not my favorite solution (but it could WORK).

The ideal solution would be to not have any threading but only use 1 event loop and scale uniform instances of the entire app on isolated threads, lika NUMA-aware solutions (this is the intended architecture)

But it depends, I don't know your circumstances and it's not for me to do your job ;)

[oscarthorn]

Thanks for the feedback! I'll try it with defer first and do some tests to see what ends up being most efficient.

Yeah, single event loop and no threading would be much easier in many respects. Unfortunately horizontal scaling does nothing for us as we are dealing with one set of data being processed almost always going to one client. So there are going to be thread boundaries somewhere since the processing is spread out over multiple cores.