Spin up separate JSON parser web workers for each event channel

[EzraBrooks]

• incrementally parse the JSON to get the field that determines which event it will be emitted on
• spin up a Web Worker for each unique event name
• do the JSON parse in that web worker each time an event comes through on that event name

Originally posted by @EzraBrooks in #982

[EzraBrooks]

@drewhoener it's probably worth evaluating alternatives to EventEmitter that might make this easier - I've just noticed EventEmitter3 is just as dead as EventEmitter2 was when I migrated us to 3. Latest release in 2023.

[drewhoener]

Here's my napkin thoughts on how to structure the problem.

• Ros Client manages a worker instead of the transport, the worker uses the existing Transport code. Communication is done via events through postMessage which accomplishes basically the same thing as EventEmitter
• Either the socket worker or the ros client maintains a map of active topics and manages a map of broadcast workers that Topic, Service, Action, etc. can all join. Alternatively the Ros client can create message channels and pass stuff along that way.
• The Ros client can create workers for topics / services / actions when requested, which will allow it to create the message channels if we go that way.
• The individual Topic/Service/Action classes do basically the same thing they already do, and since they're workers, consumers can add event listeners, or we can add another wrapper layer around it to accomplish what we need

(ignore that I only made one box for broadcast channels, the idea is that there would be a channel for each topic name. There would also have to be some sort of registry for the topic names)

Some other thoughts:

• I wanted the socket worker to be a SharedWorker but unfortunately there seems to be pretty shallow support :(
  Chromium has a feature test going for shared workers on android but that's not going to close until sometime in 2026.
  We could do a thing where a shared worker is selected if support is available, else use a regular worker. The upside is you can use the same worker for a single socket across multiple tabs, but nothing is really lost if another socket connection has to be opened.

• I thought about mixing EventEmitter and workers but ultimately just figured that if we're going that way, might as well offload everything we possibly can. As far as I can tell with this roadmap, the only things I can't remove are the socket events on ros. Open to feedback on what to do there.

Thoughts?

[EzraBrooks]

That all sounds good as an initial "before we get punched in the face" plan, to butcher the Mike Tyson quote. Did you look into the implications of shareable data types to ensure we don't incur copy overhead for large messages?

[drewhoener]

I was thinking about that a bunch actually. Seems to me like no matter where we place our workers there's going to be some form of copy, we just need to figure out where it should be.

Unless we want to get into web assembly and shared memory, the only things that can move between threads are ImageData, MessageChannel, and SharedArrayBuffer (and some others like Streams).

For a normal message (json string), it basically follows your original bullets:

1. Check for "op" field in string so we don't have to parse.
2. Send to the correct worker via shared buffer
3. worker parses and transfers.

The task the worker actually performs gets a bit nebulous because if the worker is the one doing the parse, we then have an object we can't easily move. I think the performance gains for handling and parsing will be worth it, especially since you can keep several of the same event emitter semantics like 'once' with the workers.

I'm cautiously optimistic that it'll be ok (unless the messages are HUGE)

Check out these benchmarks: https://surma.dev/things/is-postmessage-slow/

[EzraBrooks]

The MDN docs on transferable objects imply that for our use case we will probably want to ship things around using ArrayBuffers. Which is already what CBOR support in roslibjs uses. I feel like every half an hour I discover another reason why we need to switch rosbridge protocol to CBOR-only (or a similar protocol over arraybuffers)

[EzraBrooks]

(lol, nearly simultaneous comments) Agreed with most of your points above!

However, my personal white whale for roslibjs performance is parsing PointCloud2 and Octomap, which I've seen take JSON.parse - the C++-accelerated browser API - take multiple seconds. If we have to parse them back out of an ArrayBuffer I doubt we're saving time - unless we end up using something like CBOR between the worker and the main thread, at which point we have two layers of compression/serialization until we make a decision about breaking API on a rosbridge protocol v3 🤦🏻 but that might be the right approach.

[EzraBrooks]

If I was a BDFL I would say we should throw all the existing serializers and compression algorithms out and just do MessagePack across the whole stack.

(sorry for the rapid-fire comments, I need to get in the habit of composing my thoughts and making one large comment)

[EzraBrooks]

opened a discussion on rosbridge for protocol considerations for the future: RobotWebTools/rosbridge_suite#1124

[drewhoener]

If I was a BDFL I would say we should throw all the existing serializers and compression algorithms out and just do MessagePack across the whole stack.

My BDFL solution would be a Rust WASM library that does all the heavy lifting on the websocket and parsing side, since you can get zero-copy move between different threads (might not even need threads with rust + wasm)

But I just don't think that's super realistic for now. It might be worth it to do some portion of it? Just kind of spitballing...

I do think we're on the same page about the pros and cons, we might not get any value if we have to serialize multiple times

[EzraBrooks]

As long as we're spitballing, we could actually compile roslibrust to WASM and use it as the underpinnings of roslibjs 😉

Though that would leave the RobotWebTools org without a "reference implementation client library".

[EzraBrooks]

Anyway, what if the web worker supported all of the rosbridge protocols and encoding and then the "frontend" that runs on the main thread only supported CBOR, transferred to it in SharedArrayBuffers from the web worker?

[drewhoener]

As long as we're ok with the cost of encoding it back to cbor from whatever other encoding then yeah I can't think of a reason not to. CBOR is technically faster since there's no JSON syntax to parse out

we could actually compile roslibrust to WASM

Fun fact, roslibrust has an incomplete implementation of the bridge protocol so we might be out of luck there lol

I think for the topic of point clouds it's already in the developer's best interest to use cbor with rosbridge in the first place, so for most messages like that we might not have to worry about it? But then there's always fragments which always end up going through JSON.parse...
So if the message is big enough to be fragmented it's just the same problem all over again

We'd also need to be able to pull out the id and op fields to be able to properly route the cbor messages to their correct destination. That's not terrible though even if we have to roll something ourselves, since packets are only 5-6 fields max at the first level.

[drewhoener]

On another note, I think I'm starting to hit the upper limit of topics I can get updates from without starting to bottleneck my ui render loop lol
Lots of sensors on the robot I'm testing with, it's part of the reason I'm interested in seeing this through. Getting the socket messages and parsing off my main thread would give me much more to play with

[EzraBrooks]

Yeah I think we'll have to pay the tax of re-encoding to CBOR in the worker to get the benefit of a fast decoder on the main thread. At the very least it seems like a feasible approach we should experiment with?