RPC refactoring and new (message‐based) transports

This page is about the refactoring of the RPC system in XLibre and new message based transports. If you are unsure whether to add something or make a change to this page, just talk about it.

The current RPC implementation is very complex and mostly open-coded, especially due variable sized structures (often w/o any hard upper bound) and byte-swapping. That's making the code hard to understand and adding message-based transports (eg. shared memory, infiniband, ...) very hard to implement.

One of many complicating aspects is that a header field needs to be set to the amount of protocol units (=4 bytes) of the actual payload - right now we've got a lot of extra logic for computing the actual space needed before we can start constructing or sending the payload piece by piece. Various places using different techniques for this (some even have their own buffering or sg-list mechanisms). Another one is some places (eg. xkb) sometimes need to send nested replies.

All the data is finally landing in an per-client TX buffer, which is written out in pieces (eg. to prevent TX stalling).

This all demands a careful refactoring, which must be done in smaller steps:

TX path consolidation

1. moving reply payload assembly into x_rpcbuf_t

This buffer is easily to set up (just declare a struct, optionally setting some fields) and automatically handles all allocations. It also allows preallocation (eg. where we know we need a lot of space) and optional reuse of allocated space.

All request handlers are rewritten to use x_rpcbuf_t for their payloads - they'll all have writing out header and payload as the very last step (right after return Success).

2. consolidate the reply header assembly

All places of constructing a reply header struct will be changed into canonical scheme for declaring and assigning the reply structs at once. This will be after the payload assembly, everything's assigned in one shot (no further changes in the header). Note that all reply headers variables will have the same (local) name: reply

3. consolidate byte reply header byte-swap and final send-out

We're going to use macros for declaring the individual header field types - they'll do the byte-swap automatically if required. Also the final write out is put into a macro, which also takes care of size computation, initialize common fields, etc.

4. move headers into the from stack into the rpcbuf's

Moving the headers into the rpcbuf's, by adjusting the previously introduced macros/functions.