Protobuf as (internal) data exchange format

[victor-m-alvarez]

Hi there,

I've been going through the codebase and I was wondering whether you have considered using protobuf as serialization format? For example, the first place in which this came to mind was when transmitting srv_config. There is a single field called aux that can be used to exchange arbitrary (meta)-data. If srv_config was expressed as a proto message, it would allow for easy ways to extend srv_config in a backward compatible way. Moreover, by following the usages of buffer and buffer_serializer one can see that proto messages arise naturally.

Thanks in advance.

[greensky00]

We don't take that into consideration. Current internal data (srv_config, log_entry, ...) is simple, compact, and rarely changes. Introducing a new dependency for such a minimal benefit seems excessive.

However, for application data that will be passed to append_entries, it's your choice whether to use protobuf. From NuRaft's perspective, it is treated as opaque binary data, so the format doesn't matter. Some of our internal use cases follow this approach as well.

[victor-m-alvarez]

Yeah, for application data protobuf is a natural choice, and if that's the case, that dependency is readily available.

How I came to this was that I was considering adding some metadata to srv_config and at first the available aux was a bit limiting, so I started exploring the codebase and I thought that if this was a proto, then extending that type would be straightforward - I did the exercise of extending srv_config using buffer, it wasn't difficult, but requires a bit of care. In the end, since aux is a string, I could simply pack an arbitrary proto in there.

I get the point of dragging another dependency, given that the library is light on them, but the benefit is not only usability and extendibility, but it would also make the codebase more transparent in my opinion.