Proposal for unified handling of variable length objects in schema (part of #2127)

Summary

Handling variable-length objects is anything but trivial. Since the support for long names is being introduced, we can hardly afford allocating a fixed length name buffer since it results in significant wasted space.

We would like to combine all variable length parts of an object into a single allocation.

We would like to avoid allocating MAX_PATH (4Kb) buffer for vinyl path in key_def which is seldom used but inflates a key_def.

Finally, key_def already has a VLA portion — namely, parts array. We would like to see all variable length fields handled in a consistent fashion.

Proposal

All variable-length parts must be a pointer, eg:

struct key_def {

  const char    *name; /* !!! */

  struct key_opts {
    const char  *path; /* !!! */

  }              opts;

  struct key_part {
    uint32_t     field_no;
    enum field_type
                 field_type;

  }             *parts; /* !!! */
};

Schema objects are initialised either explicitly (for pre-defined objects) or by a parser in on_replace trigger in system spaces.

We suggest the following routine (using key_def for illustration):

1. The parser initialises a "prototype" key_def. The prototype may reference other memory chunks via pointers.
2. To "finalize" the prototype, the parser calls key_def_dup() on it.
3. Key_def_dup() allocates a sufficiently sized memory chunk and copies the passed prototype key_def into it. The function then copies the variable length parts after the duplicate key_def and fixes pointers in the duplicate.

It's convenient that key_def_dup() receives nothing but a prototype key_def. Compare it with key_def_new() we currently have: it has 6 parameters for various attributes of a key_def.

A prototype key_def is fully compatible with a finalised one.

Special Cases

For special cases like parts array in a key_def the added pointer may actually impede the performance.

We anticipate for the special cases like this one. The parser prepares parts in a separate buffer like it does with other variable-length fields. Key_def_dup(prototype, parts) renders the final object. Yet the basic principle remains the same: the parser passes the fully initialised object to key_def_dup and the later finalises it.

Managing memory for prototype key_def

Killing opts_create/opts_encode