Creating dictionaries by reference (eg: ZSTD_create[CD]Dict_byReference)

[coxley]

Problem

The current library wraps these two functions from zstd:

• ZSTD_createDDict
• ZSTD_createCDict

Both of these copy the input dictionary buffer, allowing the caller to release them. My issue with this is that I am storing dictionaries used for both compression and decompression in the same programs. Compressing data written to storage, and decompressing from storage.

We have a bunch of tenants, thus a bunch of dictionaries. With the current state, this means having two copies of each dictionary.

If I understand correctly, wrapping the *_byReference functions could share the same backing dictionary with both compressors and decompressors. This requires a bit more cooperation from the client program, but this can be noted in the documentation that it's an optimization path.

What are your thoughts, @valyala?

[coxley]

Created a PR and attached. :)