Encrypted data PoC

[savonarola]

This PR is not for merging, it is investigation how we may keep data encrypted

Building and running

Install rocksdb deps as system packages, e.g.

 sudo apt-get install libgflags-dev libsnappy-dev zlib1g-dev libbz2-dev liblz4-dev libzstd-dev openssl-dev libssl-dev folly folly-dev 

Clone and checkout rocksdb at 8.x version, clone encfs plugin

git clone https://github.com/facebook/rocksdb.git rocksdb-encfs
cd rocksdb-encfs
git checkout v8.11.4
git clone https://github.com/pegasus-kv/encfs.git plugin/encfs

Build with system libs and with PIC

mkdir build && pushd build && cmake -DUSE_FOLLY=0 -DWITH_GFLAGS=1 -DWITH_SNAPPY=ON -DWITH_LZ4=ON -DWITH_ZLIB=ON -DWITH_ZSTD=ON -DROCKSDB_BUILD_SHARED=0 -DROCKSDB_PLUGINS=encfs -DCMAKE_POSITION_INDEPENDENT_CODE=ON  .. && make -j16; popd

Checkout erlang-rocksdb at enc-plugin branch

git clone https://github.com/emqx/erlang-rocksdb.git erlang-rocksdb-encfs
cd erlang-rocksdb-encfs
git checkout enc-plugin

Build linking with the compiled rocksdb and dynamically linking with all the deps:

ERLANG_ROCKSDB_OPTS="-DWITH_ZSTD=ON -DWITH_SNAPPY=ON -DWITH_LZ4=ON -DWITH_SYSTEM_ROCKSDB=ON -DROCKSDB_LIBRARIES=/home/av/emqx/rocksdb-encfs/build/librocksdb.a -DROCKSDB_INCLUDE_DIRS=/home/av/emqx/rocksdb-encfs/include -DROCKSDB_VERSION=8.11.4 -DCMAKE_EXPORT_COMPILE_COMMANDS=ON" make compile

Run tests:

make test

encfs tests should succeed:

encfs: basic_test (module 'encfs')...[0.090 s] ok

[savonarola]

The destructor was actually never called 🤷

Simple memenv creation and destruction was eating memory constantly

Fn = fun F() -> spawn(fun() -> {ok, Env} = rocksdb:new_env(memenv), rocksdb:destroy_env(Env) end), F() end, Fn().

[savonarola]

This function seems useless, why is it needed at all?