Improve block compression (#1710)

* [improve-block-compression] Add improved block compression algorithm, saving 15% more space with almost the same performance. Add dynamic compression algorithm configuration.

* [improve-block-compression] Add max_decompressed_size to decompression interface

* [improve-block-compression] Fix encoding of Prunned branch cells. Add support for collated data format.

* [improve-block-compression] Change compression interface to support multiple roots. Save algorithm flag in the first byte.

* [improve-block-compression] Replace compression with a new one in method compress_candidate_data()

* [improve-block-compression] Store decompression size at the start of the compressed data

* [improve-block-compression] Remove passing max decompression size to new compression method

* [improve-block-compression] Use new compression algorithm in serialize_block_full()

* [improve-block-compression] Use new compression algorithm in serialize_block_candidate_broadcast()

* [improve-block-compression] Small performance optimization

* [improve-block-compression] Replace vector of vectors with vector of arrays for faster operations with BOC graph. Speed up by ~10%.

* [improve-block-compression] Use new compression approach in serialize_block_broadcast() method

* [improve-block-compression] Remove redundant debug output

* [improve-block-compression] Add checks and validations for improved compression

* [improve-block-compression] Remove redundant size calculation

* [improve-block-compression] Add all necessary checks and validations to improved decompression method

* [improve-block-compression] Add all necessary checks and validations to base compression methods

* [improve-block-compression] Introduce tl objects for improved compression and integrate them into all compression usages.

* [improve-block-compression] Fix support of prunned branch cells with level up to 7.

* [improve-block-compression] Replace size literals with a constant. Add checks for baseline decompress.

* [improve-block-compression] Add check against negative cell data length

* [improve-block-compression] Catch CellWriteError from cell_builder.finalize()

* [improve-block-compression] Move bits allocation and uint write into a single append function.

* [improve-block-compression] Move buffer capacity check and bits reading to a separate method

* [improve-block-compression] Revert original implementation of the compression methods, while keeping updated decompression methods

* [improve-block-compression] Revert original implementation of the compression methods, while keeping updated decompression methods

* [improve-block-compression] Store signatures directly inside tonNode_blockBroadcastCompressedV2 tl object.

* [improve-block-compression] Pass max_decompression_size from config to boc_compress method. Use it to check limit on decompressed block.

* [improve-block-compression] Increase max cell count to 2^32. Add node count validity check.

* [improve-block-compression] Refactor types. Use unsigned types where possible.

* [Compress-block-using-round-cache] Remove limitation of 2^16 root cells in BOC.

* [Compress-block-using-round-cache] Regenerate ton_api.tlo after merging testnet branch.

Author: Mustang98

crypto/CMakeLists.txt

@@ -14,6 +14,7 @@ set(TON_CRYPTO_CORE_SOURCE
   openssl/residue.cpp
   openssl/rand.cpp
   vm/boc.cpp
+  vm/boc-compression.cpp
   vm/large-boc-serializer.cpp
   tl/tlblib.cpp
 
@@ -108,6 +109,7 @@ set(TON_CRYPTO_SOURCE
   vm/arithops.h
   vm/atom.h
   vm/boc.h
+  vm/boc-compression.h
   vm/boc-writers.h
   vm/box.hpp
   vm/cellops.h