RCORE-2058 Adding clickbench importer and query tool for evaluating compression (#7664)

* update next-major

* specified part of new layout (new width encoding)

* new header format for compressed arrays

* code review

* code review

* start of classifying arrays for compression

* classification down to column types

* first attempt to cut through the BPlusTree madness

* [wip] start on 'type driven' write process

* all tests passing (but no compression enabled)

* enabled compression for signed integer leafs only

* removed some dubious constructions in cluster tree

* delete tmp array while classifying arrays

* enabled compression of links and backlinks (excl collections)

* also compress bplustree of integers/links (experimental)

* pref for compressing dicts (not working)

* wip

* wip

* finally: compressing collections (incl dicts)

* compressing timestamps now

* enabled compression on ObjectID, TypedLink and UUID

* also compressing Mixed properties (not list/dicts of Mixed)

* Array compression with collections in Mixed (#7412)

---------

Co-authored-by: Finn Schiermer Andersen <[email protected]>

* merge next-major + collection in mixed

* enable dynamic choice of compression method

* moved typed_write/typed_print for bptree into class

* Merge pull request #7432 from realm/fsa/clean_typed_write

moved typed_write/typed_print for bptree into class

* cleanup unrelated code changes

* fix compilation

* cleanup

* code review

* code review

* scaffolding for accessing unaligned memory inside compressed arrays

* introducing parallel scan for scanning in one go multiple compressed arrays

* Silence warning

* tests + point fix for upper bound

* fix ubsan

* added clickbench data ingestor

* Added 2 queries to clickbench

* separated out a clickquery benchmark

* separated out a clickquery benchmark

* update to clickbench and query

* fix warnings

* attempt to fix windows builders

* silence warnings

* fix importer

---------

Co-authored-by: Finn Schiermer Andersen <[email protected]>
Co-authored-by: Jørgen Edelbo <[email protected]>
Co-authored-by: Finn Schiermer Andersen <[email protected]>

Author: 3 people

src/realm/array_direct.hpp

@@ -693,7 +693,7 @@ inline size_t first_field_marked(size_t width, uint64_t vector)
 }
 
 template <typename VectorCompare>
-size_t parallel_subword_find(VectorCompare vector_compare, const uint64_t* data, size_t offset, size_t width,
+size_t parallel_subword_find(VectorCompare vector_compare, const uint64_t* data, size_t offset, uint8_t width,
                              uint64_t MSBs, uint64_t search_vector, size_t start, size_t end)
 {
     const auto field_count = num_fields_for_width(width);

src/realm/exec/CMakeLists.txt

@@ -24,6 +24,28 @@ if (EMSCRIPTEN)
     set_target_properties(RealmTrawler PROPERTIES EXCLUDE_FROM_ALL TRUE)
 endif()
 
+add_executable(ClickBench EXCLUDE_FROM_ALL clickbench.cpp )
+set_target_properties(ClickBench PROPERTIES
+    OUTPUT_NAME "clickbench"
+    DEBUG_POSTFIX ${CMAKE_DEBUG_POSTFIX}
+)
+if (EMSCRIPTEN)
+    set_target_properties(ClickBench PROPERTIES EXCLUDE_FROM_ALL TRUE)
+endif()
+
+target_link_libraries(ClickBench Storage)
+
+add_executable(ClickQuery EXCLUDE_FROM_ALL clickquery.cpp )
+set_target_properties(ClickQuery PROPERTIES
+    OUTPUT_NAME "clickquery"
+    DEBUG_POSTFIX ${CMAKE_DEBUG_POSTFIX}
+)
+if (EMSCRIPTEN)
+    set_target_properties(ClickQuery PROPERTIES EXCLUDE_FROM_ALL TRUE)
+endif()
+
+target_link_libraries(ClickQuery Storage)
+
 add_executable(RealmEnumerate realm_enumerate.cpp)
 set_target_properties(RealmEnumerate PROPERTIES
     OUTPUT_NAME "realm-enumerate"