Bulk doc value loading at codec level #128445
Description
Implement bulk doc values implementations that push down value loading to doc values codec level. This should be possible for our custom doc value codec that we use for logsdb and tsdb and with the compute engine would allow for bulk loading.
- Make use of low level implementations details of the codec when loading blocks. The tsdb doc value codec is organized values / ordinals in blocks of 128 values each. Today this gets decoded into an array which gets used in intermediate calls to public doc value apis. We could chose to append all 128 values in one operation to block loader builder when possible.
- Take another look at explicit vectorization and see whether this can be applied to value loading. Today we rely on auto vectorization, which maybe doesn't get applied all the time.
A first step is to push down doc value loading to the es819 doc values codec and add bulk loading support to es819 doc value codec. This is a relative large tasks but can be divided into smaller tasks:
- Add initial bulk loading to es819 doc value codec. (Push compute engine value loading for longs down to tsdb codec. #132622)
- Push down loading of singleton dense long based field types to the es819 doc value codec. (Push compute engine value loading for longs down to tsdb codec. #132622)
- Push down loading of singleton dense integer based field types to the es819 doc value codec. (Support bulk reading doc values for byte, short and integer field types. #134753)
- Push down loading of singleton dense float/double based field types to the es819 doc value codec. (Push down loading of singleton dense double based field types to the … #133397)
- Push down loading of singleton dense keyword field types to the es819 doc value codec. (Speed up loading dense singleton keyword fields #132994)
- Skip iterating DISI when reading metric values #133365