Array Accessors

[gatesn]

At the moment, arrays are typically flattened in order to make use of the underlying data.

There are a couple of problems with this:

• There's currently no way to ensure we don't decompress the same data hundreds of times.
• Compute over larger arrays doesn't make use of cache locality.
• It is harder to leverage auto-vectorisation when writing compute over arbitrary sized arrays.

One idea for improving this is to provide a way to iterate flattened chunks of an array, where each chunk is either full (with some globally fixed size, say 4096 elements), or partial (anything less than this).

• Compute functions can then operate over fixed-sized arrays enabling much better auto-vectorisation.
• We might want some way to automatically combine together adjacent partial chunks with less than 4096 elements.

Alternatively, we could just say it's up to the consumer of Vortex to ensure arrays are chunked small enough to make use of CPU caching? In this world, we might want to lean on this pattern to say that all Vortex arrays must have <u32 elements, and larger arrays can be automatically turned into chunked arrays?