Support rectilinear chunk grids

[paraseba]

zarr-python is adding support for rectilinear grids soon. IC2 format needs to support it.

We have to investigate how we would support rectilinear grids, and what parts of the format need changing if any.

[paraseba]

We don't need to implement support for it, just have the right on-disk format

[TomNicholas]

It's not obvious to me that this should even require a change to our on-disk format

[ianhi]

shift_array will have to be disallowed in the case of variable chunk sizes i think? and reindex unless we add some checks that chunks are replacing the same size chunks

[TomNicholas]

Good point... I think shift array could technically still be allowed, it would just have to update the chunk sizes too... which is a bit gross 😕

[ianhi]

ahh i see. ya you are allowed to re-order chunks so maybe all fine. might not even need any extra logic

[paraseba]

We currently embed this datastructure in snapshot files:

pub struct DimensionShape {
    dim_length: u64,
    chunk_length: u64,
}

At least that will require some changes