Rectilinear (variable-length) chunk grid #25
Conversation
There was a problem hiding this comment.
Looks good to me. Do we want to include a recommendation that implementations SHOULD use run length encoding where appropriate when saving metadata?
Implementation here: zarrs/zarrs#284
| [[1, 6]], // expands to [1, 1, 1, 1, 1, 1] | ||
| [1, [2, 1], 3], // expands to [1, 2, 3] | ||
| [[1, 3], 3], // expands to [1, 1, 1, 3] | ||
| [6], // expands to [6] |
| #### Chunk edge lengths | ||
|
|
||
| The edge lengths of the chunks along an array axis `A` are represented by an array that can contain two types of elements: | ||
| - an integer that explicitly denotes denotes an edge length |
|
|
||
| #### Chunk edge lengths | ||
|
|
||
| The edge lengths of the chunks along an array axis `A` are represented by an array that can contain two types of elements: |
There was a problem hiding this comment.
A previous proposal for rectilinear chunking also allowed a plain integer here, to indicate uniform chunking along the dimension. That makes this strictly a generalization of regular chunking and seems like a good idea to include.
While the run-length encoding makes regular chunking along a dimension still efficient to specify, a plain integer indicating uniform chunking has the advantage of allowing the dimension to be resized without also specifying the new chunk sizes.
There was a problem hiding this comment.
Potentially you could allow the last item in the chunk_shapes entry for a given dimension to be [n, -1] to mean that all remaining chunks are size n --- that would allow the dimension to be resized since it would indicate the chunk size for new chunks.
That would reduce the need for also allowing a plain integer to specify uniform chunking, but it is still probably good to allow a plain integer.
3 participants
Opening this PR with a draft of a rectilinear chunk grid spec