Add summaries for benchmarking and tools for working with chunked datasets #1
Conversation
| - [Holding space - Better document acceptable values for StoreLike] | ||
| - [Holding space - LRUStoreCache] | ||
|
|
||
| PRs have also been opened for: |
There was a problem hiding this comment.
Only issues have been opened for dask / skimage - not PRs
| ## Benchmarking for Zarr | ||
|
|
||
| We created a set of benchmarks for writing data to Zarr with a range of different configurations as guidance for the options available when reading and writing 3D imaging data. | ||
| The different parameters were: |
There was a problem hiding this comment.
Link to the final report?
e.g. 'The final benchmarking results are summarised into a short report to help researchers choose the best options for their image datasets'
| We created a set of benchmarks for writing data to Zarr with a range of different configurations as guidance for the options available when reading and writing 3D imaging data. | ||
| The different parameters were: | ||
|
|
||
| - Type of image |
There was a problem hiding this comment.
Here you're mixing the different parameters (e.g. type of image = heart / dense /sparse) with benchmarking results (e.g. compressor = blosc-zstd provides the best compression ratio). Maybe just list the different parameters options here, and link to the final report for results? Not sure what @dstansby 's preference is.
|
|
||
| PRs have been opened in the zarr-python repository: | ||
|
|
||
| - [Prevent creation of arrays/groups under a parent array](https://github.com/zarr-developers/zarr-python/pull/3407) |
There was a problem hiding this comment.
This PR (prevent creation of arrays/groups../) has been merged now - so can move up into 'contributions made'
|
|
||
| ## Tools for working with chunked datasets | ||
|
|
||
| Contributions have been made to the zarr-python repository: |
There was a problem hiding this comment.
Do we need a brief description of what each of these contributions does + why it was useful? Again, not sure how much detail we need to go into here
| Contributions have been made to the zarr-python repository: | ||
|
|
||
| - [Add CLI for converting v2 metadata to v3](https://github.com/zarr-developers/zarr-python/pull/3257) | ||
| - [Added ArrayNotFoundError](https://github.com/zarr-developers/zarr-python/pull/3367) |
There was a problem hiding this comment.
Not sure if we want to include them, but I did do a couple of other minor PRs:
README.md
Outdated
| PRs have been opened in the zarr-python repository: | ||
|
|
||
| - [Prevent creation of arrays/groups under a parent array](https://github.com/zarr-developers/zarr-python/pull/3407) | ||
| - [Holding space - Better document acceptable values for StoreLike] |
There was a problem hiding this comment.
| - [Holding space - Better document acceptable values for StoreLike] |
This PR has been merged now! So the following can move up into the 'contributions made' section.
Co-authored-by: Kimberly Meechan <[email protected]>
|
Thanks - I'll merge and follow up on @K-Meech comments in follow up commits. |
3 participants
N.B. Some of the points I mentioned in the tools section are still WIP