Handle zarr 3.1.0

[d-v-b]

Implements a data type adapter layer for numcodecs.zarr3. Ensures that zarr data structures get the right data type.

this hopefully fixes the breakage caused by the zarr 3.1 release.

TODO:

• Unit tests and/or doctests in docstrings
• Tests pass locally
• Docstrings and API docs for any new/modified user-facing classes and functions
• Changes documented in docs/release.rst
• Docs build locally
• GitHub Actions CI passes
• Test coverage to 100% (Codecov passes)

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (506c89b) to head (80fb4d2).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff             @@
##             main      #766      +/-   ##
===========================================
+ Coverage   99.96%   100.00%   +0.03%     
===========================================
  Files          64        64              
  Lines        2789      2804      +15     
===========================================
+ Hits         2788      2804      +16     
+ Misses          1         0       -1     

Files with missing lines	Coverage Δ
numcodecs/zarr3.py	100.00% <100.00%> (+0.60%)	⬆️

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[dstansby]

LGTM - the new API should definitely be made private, and it would be good to see at least one CI run against Zarr 3.0.x to make sure we're retaining support there.

[d-v-b]

LGTM - the new API should definitely be made private, and it would be good to see at least one CI run against Zarr 3.0.x to make sure we're retaining support there.

what's the easiest way to set up a test matrix that includes multiple zarr versions that can be run locally easily? Maybe we need to bite the bullet and bring in #704

[dstansby]

For now, can we not just have one test run on the GH Actions CI that tests against zarr-python 3.0.x?

[d-v-b]

For now, can we not just have one test run on the GH Actions CI that tests against zarr-python 3.0.x?

sure! can you write it?

[d-v-b]

I added pixi + hatch to pyproject.toml exclusively for running the zarr-python tests against two different versions of zarr-python. We can always add to this later.

[dstansby]

Big 👍 for pixi here. A couple of things:

• I think we shouldn't commit a lockfile, see inline comment
• In the test logs the version of Zarr the tests are run with isn't printed, meaning it's not possible to verify tests are being run against 3.0.x and latest zarr. Can you print the Zarr version somehow?

[d-v-b]

In the test logs the version of Zarr the tests are run with isn't printed, meaning it's not possible to verify tests are being run against 3.0.x and latest zarr. Can you print the Zarr version somehow?

437689f adds a step in CI that lists the deps

[TomNicholas]

This is important to get in because currently there is zarr data written in the wild that cannot be read with recent zarr-python versions without this PR.

[rabernat]

Thanks for making this critical fix. Currently, lots of data that was written by Zarr 3.0 is unreadably by 3.1. That needs to be resolved urgently.

This PR reveals something well known already, which is that the interface between Zarr and Numcodecs is not well defined anymore.

However, I think it's critical to get this PR out asap and deal with that broader question later.