Support using dev versions of numcodecs with zstd #2422
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I received the following error when using zstd with
'0.13.2.dev40'(frompip install git+https://github.com/zarr-developers/numcodecs@zarr3-codecs). This change only uses theX.Y.Zpart from the version when checking compatibility:--------------------------------------------------------------------------- ValueError Traceback (most recent call last) Cell In[11], line 12 8 ds = xr.open_zarr(virtual_store, zarr_version=3, consolidated=False) 9 ds = ds.drop_encoding() 10 encoding = { 11 variable: { ---> 12 "codecs": [zarr.codecs.BytesCodec(), zarr.codecs.ZstdCodec()], 13 "chunks": (1, 4096, 4096) 14 } 15 } 16 ds.to_zarr(output_uri, zarr_version=3, mode="w", encoding=encoding, storage_options={"use_listings_cache": False}) File /opt/conda/lib/python3.11/site-packages/zarr/codecs/zstd.py:46, in ZstdCodec.__init__(self, level, checksum) 44 def __init__(self, *, level: int = 0, checksum: bool = False) -> None: 45 # numcodecs 0.13.0 introduces the checksum attribute for the zstd codec ---> 46 _numcodecs_version = tuple(map(int, version("numcodecs").split("."))) 47 if _numcodecs_version < (0, 13, 0): # pragma: no cover 48 raise RuntimeError( 49 "numcodecs version >= 0.13.0 is required to use the zstd codec. " 50 f"Version {_numcodecs_version} is currently installed." 51 ) ValueError: invalid literal for int() with base 10: 'dev40'