@@ -13,66 +13,66 @@ Custom stores
1313Custom codecs
1414-------------
1515
16- There are three types of codecs in Zarr: array-to-array, array-to-bytes, and bytes-to-bytes.
17- Array-to-array codecs are used to transform the n-dimensional array data before serializing
16+ There are three types of codecs in Zarr: array-to-array, array-to-bytes, and bytes-to-bytes.
17+ Array-to-array codecs are used to transform the n-dimensional array data before serializing
1818to bytes. Examples include delta encoding or scaling codecs. Array-to-bytes codecs are used
19- for serializing the array data to bytes. In Zarr, the main codec to use for numeric arrays
20- is the :class: `zarr.codecs.BytesCodec `. Bytes-to-bytes transform the serialized bytestreams
21- of the array data. Examples include compression codecs, such as
22- :class: `zarr.codecs.GzipCodec `, :class: `zarr.codecs.BloscCodec ` or
23- :class: `zarr.codecs.ZstdCodec `, and codecs that add a checksum to the bytestream, such as
19+ for serializing the array data to bytes. In Zarr, the main codec to use for numeric arrays
20+ is the :class: `zarr.codecs.BytesCodec `. Bytes-to-bytes transform the serialized bytestreams
21+ of the array data. Examples include compression codecs, such as
22+ :class: `zarr.codecs.GzipCodec `, :class: `zarr.codecs.BloscCodec ` or
23+ :class: `zarr.codecs.ZstdCodec `, and codecs that add a checksum to the bytestream, such as
2424:class: `zarr.codecs.Crc32cCodec `.
2525
26- Custom codecs for Zarr are implemented by subclassing the relevant base class, see
27- :class: `zarr.abc.codec.ArrayArrayCodec `, :class: `zarr.abc.codec.ArrayBytesCodec ` and
28- :class: `zarr.abc.codec.BytesBytesCodec `. Most custom codecs should implemented the
29- ``_encode_single `` and ``_decode_single `` methods. These methods operate on single chunks
26+ Custom codecs for Zarr are implemented by subclassing the relevant base class, see
27+ :class: `zarr.abc.codec.ArrayArrayCodec `, :class: `zarr.abc.codec.ArrayBytesCodec ` and
28+ :class: `zarr.abc.codec.BytesBytesCodec `. Most custom codecs should implemented the
29+ ``_encode_single `` and ``_decode_single `` methods. These methods operate on single chunks
3030of the array data. Alternatively, custom codecs can implement the ``encode `` and ``decode ``
31- methods, which operate on batches of chunks, in case the codec is intended to implement
31+ methods, which operate on batches of chunks, in case the codec is intended to implement
3232its own batch processing.
3333
3434Custom codecs should also implement the following methods:
3535
36- - ``compute_encoded_size ``, which returns the byte size of the encoded data given the byte
37- size of the original data. It should raise ``NotImplementedError `` for codecs with
36+ - ``compute_encoded_size ``, which returns the byte size of the encoded data given the byte
37+ size of the original data. It should raise ``NotImplementedError `` for codecs with
3838 variable-sized outputs, such as compression codecs.
39- - ``validate ``, which can be used to check that the codec metadata is compatible with the
39+ - ``validate ``, which can be used to check that the codec metadata is compatible with the
4040 array metadata. It should raise errors if not.
41- - ``resolve_metadata `` (optional), which is important for codecs that change the shape,
41+ - ``resolve_metadata `` (optional), which is important for codecs that change the shape,
4242 dtype or fill value of a chunk.
43- - ``evolve_from_array_spec `` (optional), which can be useful for automatically filling in
43+ - ``evolve_from_array_spec `` (optional), which can be useful for automatically filling in
4444 codec configuration metadata from the array metadata.
4545
46- To use custom codecs in Zarr, they need to be registered using the
46+ To use custom codecs in Zarr, they need to be registered using the
4747`entrypoint mechanism <https://packaging.python.org/en/latest/specifications/entry-points/ >`_.
48- Commonly, entrypoints are declared in the ``pyproject.toml `` of your package under the
49- ``[project.entry-points."zarr.codecs"] `` section. Zarr will automatically discover and
48+ Commonly, entrypoints are declared in the ``pyproject.toml `` of your package under the
49+ ``[project.entry-points."zarr.codecs"] `` section. Zarr will automatically discover and
5050load all codecs registered with the entrypoint mechanism from imported modules.
5151
5252.. code-block :: toml
5353
5454 [project.entry-points."zarr.codecs"]
5555 "custompackage.fancy_codec" = "custompackage:FancyCodec"
5656
57- New codecs need to have their own unique identifier. To avoid naming collisions, it is
58- strongly recommended to prefix the codec identifier with a unique name. For example,
57+ New codecs need to have their own unique identifier. To avoid naming collisions, it is
58+ strongly recommended to prefix the codec identifier with a unique name. For example,
5959the codecs from ``numcodecs `` are prefixed with ``numcodecs. ``, e.g. ``numcodecs.delta ``.
6060
6161.. note ::
62- Note that the extension mechanism for the Zarr version 3 is still under development.
63- Requirements for custom codecs including the choice of codec identifiers might
62+ Note that the extension mechanism for the Zarr version 3 is still under development.
63+ Requirements for custom codecs including the choice of codec identifiers might
6464 change in the future.
6565
66- It is also possible to register codecs as replacements for existing codecs. This might be
67- useful for providing specialized implementations, such as GPU-based codecs. In case of
66+ It is also possible to register codecs as replacements for existing codecs. This might be
67+ useful for providing specialized implementations, such as GPU-based codecs. In case of
6868multiple codecs, the :mod: `zarr.core.config ` mechanism can be used to select the preferred
69- implementation.
69+ implementation.
7070
7171.. note ::
7272 This sections explains how custom codecs can be created for Zarr version 3. For Zarr
73- version 2, codecs should subclass the
74- `numcodecs.abc.Codec <https://numcodecs.readthedocs.io/en/stable/abc.html#numcodecs.abc.Codec >`_
75- base class and register through
73+ version 2, codecs should subclass the
74+ `numcodecs.abc.Codec <https://numcodecs.readthedocs.io/en/stable/abc.html#numcodecs.abc.Codec >`_
75+ base class and register through
7676 `numcodecs.registry.register_codec <https://numcodecs.readthedocs.io/en/stable/registry.html#numcodecs.registry.register_codec >`_.
7777
7878
0 commit comments