Skip to content

Commit 607777c

Browse files
committed
custom codecs
1 parent e5e5fb3 commit 607777c

File tree

1 file changed

+27
-20
lines changed

1 file changed

+27
-20
lines changed

docs/user-guide/extending.rst

Lines changed: 27 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -6,12 +6,12 @@ Zarr-Python 3 was designed to be extensible. This means that you can extend
66
the library by writing custom classes and plugins. Currently, Zarr can be extended
77
in the following ways:
88

9-
Writing custom stores
10-
---------------------
9+
Custom stores
10+
-------------
1111

1212

13-
Writing custom codecs
14-
---------------------
13+
Custom codecs
14+
-------------
1515

1616
There are three types of codecs in Zarr: array-to-array, array-to-bytes, and bytes-to-bytes.
1717
Array-to-array codecs are used to transform the n-dimensional array data before serializing
@@ -23,15 +23,16 @@ of the array data. Examples include compression codecs, such as
2323
:class:`zarr.codecs.ZstdCodec`, and codecs that add a checksum to the bytestream, such as
2424
:class:`zarr.codecs.Crc32cCodec`.
2525

26-
Custom codecs for Zarr are implemented as classes that inherit from the relevant base class,
27-
see :class:`zarr.abc.codecs.ArrayArrayCodec`, :class:`zarr.abc.codecs.ArrayBytesCodec` and
28-
:class:`zarr.abc.codecs.BytesBytesCodec`. Most custom codecs should implemented the
26+
Custom codecs for Zarr are implemented by subclassing the relevant base class, see
27+
:class:`zarr.abc.codec.ArrayArrayCodec`, :class:`zarr.abc.codec.ArrayBytesCodec` and
28+
:class:`zarr.abc.codec.BytesBytesCodec`. Most custom codecs should implemented the
2929
``_encode_single`` and ``_decode_single`` methods. These methods operate on single chunks
30-
of the array data. Custom codecs can also implement the ``encode`` and ``decode`` methods,
31-
which operate on batches of chunks, in case the codec is intended to implement its own
32-
batch processing.
30+
of the array data. Alternatively, custom codecs can implement the ``encode`` and ``decode``
31+
methods, which operate on batches of chunks, in case the codec is intended to implement
32+
its own batch processing.
33+
34+
Custom codecs should also implement the following methods:
3335

34-
Custom codecs should also implement these methods:
3536
- ``compute_encoded_size``, which returns the byte size of the encoded data given the byte
3637
size of the original data. It should raise ``NotImplementedError`` for codecs with
3738
variable-sized outputs, such as compression codecs.
@@ -43,10 +44,12 @@ Custom codecs should also implement these methods:
4344
codec configuration metadata from the array metadata.
4445

4546
To use custom codecs in Zarr, they need to be registered using the
46-
`entrypoint mechanism <https://packaging.python.org/en/latest/specifications/entry-points/>_`.
47+
`entrypoint mechanism <https://packaging.python.org/en/latest/specifications/entry-points/>`_.
4748
Commonly, entrypoints are declared in the ``pyproject.toml`` of your package under the
48-
``[project.entry-points]`` section. Zarr will automatically discover and load all codecs
49-
registered with the entrypoint mechanism from imported modules.
49+
``[project.entry-points."zarr.codecs"]`` section. Zarr will automatically discover and
50+
load all codecs registered with the entrypoint mechanism from imported modules.
51+
52+
.. code-block:: toml
5053
5154
[project.entry-points."zarr.codecs"]
5255
"custompackage.fancy_codec" = "custompackage:FancyCodec"
@@ -56,9 +59,9 @@ strongly recommended to prefix the codec identifier with a unique name. For exam
5659
the codecs from ``numcodecs`` are prefixed with ``numcodecs.``, e.g. ``numcodecs.delta``.
5760

5861
.. note::
59-
Note that the extension mechanism for the Zarr specification version 3 is still
60-
under development. Requirements for custom codecs including the choice of codec
61-
identifiers might change in the future.
62+
Note that the extension mechanism for the Zarr version 3 is still under development.
63+
Requirements for custom codecs including the choice of codec identifiers might
64+
change in the future.
6265

6366
It is also possible to register codecs as replacements for existing codecs. This might be
6467
useful for providing specialized implementations, such as GPU-based codecs. In case of
@@ -69,10 +72,14 @@ TODO: Link to documentation of :mod:`zarr.core.config`
6972

7073
.. note::
7174
This sections explains how custom codecs can be created for Zarr version 3. For Zarr
72-
version 2, codecs should implement the
73-
```numcodecs.abc.Codec`` <https://numcodecs.readthedocs.io/en/stable/abc.html>_`
74-
base class.
75+
version 2, codecs should subclass the
76+
`numcodecs.abc.Codec <https://numcodecs.readthedocs.io/en/stable/abc.html#numcodecs.abc.Codec>`_
77+
base class and register through
78+
`numcodecs.registry.register_codec <https://numcodecs.readthedocs.io/en/stable/registry.html#numcodecs.registry.register_codec>`_.
79+
7580

81+
Other extensions
82+
----------------
7683

7784
In the future, Zarr will support writing custom custom data types and chunk grids.
7885

0 commit comments

Comments
 (0)