Skip to content

Commit 4f3381f

Browse files
committed
define native data type and native scalar
1 parent 44369d6 commit 4f3381f

File tree

1 file changed

+7
-5
lines changed

1 file changed

+7
-5
lines changed

docs/user-guide/data_types.rst

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -58,21 +58,23 @@ Zarr-Python supports two different Zarr formats, and those two formats specify d
5858
data types in Zarr version 2 are encoded as Numpy-compatible strings, while data types in Zarr version 3 are encoded as either strings or ``JSON`` objects,
5959
and the Zarr V3 data types don't have any associated endianness information, unlike Zarr V2 data types.
6060

61-
We also want Zarr-Python to support data types beyond what's available in Numpy. So it's crucial that we have a
62-
model of array data types that can adapt to the differences between Zarr V2 and V3 and doesn't over-fit to Numpy.
61+
We aspire for Zarr-Python to eventually be array-library-agnostic.
62+
In the context of data types, this means that we should not design an API that overfits to Numpy's data types.
63+
We will use the term "native data type" to refer to a data type used by any external array library (including Numpy), e.g. ``np.dtypes.Float64DType()``.
64+
We will also use the term "native scalar" or "native scalar type" to refer to a scalar value of a native data type. For example, ``np.float64(0)`` generates a scalar with the data dtype ``np.dtypes.Float64DType``
6365

64-
Here are the operations we need to perform on data types in Zarr-Python:
66+
Zarr-Python needs to support the following operations on native data types:
6567

6668
* Round-trip native data types to fields in array metadata documents.
6769
For example, the Numpy data type ``np.dtype('>i2')`` should be saved as ``{..., "dtype" : ">i2"}`` in Zarr V2 metadata.
6870

6971
In Zarr V3 metadata, the same Numpy data type would be saved as ``{..., "data_type": "int16", "codecs": [..., {"name": "bytes", "configuration": {"endian": "big"}, ...]}``
7072

71-
* Define a default fill value. This is not mandated by the Zarr specifications, but it's convenient for users
73+
* Associate a default fill value with a native data type. This is not mandated by the Zarr specifications, but it's convenient for users
7274
to have a useful default. For numeric types like integers and floats the default can be statically set to 0, but for
7375
parametric data types like fixed-length strings the default can only be generated after the data type has been parametrized at runtime.
7476

75-
* Round-trip scalars to the ``fill_value`` field in Zarr V2 and V3 array metadata documents. The Zarr V2 and V3 specifications
77+
* Round-trip native scalars to the ``fill_value`` field in Zarr V2 and V3 array metadata documents. The Zarr V2 and V3 specifications
7678
define how scalars of each data type should be stored as JSON in array metadata documents, and in principle each data type
7779
can define this encoding separately.
7880

0 commit comments

Comments
 (0)