Skip to content

Commit 78c5f89

Browse files
committed
container-specific considerations
1 parent 2896d72 commit 78c5f89

File tree

1 file changed

+18
-1
lines changed

1 file changed

+18
-1
lines changed

spec/latest/index.bs

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -868,5 +868,22 @@ manner, using the corresponding dataset names previously defined.
868868
Supported Binary Containers {#supported_containers}
869869
---------------------------------------------------
870870

871-
Currently supported binary containers include HDF5 and NetCDF (but should include more).
871+
Currently supported binary containers include HDF5, NPZ, Zarr, and an In-Memory interchange format.
872872

873+
HDF5-Specific Considerations
874+
----------------------------
875+
876+
In HDF5, arrays can be stored in nested "groups", similar to directories in a
877+
filesystem. We assume that each binsparse tensor will take the form of an HDF5
878+
group. In addition to normal datasets stored in the group, some metadata for
879+
each group may be stored as an "attribute". The JSON header for a binsparse
880+
group is stored in the HDF5 group string attribute named "binsparse", and each
881+
constituent data array (e.g. `pointers_to_1`, `fill_value`), is stored as an
882+
HDF5 dataset with the name prescribed by this spec.
883+
884+
NPY-Specific Considerations
885+
---------------------------
886+
The NPY file format is a simple file format provided by Numpy, which stores a
887+
single array to disk. A standard binsparse file in NPY consists of a directory
888+
containing the multiple binsparse data array files, together with a standard
889+
json string header file in the same directory named "binsparse.json".

0 commit comments

Comments
 (0)