Skip to content

Conversation

jhamman
Copy link
Member

@jhamman jhamman commented Oct 18, 2024

This is a work in progress. Running through the tutorial helped identify a number of issues that we likely want to address before we release 3.0.

TODO:

  • Add unit tests and/or doctests in docstrings
  • Add docstrings and API docs for any new/modified user-facing classes and functions
  • New/modified features documented in docs/tutorial.rst
  • Changes documented in docs/release.rst
  • GitHub Actions have all passed
  • Test coverage is 100% (Codecov passes)

Copy link
Member Author

@jhamman jhamman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

self review 👇

>>> compressor = Blosc(cname='zstd', clevel=3, shuffle=Blosc.BITSHUFFLE)
>>> data = np.arange(100000000, dtype='i4').reshape(10000, 10000)
>>> z = zarr.array(data, chunks=(1000, 1000), compressor=compressor)
>>> z = zarr.array(data, chunks=(1000, 1000), compressor=compressor, zarr_format=2)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This example works for zarr_format=2 but requires #2398 to work for zarr_format=3

Groups can also contain arrays, e.g.::

>>> z1 = bar.zeros('baz', shape=(10000, 10000), chunks=(1000, 1000), dtype='i4')
>>> z1 = bar.zeros(name='baz', shape=(10000, 10000), chunks=(1000, 1000), dtype='i4')
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
>>> z1 = bar.zeros(name='baz', shape=(10000, 10000), chunks=(1000, 1000), dtype='i4')
>>> z1 = bar.zeros('baz', shape=(10000, 10000), chunks=(1000, 1000), dtype='i4')

see #2402

Arrays can also be created with the ``create_array()`` and ``require_array()`` methods, e.g.::

>>> z = bar.create_dataset('quux', shape=(10000, 10000), chunks=(1000, 1000), dtype='i4')
>>> z = bar.create_array(name='quux', shape=(10000, 10000), chunks=(1000, 1000), dtype='i4')
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
>>> z = bar.create_array(name='quux', shape=(10000, 10000), chunks=(1000, 1000), dtype='i4')
>>> z = bar.create_array('quux', shape=(10000, 10000), chunks=(1000, 1000), dtype='i4')

see #2402

<zarr.hierarchy.Group '/'>
>>> z = root.zeros('foo/bar/baz', shape=(10000, 10000), chunks=(1000, 1000), dtype='i4')
<Group file://data/group.zarr>
>>> z = root.zeros(name='foo/bar/baz', shape=(10000, 10000), chunks=(1000, 1000), dtype='i4')
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
>>> z = root.zeros(name='foo/bar/baz', shape=(10000, 10000), chunks=(1000, 1000), dtype='i4')
>>> z = root.zeros('foo/bar/baz', shape=(10000, 10000), chunks=(1000, 1000), dtype='i4')

#2402

>>> root = zarr.group()
>>> foo = root.create_group('foo')
>>> bar = foo.zeros('bar', shape=1000000, chunks=100000, dtype='i8')
>>> bar = foo.zeros(name='bar', shape=1000000, chunks=100000, dtype='i8')
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
>>> bar = foo.zeros(name='bar', shape=1000000, chunks=100000, dtype='i8')
>>> bar = foo.zeros('bar', shape=1000000, chunks=100000, dtype='i8')

#2402

>>> bar = foo.zeros(name='bar', shape=1000000, chunks=100000, dtype='i8')
>>> bar[:] = 42
>>> baz = foo.zeros('baz', shape=(1000, 1000), chunks=(100, 100), dtype='f4')
>>> baz = foo.zeros(name='baz', shape=(1000, 1000), chunks=(100, 100), dtype='f4')
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
>>> baz = foo.zeros(name='baz', shape=(1000, 1000), chunks=(100, 100), dtype='f4')
>>> baz = foo.zeros('baz', shape=(1000, 1000), chunks=(100, 100), dtype='f4')

#2402

>>> root = zarr.group()
>>> root.attrs['foo'] = 'bar'
>>> z = root.zeros('zzz', shape=(10000, 10000))
>>> z = root.zeros(name='zzz', shape=(10000, 10000))
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
>>> z = root.zeros(name='zzz', shape=(10000, 10000))
>>> z = root.zeros('zzz', shape=(10000, 10000))

#2402

arrays, as long as the units are specified. E.g.::

>>> z = zarr.array(['2007-07-13', '2006-01-13', '2010-08-13'], dtype='M8[D]')
>>> z = zarr.array(['2007-07-13', '2006-01-13', '2010-08-13'], dtype='M8[D]', zarr_format=2)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add note above explaining that datetimes/timedeltas are only supported by v2 data format for now.

@dstansby
Copy link
Contributor

Woop woop! In case you don't know about it already, https://github.com/scientific-python/pytest-doctestplus?tab=readme-ov-file#fixing-existing-docstrings might be helpful

@jhamman jhamman added the V3 label Nov 29, 2024
@dstansby dstansby added this to the 3.0.0 milestone Dec 9, 2024
@dstansby dstansby added documentation Improvements to the documentation and removed V3 labels Dec 12, 2024
@jhamman
Copy link
Member Author

jhamman commented Dec 30, 2024

closing in favor of #2589

@jhamman jhamman closed this Dec 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements to the documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants