Skip to content

Commit 895640c

Browse files
normanrznormanrz
committed
merge
2 parents 49941bf + 01b73a7 commit 895640c

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

77 files changed

+4031
-2701
lines changed

‎.github/workflows/gpu_test.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ jobs:
2525
strategy:
2626
matrix:
2727
python-version: ['3.11']
28-
numpy-version: ['2.0']
28+
numpy-version: ['2.1']
2929
dependency-set: ["minimal"]
3030

3131
steps:

‎.github/workflows/hypothesis.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ jobs:
2626
strategy:
2727
matrix:
2828
python-version: ['3.11']
29-
numpy-version: ['1.26']
29+
numpy-version: ['2.1']
3030
dependency-set: ["optional"]
3131

3232
steps:

‎.github/workflows/releases.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,7 @@ jobs:
5555
with:
5656
name: releases
5757
path: dist
58-
- uses: pypa/gh-action-pypi-publish@v1.10.3
58+
- uses: pypa/gh-action-pypi-publish@v1.12.2
5959
with:
6060
user: __token__
6161
password: ${{ secrets.pypi_password }}

‎.github/workflows/test.yml‎

Lines changed: 40 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -16,14 +16,32 @@ concurrency:
1616

1717
jobs:
1818
test:
19-
name: py=${{ matrix.python-version }}, np=${{ matrix.numpy-version }}, deps=${{ matrix.dependency-set }}
19+
name: os=${{ matrix.os }}, py=${{ matrix.python-version }}, np=${{ matrix.numpy-version }}, deps=${{ matrix.dependency-set }}
2020

21-
runs-on: ubuntu-latest
2221
strategy:
2322
matrix:
2423
python-version: ['3.11', '3.12', '3.13']
25-
numpy-version: ['1.25', '1.26', '2.0']
24+
numpy-version: ['1.25', '2.1']
2625
dependency-set: ["minimal", "optional"]
26+
os: ["ubuntu-latest"]
27+
include:
28+
- python-version: '3.11'
29+
numpy-version: '1.25'
30+
dependency-set: 'optional'
31+
os: 'macos-latest'
32+
- python-version: '3.13'
33+
numpy-version: '2.1'
34+
dependency-set: 'optional'
35+
os: 'macos-latest'
36+
- python-version: '3.11'
37+
numpy-version: '1.25'
38+
dependency-set: 'optional'
39+
os: 'windows-latest'
40+
- python-version: '3.13'
41+
numpy-version: '2.1'
42+
dependency-set: 'optional'
43+
os: 'windows-latest'
44+
runs-on: ${{ matrix.os }}
2745

2846
steps:
2947
- uses: actions/checkout@v4
@@ -75,3 +93,22 @@ jobs:
7593
- name: Run Tests
7694
run: |
7795
hatch env run --env ${{ matrix.dependency-set }} run
96+
97+
test-complete:
98+
name: Test complete
99+
100+
needs:
101+
[
102+
test,
103+
test-upstream-and-min-deps,
104+
]
105+
if: always()
106+
runs-on: ubuntu-latest
107+
steps:
108+
- name: Check failure
109+
if: |
110+
contains(needs.*.result, 'failure') ||
111+
contains(needs.*.result, 'cancelled')
112+
run: exit 1
113+
- name: Success
114+
run: echo Success!

‎.pre-commit-config.yaml‎

Lines changed: 5 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,12 @@
11
ci:
22
autoupdate_commit_msg: "chore: update pre-commit hooks"
3+
autoupdate_schedule: "monthly"
34
autofix_commit_msg: "style: pre-commit fixes"
45
autofix_prs: false
56
default_stages: [pre-commit, pre-push]
6-
default_language_version:
7-
python: python3
87
repos:
98
- repo: https://github.com/astral-sh/ruff-pre-commit
10-
rev: v0.6.9
9+
rev: v0.8.2
1110
hooks:
1211
- id: ruff
1312
args: ["--fix", "--show-fixes"]
@@ -16,29 +15,25 @@ repos:
1615
rev: v2.3.0
1716
hooks:
1817
- id: codespell
19-
args: ["-L", "ba,ihs,kake,nd,noe,nwo,te,fo,zar", "-S", "fixture"]
18+
args: ["-L", "fo,ihs,kake,te", "-S", "fixture"]
2019
- repo: https://github.com/pre-commit/pre-commit-hooks
2120
rev: v5.0.0
2221
hooks:
2322
- id: check-yaml
2423
- repo: https://github.com/pre-commit/mirrors-mypy
25-
rev: v1.11.2
24+
rev: v1.13.0
2625
hooks:
2726
- id: mypy
2827
files: src|tests
2928
additional_dependencies:
3029
# Package dependencies
31-
- asciitree
32-
- crc32c
3330
- donfig
34-
- numcodecs
31+
- numcodecs[crc32c]
3532
- numpy
3633
- typing_extensions
3734
- universal-pathlib
3835
# Tests
3936
- pytest
40-
# Zarr v2
41-
- types-redis
4237
- repo: https://github.com/scientific-python/cookie
4338
rev: 2024.08.19
4439
hooks:

‎README.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,13 @@
7878
</td>
7979
</tr>
8080
<tr>
81+
<td>Funding</td>
82+
<td>
83+
<a href="https://chanzuckerberg.com/eoss/">
84+
<img src="https://img.shields.io/badge/funded%20by-EOSS-FF414B.svg?logo=data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSI0NSIgaGVpZ2h0PSI0NSI+PHBhdGggZD0iTTIyLjA5OCAyMS4wNzRjLTEuMjM1Ljg2LTIuNjggMS4zOTktNC42ODggMS40MzgtMi40MjYuMDUtNC4yMzgtMS41NTEtNC40MDYtMy45MThhNC40NjUgNC40NjUgMCAwIDEgMS4xODctMy4zOCA0LjQ4IDQuNDggMCAwIDEgMy4yODYtMS40NDQgNC42NzQgNC42NzQgMCAwIDEgMS44OTQuMzc1cy0uMTU2IDEuMzItLjIxIDEuOTE0bDEuOTg3LjAxNS4zNTYtMi45NzYtLjU2My0uMzU2YTYuNTQxIDYuNTQxIDAgMCAwLTMuNDg0LS45NjkgNi41OTMgNi41OTMgMCAwIDAtNC43NSAyLjA4NiA2LjQxNSA2LjQxNSAwIDAgMC0xLjcxNSA0Ljg3NWMuMTEzIDEuNjEuODA5IDMuMDc4IDEuOTUgNC4xNDEgMS4xNTYgMS4wNyAyLjcxNCAxLjY0NSA0LjM5OCAxLjYzMy4wMzkgMCAuMDc4IDAgLjEyNS0uMDA0YTkuOTE4IDkuOTE4IDAgMCAwIDMuNDMzLS43MjNsMS40MzQtMi43ODlzLS4wMjctLjA2Mi0uMjM0LjA3OCIgc3R5bGU9InN0cm9rZTpub25lO2ZpbGwtcnVsZTpub256ZXJvO2ZpbGw6I2ZmZjtmaWxsLW9wYWNpdHk6MSIvPjxwYXRoIGQ9Im0yOC44NjMgMjguMjE1LS4yNDIgMi43NDJ2LjAzMWMtLjEwMSAxLjAzNS0uNjUyIDEuOTE0LTEuNDg0IDIuMzUyLS42Ni4zNDctMS4zOC4zNDctMi4wMjgtLjAwOC0uNjc1LS4zNjMtLjk5Mi0uOTE4LTEuMTE3LTEuNjEzLS4xODctMS4xMDIuNDM4LTIuMjU0IDEuMjkzLTIuODMybDEzLjM1Mi04LjY4OGMuMDk3LjczOC4xNTIgMS40ODUuMTUyIDIuMjUgMCA4Ljk5Mi03LjMwOSAxNi4zMDUtMTYuMjkzIDE2LjMwNS04Ljk4OCAwLTE2LjI4OS03LjMxMy0xNi4yODktMTYuMzA1IDAtOC45ODggNy4zMDUtMTYuMyAxNi4yOTMtMTYuM3MxMi40OTYgNC4wOSAxNC45ODQgOS45MUwzOS4xMTQgMTVDMzYuMjYxIDguNjY0IDI5Ljg5MyA0LjIzNCAyMi41IDQuMjM0Yy03LjM5NSAwLTE4LjIxNSA4LjE3Mi0xOC4yMTUgMTguMjE1IDAgMTAuMDQ3IDguMTcyIDE4LjIxMSAxOC4yMTEgMTguMjExIDEwLjA0IDAgMTguMjE1LTguMTcyIDE4LjIxNS0xOC4yMSAwLTEwLjA0LS4xMS0yLjI5NC0uMzEzLTMuMzkxYTE5Ljk5NyAxOS45OTcgMCAwIDAtLjQ1My0xLjgzMmwtMy43OTMgMi4zNy01LjAyIDMuMThjLS4xNzUtLjY2OC0uNTgxLTEuMzQzLTEuNDQtMS43My0xLjAxMi0uNDY1LTIuNjYtLjExLTMuODY4LjU4MiAwIDAgMy4zMDUtNi40MDIgMy44Ni03LjQ1LjAzNS0uMDY2LS4wMTYtLjE0OC0uMDk4LS4xNDhoLTYuMThsLS4yNSAyLjA2N0gyNi41MDhsLTQuNzMgOS4wNjJjLS4wOTQuMTc2LjA5Ny4zNjMuMjc3LjI3NGwyLjUyMy0xLjM0YzEuMjE5LS42MjUgMy4yNS0xLjkxIDQuMjU0LTEuMTY4LjE0NS4xMS4zMzIuMzk4LjM0OC42OTVhLjM1Ny4zNTcgMCAwIDEtLjE0OS4yOTNsLTUuMDQzIDMuNDA2Yy0xLjYzNiAxLjE2OC0yLjI3NyAyLjkxLTIuMTIgNC41NTUuMTI0IDEuMzc1Ljk4NCAyLjU2NiAyLjI1NyAzLjI2MmE0LjE3IDQuMTcgMCAwIDAgMi4xMTMuNTE1IDQuMTY5IDQuMTY5IDAgMCAwIDEuODY0LS41YzEuNDM3LS43NTQgMi4zOTQtMi4yMzQgMi41NjItMy45NDVsLjQwMi00LjYxMy0yLjE5OSAxLjYzM1ptMCAwIiBzdHlsZT0ic3Ryb2tlOm5vbmU7ZmlsbC1ydWxlOm5vbnplcm87ZmlsbDojZmZmO2ZpbGwtb3BhY2l0eToxIi8+PC9zdmc+" alt="CZI's Essential Open Source Software for Science">
85+
</a>
86+
</td>
87+
</tr>
8188
<td>Citation</td>
8289
<td>
8390
<a href="https://doi.org/10.5281/zenodo.3773450">

‎docs/contributing.rst‎

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -92,12 +92,11 @@ the following::
9292
$ mkdir -p ~/pyenv/zarr-dev
9393
$ python -m venv ~/pyenv/zarr-dev
9494
$ source ~/pyenv/zarr-dev/bin/activate
95-
$ pip install -r requirements_dev_minimal.txt -r requirements_dev_numpy.txt
96-
$ pip install -e .[docs]
95+
$ pip install -e .[test,docs]
9796

9897
To verify that your development environment is working, you can run the unit tests::
9998

100-
$ python -m pytest -v zarr
99+
$ python -m pytest -v tests
101100

102101
Creating a branch
103102
~~~~~~~~~~~~~~~~~
@@ -149,7 +148,7 @@ and invoke::
149148
Some tests require optional dependencies to be installed, otherwise
150149
the tests will be skipped. To install all optional dependencies, run::
151150

152-
$ pip install -r requirements_dev_optional.txt
151+
$ pip install pytest-doctestplus
153152

154153
To also run the doctests within docstrings (requires optional
155154
dependencies to be installed), run::
@@ -234,7 +233,7 @@ should run and pass as doctests under Python 3.8. To run doctests,
234233
activate your development environment, install optional requirements,
235234
and run::
236235

237-
$ python -m pytest -v --doctest-plus zarr
236+
$ python -m pytest -v --doctest-plus tests
238237

239238
Zarr uses Sphinx for documentation, hosted on readthedocs.org. Documentation is
240239
written in the RestructuredText markup language (.rst files) in the ``docs`` folder.

‎docs/guide/storage.rst‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ Storage
22
=======
33

44
Zarr-Python supports multiple storage backends, including: local file systems,
5-
Zip files, remote stores via ``fspec`` (S3, HTTP, etc.), and in-memory stores. In
5+
Zip files, remote stores via ``fsspec`` (S3, HTTP, etc.), and in-memory stores. In
66
Zarr-Python 3, stores must implement the abstract store API from
77
:class:`zarr.abc.store.Store`.
88

@@ -19,9 +19,9 @@ to Zarr's top level API will result in the store being created automatically.
1919
.. code-block:: python
2020
2121
>>> import zarr
22-
>>> zarr.open("data/foo/bar", mode="r") # implicitly creates a LocalStore
22+
>>> zarr.open("data/foo/bar", mode="r") # implicitly creates a read-only LocalStore
2323
<Group file://data/foo/bar>
24-
>>> zarr.open("s3://foo/bar", mode="r") # implicitly creates a RemoteStore
24+
>>> zarr.open("s3://foo/bar", mode="r") # implicitly creates a read-only RemoteStore
2525
<Group s3://foo/bar>
2626
>>> data = {}
2727
>>> zarr.open(data, mode="w") # implicitly creates a MemoryStore
@@ -43,7 +43,7 @@ filesystem.
4343
.. code-block:: python
4444
4545
>>> import zarr
46-
>>> store = zarr.storage.LocalStore("data/foo/bar", mode="r")
46+
>>> store = zarr.storage.LocalStore("data/foo/bar", read_only=True)
4747
>>> zarr.open(store=store)
4848
<Group file://data/foo/bar>
4949
@@ -72,7 +72,7 @@ that implements the `AbstractFileSystem` API,
7272
.. code-block:: python
7373
7474
>>> import zarr
75-
>>> store = zarr.storage.RemoteStore("gs://foo/bar", mode="r")
75+
>>> store = zarr.storage.RemoteStore.from_url("gs://foo/bar", read_only=True)
7676
>>> zarr.open(store=store)
7777
<Array <RemoteStore(GCSFileSystem, foo/bar)> shape=(10, 20) dtype=float32>
7878
@@ -86,7 +86,7 @@ Zarr data (metadata and chunks) to a dictionary.
8686
8787
>>> import zarr
8888
>>> data = {}
89-
>>> store = zarr.storage.MemoryStore(data, mode="w")
89+
>>> store = zarr.storage.MemoryStore(data)
9090
>>> zarr.open(store=store, shape=(2, ))
9191
<Array memory://4943638848 shape=(2,) dtype=float64>
9292

‎docs/installation.rst‎

Lines changed: 24 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,27 +1,37 @@
11
Installation
22
============
33

4-
Zarr depends on NumPy. It is generally best to `install NumPy
5-
<https://numpy.org/doc/stable/user/install.html>`_ first using whatever method is most
6-
appropriate for your operating system and Python distribution. Other dependencies should be
7-
installed automatically if using one of the installation methods below.
4+
pip
5+
---
86

9-
Note: Zarr has endorsed `Scientific-Python SPEC 0 <https://scientific-python.org/specs/spec-0000/>`_ and now follows the version support window as outlined below:
7+
.. code-block:: console
108
11-
- Python: 36 months after initial release
12-
- Core package dependencies (e.g. NumPy): 24 months after initial release
9+
$ pip install zarr
1310
14-
Install Zarr from PyPI::
11+
There are a number of optional dependency groups you can install for extra functionality.
12+
These can be installed using ``pip install "zarr[<extra>]"``, e.g. ``pip install "zarr[gpu]"``
1513

16-
$ pip install zarr
14+
- ``gpu``: support for GPUs
15+
- ``fsspec``: support for reading/writing to remote data stores
16+
- ``tree``: support for pretty printing of directory trees
1717

18-
Alternatively, install Zarr via conda::
18+
conda
19+
-----
20+
21+
.. code-block:: console
1922
2023
$ conda install -c conda-forge zarr
2124
22-
To install the latest development version of Zarr, you can use pip with the
23-
latest GitHub main::
25+
Conda does not support optional dependencies, so you will have to manually install any packages
26+
needed to enable extra functionality.
27+
28+
Dependency support
29+
------------------
30+
Zarr has endorsed `Scientific-Python SPEC 0 <https://scientific-python.org/specs/spec-0000/>`_ and now follows the version support window as outlined below:
2431

25-
$ pip install git+https://github.com/zarr-developers/zarr-python.git
32+
- Python: 36 months after initial release
33+
- Core package dependencies (e.g. NumPy): 24 months after initial release
2634

27-
To work with Zarr source code in development, see `Contributing <contributing.html>`_.
35+
Development
36+
-----------
37+
To install the latest development version of Zarr, see `the contributing guide <contributing.html>`_.

‎docs/release.rst‎

Lines changed: 24 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,10 +18,32 @@ Release notes
1818
See `GH1777 <https://github.com/zarr-developers/zarr-python/issues/1777>`_ for more details on the upcoming
1919
3.0 release.
2020

21+
.. release_3.0.0-beta:
22+
23+
3.0.0-beta series
24+
-----------------
25+
26+
.. warning::
27+
Zarr-Python 3.0.0-beta is a pre-release of the upcoming 3.0 release. This release is not feature complete or
28+
expected to be ready for production applications.
29+
30+
.. note::
31+
The complete release notes for 3.0 have not been added to this document yet. See the
32+
`3.0.0-beta <https://github.com/zarr-developers/zarr-python/releases/tag/v3.0.0-beta>`_ release on GitHub
33+
for a record of changes included in this release.
34+
35+
Dependency Changes
36+
~~~~~~~~~~~~~~~~~~
37+
38+
* fsspec was moved from a required dependency to an optional one. Users should install
39+
fsspec and any relevant implementations (e.g. s3fs) before using the ``RemoteStore``.
40+
By :user:`Joe Hamman <jhamman>` :issue:`2391`.
41+
42+
2143
.. release_3.0.0-alpha:
2244
23-
3.0.0-alpha
24-
-----------
45+
3.0.0-alpha series
46+
------------------
2547

2648
.. warning::
2749
Zarr-Python 3.0.0-alpha is a pre-release of the upcoming 3.0 release. This release is not feature complete or

0 commit comments

Comments
 (0)