Merge branch 'main' into 3.0.9-3.0.10-release-notes

Author: d-v-b

.github/ISSUE_TEMPLATE/release-checklist.md

@@ -0,0 +1,56 @@
+---
+name: Zarr-Python release checklist
+about: Checklist for a new Zarr-Python release. [For project maintainers only!]
+title: Release Zarr-Python vX.Y.Z
+labels: release-checklist
+assignees: ''
+
+---
+
+**Release**: [v0.x.x](https://github.com/zarr-developers/zarr-python/milestones/?)
+**Scheduled Date**: 20YY/MM/DD
+
+**Priority PRs/issues to complete prior to release**
+
+- [ ] Priority pull request #X
+
+**Before release**:
+
+- [ ] Make sure that all pull requests which will be included in the release have been properly documented as changelog files in the [`changes/` directory](https://github.com/zarr-developers/zarr-python/tree/main/changes).
+- [ ] Run ``towncrier build --version x.y.z`` to create the changelog, and commit the result to the main branch.
+- [ ] Check [SPEC 0](https://scientific-python.org/specs/spec-0000/#support-window) to see if the minimum supported version of Python or NumPy needs bumping.
+- [ ] Check to ensure that:
+  - [ ] Deprecated workarounds/codes/tests are removed. Run `grep "# TODO" **/*.py` to find all potential TODOs.
+  - [ ] All tests pass in the ["Tests" workflow](https://github.com/zarr-developers/zarr-python/actions/workflows/test.yml).
+  - [ ] All tests pass in the ["GPU Tests" workflow](https://github.com/zarr-developers/zarr-python/actions/workflows/gpu_test.yml).
+  - [ ] All tests pass in the ["Hypothesis" workflow](https://github.com/zarr-developers/zarr-python/actions/workflows/hypothesis.yaml).
+  - [ ] Check that downstream libraries work well (maintainers can make executive decisions about whether all checks are required for this release).
+    - [ ] Xarray (@jhamman @dcherian @TomNicholas)
+        - Zarr's upstream compatibility is tested via the [Upstream Dev CI worklow](https://github.com/pydata/xarray/actions/workflows/upstream-dev-ci.yaml).
+        - Click on the most recent workflow and check that the `upstream-dev` job has run and passed. `upstream-dev` is not run on all all workflow runs.
+        - Check that the expected version of Zarr-Python was tested using the `Version Info` step of the `upstream-dev` job.
+        - If testing on a branch other than `main` is needed, open a PR modifying https://github.com/pydata/xarray/blob/90ee30943aedba66a37856b2332a41264e288c20/ci/install-upstream-wheels.sh#L56 and add the `run-upstream` label.
+    - [ ] Titiler.Xarray (@maxrjones)
+        - [Modify dependencies](https://github.com/developmentseed/titiler/blob/main/src/titiler/xarray/pyproject.toml) for titiler.xarray.
+        - Modify triggers for running [the test workflow](https://github.com/developmentseed/titiler/blob/61549f2de07b20cca8fb991cfcdc89b23e18ad05/.github/workflows/ci.yml#L5-L7).
+        - Push the branch to the repository and check for the actions for any failures.
+
+**Release**:
+
+- [ ] Go to https://github.com/zarr-developers/zarr-python/releases.
+  - [ ] Click "Draft a new release".
+  - [ ] Choose a version number prefixed with a `v` (e.g. `v0.0.0`). For pre-releases, include the appropriate suffix (e.g. `v0.0.0a1` or `v0.0.0rc2`).
+  - [ ] Set the description of the release to: `See release notes https://zarr.readthedocs.io/en/stable/release-notes.html#release-0-0-0`, replacing the correct version numbers. For pre-release versions, the URL should omit the pre-release suffix, e.g. "a1" or "rc1".
+  - [ ] Click on "Generate release notes" to auto-fill the description.
+  - [ ] Make a release by clicking the 'Publish Release' button, this will automatically create a tag too.
+- [ ] Verify that release workflows succeeded.
+  - [ ] The latest version is correct on [PyPI](https://pypi.org/project/zarr/).
+  - [ ] The stable version is correct on [ReadTheDocs](https://zarr.readthedocs.io/en/stable/).
+
+**After release**:
+
+- [ ] Review and merge the pull request on the conda-forge [zarr-feedstock](https://github.com/conda-forge/zarr-feedstock) that will be automatically generated.
+
+---
+
+- [ ] Party :tada:

.github/codecov.yml

@@ -6,9 +6,9 @@ ci:
 default_stages: [pre-commit, pre-push]
 repos:
   - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.11.9
+    rev: v0.12.2
     hooks:
-      - id: ruff
+      - id: ruff-check
         args: ["--fix", "--show-fixes"]
       - id: ruff-format
   - repo: https://github.com/codespell-project/codespell

.pre-commit-config.yaml

@@ -1,11 +1,11 @@
 import sys
 import timeit
 
+import blosc
 import line_profiler
 import numpy as np
 
 import zarr
-from zarr import blosc
 
 if __name__ == "__main__":
     sys.path.insert(0, "..")

bench/compress_normal.py

@@ -0,0 +1 @@
+Add human readable size for No. bytes stored to `info_complete`

changes/3190.bugfix.rst

@@ -0,0 +1,4 @@
+- Add a description on how to create a RemoteStore of a specific filesystem to the `Remote Store` section in `docs\user-guide\storage.rst`.
+- State in the docstring of `FsspecStore.from_url` that the filesystem type is inferred from the URL scheme.
+
+It should help a user handling the case when the type of FsspecStore doesn't match the URL scheme.

changes/3212.doc.rst

@@ -3,12 +3,19 @@ coverage:
     patch:
       default:
         target: auto
-        after_n_builds: 10  # Wait for all 10 reports before updating the status
     project:
       default:
         target: auto
         threshold: 0.1
-        after_n_builds: 2   # Wait for all 10 reports before updating the status
-comment: false
+codecov:
+  notify:
+    after_n_builds: 10 # Wait for all 10 reports before updating the status
+    wait_for_ci: yes
+comment:
+  layout: "diff, files"
+  behavior: default
+  require_changes: true  # if true: only post the comment if coverage changes
+  branches:               # branch names that can post comment
+    - "main"
 github_checks:
-    annotations: false
+  annotations: false

codecov.yml

@@ -339,36 +339,6 @@ breaking changes may be more frequent than usual.
 Release procedure
 -----------------
 
-Pre-release
-~~~~~~~~~~~
-1. Make sure that all pull requests which will be included in the release
-   have been properly documented as changelog files in the :file:`changes/` directory.
-2. Run ``towncrier build --version x.y.z`` to create the changelog, and commit the result
-   to the main branch.
-
-Releasing
-~~~~~~~~~
-1. Go to https://github.com/zarr-developers/zarr-python/releases
-2. Click "Draft a new release".
-3. Choose a version number prefixed with a `v` (e.g. `v0.0.0`).
-   For pre-releases, include the appropriate suffix (e.g. `v0.0.0a1` or `v0.0.0rc2`).
-4. Set the description of the release to::
-
-    See release notes https://zarr.readthedocs.io/en/stable/release-notes.html#release-0-0-0
-
-   replacing the correct version numbers. For pre-release versions,
-   the URL should omit the pre-release suffix, e.g. "a1" or "rc1".
-5. Click on "Generate release notes" to auto-fill the description.
-
-After creating the release, the documentation will be built on
-https://readthedocs.io. Full releases will be available under
-`/stable <https://zarr.readthedocs.io/en/stable>`_ while
-pre-releases will be available under
-`/latest <https://zarr.readthedocs.io/en/latest>`_.
-
-Post-release
-~~~~~~~~~~~~
-
-- Review and merge the pull request on the
-  `conda-forge feedstock <https://github.com/conda-forge/zarr-feedstock>`_ that will be
-  automatically generated.
+Open an issue on GitHub announcing the release using the release checklist template:
+`https://github.com/zarr-developers/zarr-python/issues/new?template=release-checklist.md <https://github.com/zarr-developers/zarr-python/issues/new?template=release-checklist.md>`_.
+The release checklist includes all steps necessary for the release.

docs/developers/contributing.rst

@@ -212,7 +212,7 @@ prints additional diagnostics, e.g.::
    Serializer         : BytesCodec(endian=<Endian.little: 'little'>)
    Compressors        : (BloscCodec(typesize=4, cname=<BloscCname.zstd: 'zstd'>, clevel=3, shuffle=<BloscShuffle.bitshuffle: 'bitshuffle'>, blocksize=0),)
    No. bytes          : 400000000 (381.5M)
-   No. bytes stored   : 3558573
+   No. bytes stored   : 3558573 (3.4M)
    Storage ratio      : 112.4
    Chunks Initialized : 100
 
@@ -286,7 +286,7 @@ Here is an example using a delta filter with the Blosc compressor::
    >>> compressors = zarr.codecs.BloscCodec(cname='zstd', clevel=1, shuffle=zarr.codecs.BloscShuffle.shuffle)
    >>> data = np.arange(100000000, dtype='int32').reshape(10000, 10000)
    >>> z = zarr.create_array(store='data/example-9.zarr', shape=data.shape, dtype=data.dtype, chunks=(1000, 1000), filters=filters, compressors=compressors)
-   >>> z.info
+   >>> z.info_complete()
    Type               : Array
    Zarr format        : 3
    Data type          : Int32(endianness='little')
@@ -300,6 +300,9 @@ Here is an example using a delta filter with the Blosc compressor::
    Serializer         : BytesCodec(endian=<Endian.little: 'little'>)
    Compressors        : (BloscCodec(typesize=4, cname=<BloscCname.zstd: 'zstd'>, clevel=1, shuffle=<BloscShuffle.shuffle: 'shuffle'>, blocksize=0),)
    No. bytes          : 400000000 (381.5M)
+   No. bytes stored   : 826
+   Storage ratio      : 484261.5
+   Chunks Initialized : 0
 
 For more information about available filter codecs, see the `Numcodecs
 <https://numcodecs.readthedocs.io/>`_ documentation.
@@ -616,7 +619,7 @@ Sharded arrays can be created by providing the ``shards`` parameter to :func:`za
   Serializer         : BytesCodec(endian=None)
   Compressors        : (ZstdCodec(level=0, checksum=False),)
   No. bytes          : 100000000 (95.4M)
-  No. bytes stored   : 3981473
+  No. bytes stored   : 3981473 (3.8M)
   Storage ratio      : 25.1
   Shards Initialized : 100
 

docs/user-guide/arrays.rst

@@ -139,7 +139,7 @@ property. E.g.::
    Serializer         : BytesCodec(endian=<Endian.little: 'little'>)
    Compressors        : (ZstdCodec(level=0, checksum=False),)
    No. bytes          : 8000000 (7.6M)
-   No. bytes stored   : 1614
+   No. bytes stored   : 1614 (1.6K)
    Storage ratio      : 4956.6
    Chunks Initialized : 10
    >>> baz.info