Merge remote-tracking branch 'upstream/main' into gpu-codecs

Author: TomAugspurger

.github/CONTRIBUTING.md

@@ -1,4 +1,4 @@
 Contributing
 ============
 
-Please see the [project documentation](https://zarr.readthedocs.io/en/stable/contributing.html) for information about contributing to Zarr.
+Please see the [project documentation](https://zarr.readthedocs.io/en/stable/developers/contributing.html) for information about contributing to Zarr.

.github/ISSUE_TEMPLATE/bug_report.yml

@@ -57,7 +57,22 @@ body:
     id: reproduce
     attributes:
       label: Steps to reproduce
-      description: Minimal, reproducible code sample, a copy-pastable example if possible.
+      description: Minimal, reproducible code sample. Must list dependencies in [inline script metadata](https://packaging.python.org/en/latest/specifications/inline-script-metadata/#example). When put in a file named `issue.py` calling `uv run issue.py` should show the issue.
+      value: |
+        ```python
+        # /// script
+        # requires-python = ">=3.11"
+        # dependencies = [
+        #   "zarr@git+https://github.com/zarr-developers/zarr-python.git@main",
+        # ]
+        # ///
+        #
+        # This script automatically imports the development branch of zarr to check for issues
+
+        import zarr
+        # your reproducer code
+        # zarr.print_debug_info()
+        ```
     validations:
       required: true
   - type: textarea

.github/ISSUE_TEMPLATE/config.yml

@@ -5,7 +5,7 @@ contact_links:
     about: A new major feature should be discussed in the Zarr specifications repository.
   - name: Discuss something on ZulipChat
     url: https://ossci.zulipchat.com/
-    about: For questions like "How do I do X with Zarr?", you can move to our ZulipChat.
+    about: For questions like "How do I do X with Zarr?", consider posting your question to our developer chat.
   - name: Discuss something on GitHub Discussions
     url: https://github.com/zarr-developers/zarr-python/discussions
     about: For questions like "How do I do X with Zarr?", you can move to GitHub Discussions.

.github/ISSUE_TEMPLATE/release-checklist.md

@@ -0,0 +1,56 @@
+---
+name: Zarr-Python release checklist
+about: Checklist for a new Zarr-Python release. [For project maintainers only!]
+title: Release Zarr-Python vX.Y.Z
+labels: release-checklist
+assignees: ''
+
+---
+
+**Release**: [v0.x.x](https://github.com/zarr-developers/zarr-python/milestones/?)
+**Scheduled Date**: 20YY/MM/DD
+
+**Priority PRs/issues to complete prior to release**
+
+- [ ] Priority pull request #X
+
+**Before release**:
+
+- [ ] Make sure that all pull requests which will be included in the release have been properly documented as changelog files in the [`changes/` directory](https://github.com/zarr-developers/zarr-python/tree/main/changes).
+- [ ] Run ``towncrier build --version x.y.z`` to create the changelog, and commit the result to the main branch.
+- [ ] Check [SPEC 0](https://scientific-python.org/specs/spec-0000/#support-window) to see if the minimum supported version of Python or NumPy needs bumping.
+- [ ] Check to ensure that:
+  - [ ] Deprecated workarounds/codes/tests are removed. Run `grep "# TODO" **/*.py` to find all potential TODOs.
+  - [ ] All tests pass in the ["Tests" workflow](https://github.com/zarr-developers/zarr-python/actions/workflows/test.yml).
+  - [ ] All tests pass in the ["GPU Tests" workflow](https://github.com/zarr-developers/zarr-python/actions/workflows/gpu_test.yml).
+  - [ ] All tests pass in the ["Hypothesis" workflow](https://github.com/zarr-developers/zarr-python/actions/workflows/hypothesis.yaml).
+  - [ ] Check that downstream libraries work well (maintainers can make executive decisions about whether all checks are required for this release).
+    - [ ] Xarray (@jhamman @dcherian @TomNicholas)
+        - Zarr's upstream compatibility is tested via the [Upstream Dev CI worklow](https://github.com/pydata/xarray/actions/workflows/upstream-dev-ci.yaml).
+        - Click on the most recent workflow and check that the `upstream-dev` job has run and passed. `upstream-dev` is not run on all all workflow runs.
+        - Check that the expected version of Zarr-Python was tested using the `Version Info` step of the `upstream-dev` job.
+        - If testing on a branch other than `main` is needed, open a PR modifying https://github.com/pydata/xarray/blob/90ee30943aedba66a37856b2332a41264e288c20/ci/install-upstream-wheels.sh#L56 and add the `run-upstream` label.
+    - [ ] Titiler.Xarray (@maxrjones)
+        - [Modify dependencies](https://github.com/developmentseed/titiler/blob/main/src/titiler/xarray/pyproject.toml) for titiler.xarray.
+        - Modify triggers for running [the test workflow](https://github.com/developmentseed/titiler/blob/61549f2de07b20cca8fb991cfcdc89b23e18ad05/.github/workflows/ci.yml#L5-L7).
+        - Push the branch to the repository and check for the actions for any failures.
+
+**Release**:
+
+- [ ] Go to https://github.com/zarr-developers/zarr-python/releases.
+  - [ ] Click "Draft a new release".
+  - [ ] Choose a version number prefixed with a `v` (e.g. `v0.0.0`). For pre-releases, include the appropriate suffix (e.g. `v0.0.0a1` or `v0.0.0rc2`).
+  - [ ] Set the description of the release to: `See release notes https://zarr.readthedocs.io/en/stable/release-notes.html#release-0-0-0`, replacing the correct version numbers. For pre-release versions, the URL should omit the pre-release suffix, e.g. "a1" or "rc1".
+  - [ ] Click on "Generate release notes" to auto-fill the description.
+  - [ ] Make a release by clicking the 'Publish Release' button, this will automatically create a tag too.
+- [ ] Verify that release workflows succeeded.
+  - [ ] The latest version is correct on [PyPI](https://pypi.org/project/zarr/).
+  - [ ] The stable version is correct on [ReadTheDocs](https://zarr.readthedocs.io/en/stable/).
+
+**After release**:
+
+- [ ] Review and merge the pull request on the conda-forge [zarr-feedstock](https://github.com/conda-forge/zarr-feedstock) that will be automatically generated.
+
+---
+
+- [ ] Party :tada:

.github/codecov.yml

@@ -25,7 +25,7 @@ jobs:
     strategy:
       matrix:
         python-version: ['3.11']
-        numpy-version: ['2.1']
+        numpy-version: ['2.2']
         dependency-set: ["minimal"]
 
     steps:

.github/workflows/gpu_test.yml

@@ -1,11 +1,9 @@
 name: Slow Hypothesis CI
 on:
   push:
-    branches:
-      - "main"
+    branches: [main, 3.0.x]
   pull_request:
-    branches:
-      - "main"
+    branches: [main, 3.0.x]
     types: [opened, reopened, synchronize, labeled]
   schedule:
     - cron: "0 0 * * *" # Daily “At 00:00” UTC
@@ -25,12 +23,19 @@ jobs:
 
     strategy:
       matrix:
-        python-version: ['3.11']
-        numpy-version: ['2.1']
+        python-version: ['3.12']
+        numpy-version: ['2.2']
         dependency-set: ["optional"]
 
     steps:
     - uses: actions/checkout@v4
+    - name: Set HYPOTHESIS_PROFILE based on trigger
+      run: |
+        if [[ "${{ github.event_name }}" == "schedule" || "${{ github.event_name }}" == "workflow_dispatch" ]]; then
+          echo "HYPOTHESIS_PROFILE=nightly" >> $GITHUB_ENV
+        else
+          echo "HYPOTHESIS_PROFILE=ci" >> $GITHUB_ENV
+        fi
     - name: Set up Python
       uses: actions/setup-python@v5
       with:
@@ -58,6 +63,7 @@ jobs:
       if: success()
       id: status
       run: |
+        echo "Using Hypothesis profile: $HYPOTHESIS_PROFILE"
         hatch env run --env test.py${{ matrix.python-version }}-${{ matrix.numpy-version }}-${{ matrix.dependency-set }} run-hypothesis
 
     # explicitly save the cache so it gets updated, also do this even if it fails.

.github/workflows/hypothesis.yaml

@@ -5,9 +5,9 @@ name: Test
 
 on:
   push:
-    branches: [ main ]
+    branches: [ main, 3.0.x ]
   pull_request:
-    branches: [ main ]
+    branches: [ main, 3.0.x ]
   workflow_dispatch:
 
 concurrency:
@@ -21,7 +21,7 @@ jobs:
     strategy:
       matrix:
         python-version: ['3.11', '3.12', '3.13']
-        numpy-version: ['1.25', '2.1']
+        numpy-version: ['1.25', '2.2']
         dependency-set: ["minimal", "optional"]
         os: ["ubuntu-latest"]
         include:
@@ -30,15 +30,15 @@ jobs:
             dependency-set: 'optional'
             os: 'macos-latest'
           - python-version: '3.13'
-            numpy-version: '2.1'
+            numpy-version: '2.2'
             dependency-set: 'optional'
             os: 'macos-latest'
           - python-version: '3.11'
             numpy-version: '1.25'
             dependency-set: 'optional'
             os: 'windows-latest'
           - python-version: '3.13'
-            numpy-version: '2.1'
+            numpy-version: '2.2'
             dependency-set: 'optional'
             os: 'windows-latest'
     runs-on: ${{ matrix.os }}
@@ -61,9 +61,12 @@ jobs:
         hatch env create test.py${{ matrix.python-version }}-${{ matrix.numpy-version }}-${{ matrix.dependency-set }}
         hatch env run -e test.py${{ matrix.python-version }}-${{ matrix.numpy-version }}-${{ matrix.dependency-set }} list-env
     - name: Run Tests
+      env:
+        HYPOTHESIS_PROFILE: ci
       run: |
         hatch env run --env test.py${{ matrix.python-version }}-${{ matrix.numpy-version }}-${{ matrix.dependency-set }} run-coverage
     - name: Upload coverage
+      if: ${{ matrix.dependency-set == 'optional' && matrix.os == 'ubuntu-latest' }}
       uses: codecov/codecov-action@v5
       with:
         token: ${{ secrets.CODECOV_TOKEN }}
@@ -101,7 +104,7 @@ jobs:
         hatch env run -e ${{ matrix.dependency-set }} list-env
     - name: Run Tests
       run: |
-        hatch env run --env ${{ matrix.dependency-set }} run
+        hatch env run --env ${{ matrix.dependency-set }} run-coverage
     - name: Upload coverage
       uses: codecov/codecov-action@v5
       with:

.github/workflows/test.yml

@@ -6,11 +6,11 @@ ci:
 default_stages: [pre-commit, pre-push]
 repos:
   - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.9.4
+    rev: v0.12.2
     hooks:
-    - id: ruff
-      args: ["--fix", "--show-fixes"]
-    - id: ruff-format
+      - id: ruff-check
+        args: ["--fix", "--show-fixes"]
+      - id: ruff-format
   - repo: https://github.com/codespell-project/codespell
     rev: v2.4.1
     hooks:
@@ -19,10 +19,10 @@ repos:
   - repo: https://github.com/pre-commit/pre-commit-hooks
     rev: v5.0.0
     hooks:
-    - id: check-yaml
-    - id: trailing-whitespace
+      - id: check-yaml
+      - id: trailing-whitespace
   - repo: https://github.com/pre-commit/mirrors-mypy
-    rev: v1.14.1
+    rev: v1.16.1
     hooks:
       - id: mypy
         files: src|tests
@@ -31,13 +31,16 @@ repos:
           - packaging
           - donfig
           - numcodecs[crc32c]
-          - numpy==2.1  # until https://github.com/numpy/numpy/issues/28034 is resolved
+          - numpy==2.1 # until https://github.com/numpy/numpy/issues/28034 is resolved
           - typing_extensions
           - universal-pathlib
+          - obstore>=0.5.1
           # Tests
           - pytest
+          - hypothesis
+          - s3fs
   - repo: https://github.com/scientific-python/cookie
-    rev: 2025.01.22
+    rev: 2025.05.02
     hooks:
       - id: sp-repo-review
   - repo: https://github.com/pre-commit/pygrep-hooks

.pre-commit-config.yaml

@@ -70,7 +70,7 @@
   </td>
 </tr>
 <tr>
-	<td>Zulip</td>
+	<td>Developer Chat</td>
 	<td>
 		<a href="https://ossci.zulipchat.com/">
 		<img src="https://img.shields.io/badge/zulip-join_chat-brightgreen.svg" />
@@ -101,13 +101,13 @@ Zarr is a Python package providing an implementation of compressed, chunked, N-d
 
 ## Main Features
 
-- [**Create**](https://zarr.readthedocs.io/en/stable/tutorial.html#creating-an-array) N-dimensional arrays with any NumPy `dtype`.
-- [**Chunk arrays**](https://zarr.readthedocs.io/en/stable/tutorial.html#chunk-optimizations) along any dimension.
-- [**Compress**](https://zarr.readthedocs.io/en/stable/tutorial.html#compressors) and/or filter chunks using any NumCodecs codec.
-- [**Store arrays**](https://zarr.readthedocs.io/en/stable/tutorial.html#tutorial-storage) in memory, on disk, inside a zip file, on S3, etc...
-- [**Read**](https://zarr.readthedocs.io/en/stable/tutorial.html#reading-and-writing-data) an array [**concurrently**](https://zarr.readthedocs.io/en/stable/tutorial.html#parallel-computing-and-synchronization) from multiple threads or processes.
-- Write to an array concurrently from multiple threads or processes.
-- Organize arrays into hierarchies via [**groups**](https://zarr.readthedocs.io/en/stable/tutorial.html#groups).
+- [**Create**](https://zarr.readthedocs.io/en/stable/user-guide/arrays.html#creating-an-array) N-dimensional arrays with any NumPy `dtype`.
+- [**Chunk arrays**](https://zarr.readthedocs.io/en/stable/user-guide/performance.html#chunk-optimizations) along any dimension.
+- [**Compress**](https://zarr.readthedocs.io/en/stable/user-guide/arrays.html#compressors) and/or filter chunks using any NumCodecs codec.
+- [**Store arrays**](https://zarr.readthedocs.io/en/stable/user-guide/storage.html) in memory, on disk, inside a zip file, on S3, etc...
+- [**Read**](https://zarr.readthedocs.io/en/stable/user-guide/arrays.html#reading-and-writing-data) an array [**concurrently**](https://zarr.readthedocs.io/en/stable/user-guide/performance.html#parallel-computing-and-synchronization) from multiple threads or processes.
+- [**Write**](https://zarr.readthedocs.io/en/stable/user-guide/arrays.html#reading-and-writing-data) to an array concurrently from multiple threads or processes.
+- Organize arrays into hierarchies via [**groups**](https://zarr.readthedocs.io/en/stable/quickstart.html#hierarchical-groups).
 
 ## Where to get it