Skip to content

Commit 865815b

Browse files
will-moorewill-moore
committed
Merge remote-tracking branch 'origin/main' into fix_json_indent
2 parents e3dba80 + 9138528 commit 865815b

File tree

102 files changed

+6633
-3413
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

102 files changed

+6633
-3413
lines changed

‎.github/workflows/gpu_test.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,7 @@ jobs:
5555
cache: 'pip'
5656
- name: Install Hatch and CuPy
5757
run: |
58-
python -m pip install --upgrade pip
58+
python -m pip install --upgrade pip
5959
pip install hatch
6060
- name: Set Up Hatch Env
6161
run: |

‎.github/workflows/releases.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ jobs:
2323

2424
- name: Install PyBuild
2525
run: |
26-
python -m pip install --upgrade pip
26+
python -m pip install --upgrade pip
2727
pip install hatch
2828
- name: Build wheel and sdist
2929
run: hatch build

‎.github/workflows/test.yml‎

Lines changed: 28 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ jobs:
5252
cache: 'pip'
5353
- name: Install Hatch
5454
run: |
55-
python -m pip install --upgrade pip
55+
python -m pip install --upgrade pip
5656
pip install hatch
5757
- name: Set Up Hatch Env
5858
run: |
@@ -84,7 +84,7 @@ jobs:
8484
cache: 'pip'
8585
- name: Install Hatch
8686
run: |
87-
python -m pip install --upgrade pip
87+
python -m pip install --upgrade pip
8888
pip install hatch
8989
- name: Set Up Hatch Env
9090
run: |
@@ -94,13 +94,38 @@ jobs:
9494
run: |
9595
hatch env run --env ${{ matrix.dependency-set }} run
9696
97+
doctests:
98+
name: doctests
99+
runs-on: ubuntu-latest
100+
steps:
101+
- uses: actions/checkout@v4
102+
with:
103+
fetch-depth: 0 # required for hatch version discovery, which is needed for numcodecs.zarr3
104+
- name: Set up Python
105+
uses: actions/setup-python@v5
106+
with:
107+
python-version: '3.13'
108+
cache: 'pip'
109+
- name: Install Hatch
110+
run: |
111+
python -m pip install --upgrade pip
112+
pip install hatch
113+
- name: Set Up Hatch Env
114+
run: |
115+
hatch env create doctest
116+
hatch env run -e doctest list-env
117+
- name: Run Tests
118+
run: |
119+
hatch env run --env doctest run
120+
97121
test-complete:
98122
name: Test complete
99123

100124
needs:
101125
[
102126
test,
103127
test-upstream-and-min-deps,
128+
doctests
104129
]
105130
if: always()
106131
runs-on: ubuntu-latest
@@ -111,4 +136,4 @@ jobs:
111136
contains(needs.*.result, 'cancelled')
112137
run: exit 1
113138
- name: Success
114-
run: echo Success!
139+
run: echo Success!

‎.gitignore‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,10 @@ coverage.xml
5151

5252
# Sphinx documentation
5353
docs/_build/
54-
docs/_autoapi
54+
docs/api
55+
docs/data
56+
data
57+
data.zip
5558

5659
# PyBuilder
5760
target/

‎.pre-commit-config.yaml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,16 +20,18 @@ repos:
2020
rev: v5.0.0
2121
hooks:
2222
- id: check-yaml
23+
- id: trailing-whitespace
2324
- repo: https://github.com/pre-commit/mirrors-mypy
2425
rev: v1.13.0
2526
hooks:
2627
- id: mypy
2728
files: src|tests
2829
additional_dependencies:
2930
# Package dependencies
31+
- packaging
3032
- donfig
3133
- numcodecs[crc32c]
32-
- numpy
34+
- numpy==2.1 # until https://github.com/numpy/numpy/issues/28034 is resolved
3335
- typing_extensions
3436
- universal-pathlib
3537
# Tests

‎README-v3.md‎

Lines changed: 0 additions & 49 deletions
This file was deleted.

‎bench/compress_normal.txt‎

Lines changed: 20 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ Line # Hits Time Per Hit % Time Line Contents
1919
==============================================================
2020
137 def compress(source, char* cname, int clevel, int shuffle):
2121
138 """Compress data in a numpy array.
22-
139
22+
139
2323
140 Parameters
2424
141 ----------
2525
142 source : array-like
@@ -30,33 +30,33 @@ Line # Hits Time Per Hit % Time Line Contents
3030
147 Compression level.
3131
148 shuffle : int
3232
149 Shuffle filter.
33-
150
33+
150
3434
151 Returns
3535
152 -------
3636
153 dest : bytes-like
3737
154 Compressed data.
38-
155
38+
155
3939
156 """
40-
157
40+
157
4141
158 cdef:
4242
159 char *source_ptr
4343
160 char *dest_ptr
4444
161 Py_buffer source_buffer
4545
162 size_t nbytes, cbytes, itemsize
4646
163 200 506 2.5 0.2 array.array char_array_template = array.array('b', [])
4747
164 array.array dest
48-
165
48+
165
4949
166 # setup source buffer
5050
167 200 458 2.3 0.2 PyObject_GetBuffer(source, &source_buffer, PyBUF_ANY_CONTIGUOUS)
5151
168 200 119 0.6 0.0 source_ptr = <char *> source_buffer.buf
52-
169
52+
169
5353
170 # setup destination
5454
171 200 239 1.2 0.1 nbytes = source_buffer.len
5555
172 200 103 0.5 0.0 itemsize = source_buffer.itemsize
5656
173 200 2286 11.4 0.8 dest = array.clone(char_array_template, nbytes + BLOSC_MAX_OVERHEAD,
5757
174 zero=False)
5858
175 200 129 0.6 0.0 dest_ptr = <char *> dest.data.as_voidptr
59-
176
59+
176
6060
177 # perform compression
6161
178 200 1734 8.7 0.6 if _get_use_threads():
6262
179 # allow blosc to use threads internally
@@ -67,24 +67,24 @@ Line # Hits Time Per Hit % Time Line Contents
6767
184 cbytes = blosc_compress(clevel, shuffle, itemsize, nbytes,
6868
185 source_ptr, dest_ptr,
6969
186 nbytes + BLOSC_MAX_OVERHEAD)
70-
187
70+
187
7171
188 else:
7272
189 with nogil:
7373
190 cbytes = blosc_compress_ctx(clevel, shuffle, itemsize, nbytes,
7474
191 source_ptr, dest_ptr,
7575
192 nbytes + BLOSC_MAX_OVERHEAD, cname,
7676
193 0, 1)
77-
194
77+
194
7878
195 # release source buffer
7979
196 200 616 3.1 0.2 PyBuffer_Release(&source_buffer)
80-
197
80+
197
8181
198 # check compression was successful
8282
199 200 120 0.6 0.0 if cbytes <= 0:
8383
200 raise RuntimeError('error during blosc compression: %d' % cbytes)
84-
201
84+
201
8585
202 # resize after compression
8686
203 200 1896 9.5 0.6 array.resize(dest, cbytes)
87-
204
87+
204
8888
205 200 186 0.9 0.1 return dest
8989

9090
*******************************************************************************
@@ -100,19 +100,19 @@ Line # Hits Time Per Hit % Time Line Contents
100100
==============================================================
101101
75 def decompress(source, dest):
102102
76 """Decompress data.
103-
77
103+
77
104104
78 Parameters
105105
79 ----------
106106
80 source : bytes-like
107107
81 Compressed data, including blosc header.
108108
82 dest : array-like
109109
83 Object to decompress into.
110-
84
110+
84
111111
85 Notes
112112
86 -----
113113
87 Assumes that the size of the destination buffer is correct for the size of
114114
88 the uncompressed data.
115-
89
115+
89
116116
90 """
117117
91 cdef:
118118
92 int ret
@@ -122,7 +122,7 @@ Line # Hits Time Per Hit % Time Line Contents
122122
96 array.array source_array
123123
97 Py_buffer dest_buffer
124124
98 size_t nbytes
125-
99
125+
99
126126
100 # setup source buffer
127127
101 200 573 2.9 0.2 if PY2 and isinstance(source, array.array):
128128
102 # workaround fact that array.array does not support new-style buffer
@@ -134,13 +134,13 @@ Line # Hits Time Per Hit % Time Line Contents
134134
108 200 112 0.6 0.0 release_source_buffer = True
135135
109 200 144 0.7 0.1 PyObject_GetBuffer(source, &source_buffer, PyBUF_ANY_CONTIGUOUS)
136136
110 200 98 0.5 0.0 source_ptr = <char *> source_buffer.buf
137-
111
137+
111
138138
112 # setup destination buffer
139139
113 200 552 2.8 0.2 PyObject_GetBuffer(dest, &dest_buffer,
140140
114 PyBUF_ANY_CONTIGUOUS | PyBUF_WRITEABLE)
141141
115 200 100 0.5 0.0 dest_ptr = <char *> dest_buffer.buf
142142
116 200 84 0.4 0.0 nbytes = dest_buffer.len
143-
117
143+
117
144144
118 # perform decompression
145145
119 200 1856 9.3 0.8 if _get_use_threads():
146146
120 # allow blosc to use threads internally
@@ -149,12 +149,12 @@ Line # Hits Time Per Hit % Time Line Contents
149149
123 else:
150150
124 with nogil:
151151
125 ret = blosc_decompress_ctx(source_ptr, dest_ptr, nbytes, 1)
152-
126
152+
126
153153
127 # release buffers
154154
128 200 754 3.8 0.3 if release_source_buffer:
155155
129 200 326 1.6 0.1 PyBuffer_Release(&source_buffer)
156156
130 200 165 0.8 0.1 PyBuffer_Release(&dest_buffer)
157-
131
157+
131
158158
132 # handle errors
159159
133 200 128 0.6 0.1 if ret <= 0:
160160
134 raise RuntimeError('error during blosc decompression: %d' % ret)

‎data/donotdelete‎

Lines changed: 0 additions & 1 deletion
This file was deleted.

‎docs/Makefile‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ help:
5252
.PHONY: clean
5353
clean:
5454
rm -rf $(BUILDDIR)/*
55-
rm -rf $(BUILDDIR)/../_autoapi
55+
rm -rf $(BUILDDIR)/../api
5656

5757
.PHONY: html
5858
html:

‎docs/about.rst‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
About
2+
=====
3+
4+
Zarr is a format for the storage of chunked, compressed, N-dimensional arrays
5+
inspired by `HDF5 <https://www.hdfgroup.org/HDF5/>`_, `h5py
6+
<https://www.h5py.org/>`_ and `bcolz <https://bcolz.readthedocs.io/>`_.
7+
8+
These documents describe the Zarr-Python implementation. More information
9+
about the Zarr format can be found on the `main website <https://zarr.dev>`_.
10+
11+
Projects using Zarr
12+
-------------------
13+
14+
If you are using Zarr-Python, we would `love to hear about it
15+
<https://github.com/zarr-developers/community/issues/19>`_.
16+
17+
Funding
18+
-------
19+
The project is fiscally sponsored by `NumFOCUS <https://numfocus.org/>`_, a US
20+
501(c)(3) public charity, and development is supported by the
21+
`MRC Centre for Genomics and Global Health <https://www.cggh.org>`_
22+
and the `Chan Zuckerberg Initiative <https://chanzuckerberg.com/>`_.
23+
24+
.. _NumCodecs: https://numcodecs.readthedocs.io/

0 commit comments

Comments
 (0)