Skip to content

Commit 45756b0

Browse files
K-Meechd-v-b
andauthored
Document StoreLike values in user guide (#3480)
* document valid values for StoreLike in user guide * fix doctests for storelike examples * make links mkdocs style * fix user guide links for mkdocs * link all StoreLike refs to the user guide * add changes note * remove old storage rst file --------- Co-authored-by: Davis Bennett <[email protected]>
1 parent 2d3d790 commit 45756b0

File tree

8 files changed

+159
-57
lines changed

8 files changed

+159
-57
lines changed

changes/3303.doc.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Document different values of StoreLike with examples in the user guide.

docs/user-guide/storage.md

Lines changed: 49 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,8 @@ Zarr-Python 3, stores must implement the abstract store API from
1212
## Implicit Store Creation
1313

1414
In most cases, it is not required to create a `Store` object explicitly. Passing a string
15-
to Zarr's top level API will result in the store being created automatically:
15+
(or other [StoreLike value](#storelike)) to Zarr's top level API will result in the store
16+
being created automatically:
1617

1718
```python exec="true" session="storage" source="above" result="ansi"
1819
import zarr
@@ -39,6 +40,53 @@ group = zarr.create_group(store=data)
3940
print(group)
4041
```
4142

43+
[](){#user-guide-store-like}
44+
### StoreLike
45+
46+
`StoreLike` values can be:
47+
48+
- a `Path` or string indicating a location on the local file system.
49+
This will create a [local store](#local-store):
50+
```python exec="true" session="storage" source="above" result="ansi"
51+
group = zarr.open_group(store='data/foo/bar')
52+
print(group)
53+
```
54+
```python exec="true" session="storage" source="above" result="ansi"
55+
from pathlib import Path
56+
group = zarr.open_group(store=Path('data/foo/bar'))
57+
print(group)
58+
```
59+
60+
- an FSSpec URI string, indicating a [remote store](#remote-store) location:
61+
```python exec="true" session="storage" source="above" result="ansi"
62+
group = zarr.open_group(
63+
store='s3://noaa-nwm-retro-v2-zarr-pds',
64+
mode='r',
65+
storage_options={'anon': True}
66+
)
67+
print(group)
68+
```
69+
70+
- an empty dictionary or None, which will create a new [memory store](#memory-store):
71+
```python exec="true" session="storage" source="above" result="ansi"
72+
group = zarr.create_group(store={})
73+
print(group)
74+
```
75+
```python exec="true" session="storage" source="above" result="ansi"
76+
group = zarr.create_group(store=None)
77+
print(group)
78+
```
79+
80+
- a dictionary of string to [`Buffer`][zarr.abc.buffer.Buffer] mappings. This will
81+
create a [memory store](#memory-store), using this dictionary as the
82+
[`store_dict` argument][zarr.storage.MemoryStore].
83+
84+
- an FSSpec [FSMap object](https://filesystem-spec.readthedocs.io/en/latest/api.html#fsspec.FSMap),
85+
which will create an [FsspecStore](#remote-store).
86+
87+
- a [`Store`][zarr.abc.store.Store] or [`StorePath`][zarr.storage.StorePath] -
88+
see explicit store creation below.
89+
4290
## Explicit Store Creation
4391

4492
In some cases, it may be helpful to create a store instance directly. Zarr-Python offers four

mkdocs.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -107,6 +107,7 @@ extra_css:
107107
- overrides/stylesheets/extra.css
108108

109109
plugins:
110+
- autorefs
110111
- search
111112
- markdown-exec
112113
- mkdocstrings:

src/zarr/api/asynchronous.py

Lines changed: 36 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -199,7 +199,9 @@ async def consolidate_metadata(
199199
Parameters
200200
----------
201201
store : StoreLike
202-
The store-like object whose metadata you wish to consolidate.
202+
The store-like object whose metadata you wish to consolidate. See the
203+
[storage documentation in the user guide][user-guide-store-like]
204+
for a description of all valid StoreLike values.
203205
path : str, optional
204206
A path to a group in the store to consolidate at. Only children
205207
below that group will be consolidated.
@@ -293,7 +295,9 @@ async def load(
293295
Parameters
294296
----------
295297
store : StoreLike
296-
Store or path to directory in file system or name of zip file.
298+
StoreLike object to open. See the
299+
[storage documentation in the user guide][user-guide-store-like]
300+
for a description of all valid StoreLike values.
297301
path : str or None, optional
298302
The path within the store from which to load.
299303
@@ -337,7 +341,9 @@ async def open(
337341
Parameters
338342
----------
339343
store : StoreLike or None, default=None
340-
Store or path to directory in file system or name of zip file.
344+
StoreLike object to open. See the
345+
[storage documentation in the user guide][user-guide-store-like]
346+
for a description of all valid StoreLike values.
341347
mode : {'r', 'r+', 'a', 'w', 'w-'}, optional
342348
Persistence mode: 'r' means read only (must exist); 'r+' means
343349
read/write (must exist); 'a' means read/write (create if doesn't
@@ -421,7 +427,9 @@ async def save(
421427
Parameters
422428
----------
423429
store : StoreLike
424-
Store or path to directory in file system or name of zip file.
430+
StoreLike object to open. See the
431+
[storage documentation in the user guide][user-guide-store-like]
432+
for a description of all valid StoreLike values.
425433
*args : ndarray
426434
NumPy arrays with data to save.
427435
zarr_format : {2, 3, None}, optional
@@ -457,7 +465,9 @@ async def save_array(
457465
Parameters
458466
----------
459467
store : StoreLike
460-
Store or path to directory in file system or name of zip file.
468+
StoreLike object to open. See the
469+
[storage documentation in the user guide][user-guide-store-like]
470+
for a description of all valid StoreLike values.
461471
arr : ndarray
462472
NumPy array with data to save.
463473
zarr_format : {2, 3, None}, optional
@@ -513,7 +523,9 @@ async def save_group(
513523
Parameters
514524
----------
515525
store : StoreLike
516-
Store or path to directory in file system or name of zip file.
526+
StoreLike object to open. See the
527+
[storage documentation in the user guide][user-guide-store-like]
528+
for a description of all valid StoreLike values.
517529
*args : ndarray
518530
NumPy arrays with data to save.
519531
zarr_format : {2, 3, None}, optional
@@ -662,7 +674,9 @@ async def group(
662674
Parameters
663675
----------
664676
store : StoreLike or None, default=None
665-
Store or path to directory in file system or name of zip file.
677+
StoreLike object to open. See the
678+
[storage documentation in the user guide][user-guide-store-like]
679+
for a description of all valid StoreLike values.
666680
overwrite : bool, optional
667681
If True, delete any pre-existing data in `store` at `path` before
668682
creating the group.
@@ -724,7 +738,9 @@ async def create_group(
724738
Parameters
725739
----------
726740
store : StoreLike
727-
Store or path to directory in file system or name of zip file.
741+
StoreLike object to open. See the
742+
[storage documentation in the user guide][user-guide-store-like]
743+
for a description of all valid StoreLike values.
728744
path : str, optional
729745
Group path within store.
730746
overwrite : bool, optional
@@ -780,17 +796,9 @@ async def open_group(
780796
Parameters
781797
----------
782798
store : StoreLike or None, default=None
783-
Store or path to directory in file system or name of zip file.
784-
785-
Strings are interpreted as paths on the local file system
786-
and used as the ``root`` argument to [zarr.storage.LocalStore][].
787-
788-
Dictionaries are used as the ``store_dict`` argument in
789-
[zarr.storage.MemoryStore][].
790-
791-
By default (``store=None``) a new [zarr.storage.MemoryStore][]
792-
is created.
793-
799+
StoreLike object to open. See the
800+
[storage documentation in the user guide][user-guide-store-like]
801+
for a description of all valid StoreLike values.
794802
mode : {'r', 'r+', 'a', 'w', 'w-'}, optional
795803
Persistence mode: 'r' means read only (must exist); 'r+' means
796804
read/write (must exist); 'a' means read/write (create if doesn't
@@ -805,7 +813,9 @@ async def open_group(
805813
path : str, optional
806814
Group path within store.
807815
chunk_store : StoreLike or None, default=None
808-
Store or path to directory in file system or name of zip file.
816+
Separate storage for chunks. See the
817+
[storage documentation in the user guide][user-guide-store-like]
818+
for a description of all valid StoreLike values.
809819
storage_options : dict
810820
If using an fsspec URL to create the store, these will be passed to
811821
the backend implementation. Ignored otherwise.
@@ -939,7 +949,9 @@ async def create(
939949
Memory layout to be used within each chunk.
940950
If not specified, the ``array.order`` parameter in the global config will be used.
941951
store : StoreLike or None, default=None
942-
Store or path to directory in file system or name of zip file.
952+
StoreLike object to open. See the
953+
[storage documentation in the user guide][user-guide-store-like]
954+
for a description of all valid StoreLike values.
943955
synchronizer : object, optional
944956
Array synchronizer.
945957
overwrite : bool, optional
@@ -1250,7 +1262,9 @@ async def open_array(
12501262
Parameters
12511263
----------
12521264
store : StoreLike
1253-
Store or path to directory in file system or name of zip file.
1265+
StoreLike object to open. See the
1266+
[storage documentation in the user guide][user-guide-store-like]
1267+
for a description of all valid StoreLike values.
12541268
zarr_version : {2, 3, None}, optional
12551269
The zarr format to use when saving. Deprecated in favor of zarr_format.
12561270
zarr_format : {2, 3, None}, optional

src/zarr/api/synchronous.py

Lines changed: 42 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -88,7 +88,9 @@ def consolidate_metadata(
8888
Parameters
8989
----------
9090
store : StoreLike
91-
The store-like object whose metadata you wish to consolidate.
91+
The store-like object whose metadata you wish to consolidate. See the
92+
[storage documentation in the user guide][user-guide-store-like]
93+
for a description of all valid StoreLike values.
9294
path : str, optional
9395
A path to a group in the store to consolidate at. Only children
9496
below that group will be consolidated.
@@ -143,7 +145,9 @@ def load(
143145
Parameters
144146
----------
145147
store : StoreLike
146-
Store or path to directory in file system or name of zip file.
148+
StoreLike object to open. See the
149+
[storage documentation in the user guide][user-guide-store-like]
150+
for a description of all valid StoreLike values.
147151
path : str or None, optional
148152
The path within the store from which to load.
149153
@@ -183,7 +187,9 @@ def open(
183187
Parameters
184188
----------
185189
store : StoreLike or None, default=None
186-
Store or path to directory in file system or name of zip file.
190+
StoreLike object to open. See the
191+
[storage documentation in the user guide][user-guide-store-like]
192+
for a description of all valid StoreLike values.
187193
mode : {'r', 'r+', 'a', 'w', 'w-'}, optional
188194
Persistence mode: 'r' means read only (must exist); 'r+' means
189195
read/write (must exist); 'a' means read/write (create if doesn't
@@ -245,7 +251,9 @@ def save(
245251
Parameters
246252
----------
247253
store : StoreLike
248-
Store or path to directory in file system or name of zip file.
254+
StoreLike object to open. See the
255+
[storage documentation in the user guide][user-guide-store-like]
256+
for a description of all valid StoreLike values.
249257
*args : ndarray
250258
NumPy arrays with data to save.
251259
zarr_format : {2, 3, None}, optional
@@ -279,7 +287,9 @@ def save_array(
279287
Parameters
280288
----------
281289
store : StoreLike
282-
Store or path to directory in file system or name of zip file.
290+
StoreLike object to open. See the
291+
[storage documentation in the user guide][user-guide-store-like]
292+
for a description of all valid StoreLike values.
283293
arr : ndarray
284294
NumPy array with data to save.
285295
zarr_format : {2, 3, None}, optional
@@ -322,7 +332,9 @@ def save_group(
322332
Parameters
323333
----------
324334
store : StoreLike
325-
Store or path to directory in file system or name of zip file.
335+
StoreLike object to open. See the
336+
[storage documentation in the user guide][user-guide-store-like]
337+
for a description of all valid StoreLike values.
326338
*args : ndarray
327339
NumPy arrays with data to save.
328340
zarr_format : {2, 3, None}, optional
@@ -413,7 +425,9 @@ def group(
413425
Parameters
414426
----------
415427
store : StoreLike or None, default=None
416-
Store or path to directory in file system or name of zip file.
428+
StoreLike object to open. See the
429+
[storage documentation in the user guide][user-guide-store-like]
430+
for a description of all valid StoreLike values.
417431
overwrite : bool, optional
418432
If True, delete any pre-existing data in `store` at `path` before
419433
creating the group.
@@ -480,17 +494,9 @@ def open_group(
480494
Parameters
481495
----------
482496
store : StoreLike or None, default=None
483-
Store or path to directory in file system or name of zip file.
484-
485-
Strings are interpreted as paths on the local file system
486-
and used as the ``root`` argument to [zarr.storage.LocalStore][].
487-
488-
Dictionaries are used as the ``store_dict`` argument in
489-
[zarr.storage.MemoryStore][].
490-
491-
By default (``store=None``) a new [zarr.storage.MemoryStore][]
492-
is created.
493-
497+
StoreLike object to open. See the
498+
[storage documentation in the user guide][user-guide-store-like]
499+
for a description of all valid StoreLike values.
494500
mode : {'r', 'r+', 'a', 'w', 'w-'}, optional
495501
Persistence mode: 'r' means read only (must exist); 'r+' means
496502
read/write (must exist); 'a' means read/write (create if doesn't
@@ -505,7 +511,9 @@ def open_group(
505511
path : str, optional
506512
Group path within store.
507513
chunk_store : StoreLike or None, default=None
508-
Store or path to directory in file system or name of zip file.
514+
Separate storage for chunks. See the
515+
[storage documentation in the user guide][user-guide-store-like]
516+
for a description of all valid StoreLike values.
509517
storage_options : dict
510518
If using an fsspec URL to create the store, these will be passed to
511519
the backend implementation. Ignored otherwise.
@@ -570,7 +578,9 @@ def create_group(
570578
Parameters
571579
----------
572580
store : StoreLike
573-
Store or path to directory in file system or name of zip file.
581+
StoreLike object to open. See the
582+
[storage documentation in the user guide][user-guide-store-like]
583+
for a description of all valid StoreLike values.
574584
path : str, optional
575585
Group path within store.
576586
overwrite : bool, optional
@@ -672,7 +682,9 @@ def create(
672682
Memory layout to be used within each chunk.
673683
If not specified, the ``array.order`` parameter in the global config will be used.
674684
store : StoreLike or None, default=None
675-
Store or path to directory in file system or name of zip file.
685+
StoreLike object to open. See the
686+
[storage documentation in the user guide][user-guide-store-like]
687+
for a description of all valid StoreLike values.
676688
synchronizer : object, optional
677689
Array synchronizer.
678690
overwrite : bool, optional
@@ -832,7 +844,9 @@ def create_array(
832844
Parameters
833845
----------
834846
store : StoreLike
835-
Store or path to directory in file system or name of zip file.
847+
StoreLike object to open. See the
848+
[storage documentation in the user guide][user-guide-store-like]
849+
for a description of all valid StoreLike values.
836850
name : str or None, optional
837851
The name of the array within the store. If ``name`` is ``None``, the array will be located
838852
at the root of the store.
@@ -996,7 +1010,9 @@ def from_array(
9961010
Parameters
9971011
----------
9981012
store : StoreLike
999-
Store or path to directory in file system or name of zip file.
1013+
StoreLike object to open. See the
1014+
[storage documentation in the user guide][user-guide-store-like]
1015+
for a description of all valid StoreLike values.
10001016
data : Array | array-like
10011017
The array to copy.
10021018
write_data : bool, default True
@@ -1333,7 +1349,9 @@ def open_array(
13331349
Parameters
13341350
----------
13351351
store : StoreLike
1336-
Store or path to directory in file system or name of zip file.
1352+
StoreLike object to open. See the
1353+
[storage documentation in the user guide][user-guide-store-like]
1354+
for a description of all valid StoreLike values.
13371355
zarr_version : {2, 3, None}, optional
13381356
The zarr format to use when saving. Deprecated in favor of zarr_format.
13391357
zarr_format : {2, 3, None}, optional

0 commit comments

Comments
 (0)