Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changes/3303.doc.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Document different values of StoreLike with examples in the user guide.
50 changes: 49 additions & 1 deletion docs/user-guide/storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@ Zarr-Python 3, stores must implement the abstract store API from
## Implicit Store Creation

In most cases, it is not required to create a `Store` object explicitly. Passing a string
to Zarr's top level API will result in the store being created automatically:
(or other [StoreLike value](#storelike)) to Zarr's top level API will result in the store
being created automatically:

```python exec="true" session="storage" source="above" result="ansi"
import zarr
Expand All @@ -39,6 +40,53 @@ group = zarr.create_group(store=data)
print(group)
```

[](){#user-guide-store-like}
### StoreLike

`StoreLike` values can be:

- a `Path` or string indicating a location on the local file system.
This will create a [local store](#local-store):
```python exec="true" session="storage" source="above" result="ansi"
group = zarr.open_group(store='data/foo/bar')
print(group)
```
```python exec="true" session="storage" source="above" result="ansi"
from pathlib import Path
group = zarr.open_group(store=Path('data/foo/bar'))
print(group)
```

- an FSSpec URI string, indicating a [remote store](#remote-store) location:
```python exec="true" session="storage" source="above" result="ansi"
group = zarr.open_group(
store='s3://noaa-nwm-retro-v2-zarr-pds',
mode='r',
storage_options={'anon': True}
)
print(group)
```

- an empty dictionary or None, which will create a new [memory store](#memory-store):
```python exec="true" session="storage" source="above" result="ansi"
group = zarr.create_group(store={})
print(group)
```
```python exec="true" session="storage" source="above" result="ansi"
group = zarr.create_group(store=None)
print(group)
```

- a dictionary of string to [`Buffer`][zarr.abc.buffer.Buffer] mappings. This will
create a [memory store](#memory-store), using this dictionary as the
[`store_dict` argument][zarr.storage.MemoryStore].

- an FSSpec [FSMap object](https://filesystem-spec.readthedocs.io/en/latest/api.html#fsspec.FSMap),
which will create an [FsspecStore](#remote-store).

- a [`Store`][zarr.abc.store.Store] or [`StorePath`][zarr.storage.StorePath] -
see explicit store creation below.

## Explicit Store Creation

In some cases, it may be helpful to create a store instance directly. Zarr-Python offers four
Expand Down
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,7 @@ extra_css:
- overrides/stylesheets/extra.css

plugins:
- autorefs
- search
- markdown-exec
- mkdocstrings:
Expand Down
58 changes: 36 additions & 22 deletions src/zarr/api/asynchronous.py
Original file line number Diff line number Diff line change
Expand Up @@ -199,7 +199,9 @@ async def consolidate_metadata(
Parameters
----------
store : StoreLike
The store-like object whose metadata you wish to consolidate.
The store-like object whose metadata you wish to consolidate. See the
[storage documentation in the user guide][user-guide-store-like]
for a description of all valid StoreLike values.
path : str, optional
A path to a group in the store to consolidate at. Only children
below that group will be consolidated.
Expand Down Expand Up @@ -293,7 +295,9 @@ async def load(
Parameters
----------
store : StoreLike
Store or path to directory in file system or name of zip file.
StoreLike object to open. See the
[storage documentation in the user guide][user-guide-store-like]
for a description of all valid StoreLike values.
path : str or None, optional
The path within the store from which to load.

Expand Down Expand Up @@ -337,7 +341,9 @@ async def open(
Parameters
----------
store : StoreLike or None, default=None
Store or path to directory in file system or name of zip file.
StoreLike object to open. See the
[storage documentation in the user guide][user-guide-store-like]
for a description of all valid StoreLike values.
mode : {'r', 'r+', 'a', 'w', 'w-'}, optional
Persistence mode: 'r' means read only (must exist); 'r+' means
read/write (must exist); 'a' means read/write (create if doesn't
Expand Down Expand Up @@ -421,7 +427,9 @@ async def save(
Parameters
----------
store : StoreLike
Store or path to directory in file system or name of zip file.
StoreLike object to open. See the
[storage documentation in the user guide][user-guide-store-like]
for a description of all valid StoreLike values.
*args : ndarray
NumPy arrays with data to save.
zarr_format : {2, 3, None}, optional
Expand Down Expand Up @@ -457,7 +465,9 @@ async def save_array(
Parameters
----------
store : StoreLike
Store or path to directory in file system or name of zip file.
StoreLike object to open. See the
[storage documentation in the user guide][user-guide-store-like]
for a description of all valid StoreLike values.
arr : ndarray
NumPy array with data to save.
zarr_format : {2, 3, None}, optional
Expand Down Expand Up @@ -513,7 +523,9 @@ async def save_group(
Parameters
----------
store : StoreLike
Store or path to directory in file system or name of zip file.
StoreLike object to open. See the
[storage documentation in the user guide][user-guide-store-like]
for a description of all valid StoreLike values.
*args : ndarray
NumPy arrays with data to save.
zarr_format : {2, 3, None}, optional
Expand Down Expand Up @@ -662,7 +674,9 @@ async def group(
Parameters
----------
store : StoreLike or None, default=None
Store or path to directory in file system or name of zip file.
StoreLike object to open. See the
[storage documentation in the user guide][user-guide-store-like]
for a description of all valid StoreLike values.
overwrite : bool, optional
If True, delete any pre-existing data in `store` at `path` before
creating the group.
Expand Down Expand Up @@ -724,7 +738,9 @@ async def create_group(
Parameters
----------
store : StoreLike
Store or path to directory in file system or name of zip file.
StoreLike object to open. See the
[storage documentation in the user guide][user-guide-store-like]
for a description of all valid StoreLike values.
path : str, optional
Group path within store.
overwrite : bool, optional
Expand Down Expand Up @@ -780,17 +796,9 @@ async def open_group(
Parameters
----------
store : StoreLike or None, default=None
Store or path to directory in file system or name of zip file.

Strings are interpreted as paths on the local file system
and used as the ``root`` argument to [zarr.storage.LocalStore][].

Dictionaries are used as the ``store_dict`` argument in
[zarr.storage.MemoryStore][].

By default (``store=None``) a new [zarr.storage.MemoryStore][]
is created.

StoreLike object to open. See the
[storage documentation in the user guide][user-guide-store-like]
for a description of all valid StoreLike values.
mode : {'r', 'r+', 'a', 'w', 'w-'}, optional
Persistence mode: 'r' means read only (must exist); 'r+' means
read/write (must exist); 'a' means read/write (create if doesn't
Expand All @@ -805,7 +813,9 @@ async def open_group(
path : str, optional
Group path within store.
chunk_store : StoreLike or None, default=None
Store or path to directory in file system or name of zip file.
Separate storage for chunks. See the
[storage documentation in the user guide][user-guide-store-like]
for a description of all valid StoreLike values.
storage_options : dict
If using an fsspec URL to create the store, these will be passed to
the backend implementation. Ignored otherwise.
Expand Down Expand Up @@ -939,7 +949,9 @@ async def create(
Memory layout to be used within each chunk.
If not specified, the ``array.order`` parameter in the global config will be used.
store : StoreLike or None, default=None
Store or path to directory in file system or name of zip file.
StoreLike object to open. See the
[storage documentation in the user guide][user-guide-store-like]
for a description of all valid StoreLike values.
synchronizer : object, optional
Array synchronizer.
overwrite : bool, optional
Expand Down Expand Up @@ -1250,7 +1262,9 @@ async def open_array(
Parameters
----------
store : StoreLike
Store or path to directory in file system or name of zip file.
StoreLike object to open. See the
[storage documentation in the user guide][user-guide-store-like]
for a description of all valid StoreLike values.
zarr_version : {2, 3, None}, optional
The zarr format to use when saving. Deprecated in favor of zarr_format.
zarr_format : {2, 3, None}, optional
Expand Down
66 changes: 42 additions & 24 deletions src/zarr/api/synchronous.py
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,9 @@ def consolidate_metadata(
Parameters
----------
store : StoreLike
The store-like object whose metadata you wish to consolidate.
The store-like object whose metadata you wish to consolidate. See the
[storage documentation in the user guide][user-guide-store-like]
for a description of all valid StoreLike values.
path : str, optional
A path to a group in the store to consolidate at. Only children
below that group will be consolidated.
Expand Down Expand Up @@ -143,7 +145,9 @@ def load(
Parameters
----------
store : StoreLike
Store or path to directory in file system or name of zip file.
StoreLike object to open. See the
[storage documentation in the user guide][user-guide-store-like]
for a description of all valid StoreLike values.
path : str or None, optional
The path within the store from which to load.

Expand Down Expand Up @@ -183,7 +187,9 @@ def open(
Parameters
----------
store : StoreLike or None, default=None
Store or path to directory in file system or name of zip file.
StoreLike object to open. See the
[storage documentation in the user guide][user-guide-store-like]
for a description of all valid StoreLike values.
mode : {'r', 'r+', 'a', 'w', 'w-'}, optional
Persistence mode: 'r' means read only (must exist); 'r+' means
read/write (must exist); 'a' means read/write (create if doesn't
Expand Down Expand Up @@ -245,7 +251,9 @@ def save(
Parameters
----------
store : StoreLike
Store or path to directory in file system or name of zip file.
StoreLike object to open. See the
[storage documentation in the user guide][user-guide-store-like]
for a description of all valid StoreLike values.
*args : ndarray
NumPy arrays with data to save.
zarr_format : {2, 3, None}, optional
Expand Down Expand Up @@ -279,7 +287,9 @@ def save_array(
Parameters
----------
store : StoreLike
Store or path to directory in file system or name of zip file.
StoreLike object to open. See the
[storage documentation in the user guide][user-guide-store-like]
for a description of all valid StoreLike values.
arr : ndarray
NumPy array with data to save.
zarr_format : {2, 3, None}, optional
Expand Down Expand Up @@ -322,7 +332,9 @@ def save_group(
Parameters
----------
store : StoreLike
Store or path to directory in file system or name of zip file.
StoreLike object to open. See the
[storage documentation in the user guide][user-guide-store-like]
for a description of all valid StoreLike values.
*args : ndarray
NumPy arrays with data to save.
zarr_format : {2, 3, None}, optional
Expand Down Expand Up @@ -413,7 +425,9 @@ def group(
Parameters
----------
store : StoreLike or None, default=None
Store or path to directory in file system or name of zip file.
StoreLike object to open. See the
[storage documentation in the user guide][user-guide-store-like]
for a description of all valid StoreLike values.
overwrite : bool, optional
If True, delete any pre-existing data in `store` at `path` before
creating the group.
Expand Down Expand Up @@ -480,17 +494,9 @@ def open_group(
Parameters
----------
store : StoreLike or None, default=None
Store or path to directory in file system or name of zip file.

Strings are interpreted as paths on the local file system
and used as the ``root`` argument to [zarr.storage.LocalStore][].

Dictionaries are used as the ``store_dict`` argument in
[zarr.storage.MemoryStore][].

By default (``store=None``) a new [zarr.storage.MemoryStore][]
is created.

StoreLike object to open. See the
[storage documentation in the user guide][user-guide-store-like]
for a description of all valid StoreLike values.
mode : {'r', 'r+', 'a', 'w', 'w-'}, optional
Persistence mode: 'r' means read only (must exist); 'r+' means
read/write (must exist); 'a' means read/write (create if doesn't
Expand All @@ -505,7 +511,9 @@ def open_group(
path : str, optional
Group path within store.
chunk_store : StoreLike or None, default=None
Store or path to directory in file system or name of zip file.
Separate storage for chunks. See the
[storage documentation in the user guide][user-guide-store-like]
for a description of all valid StoreLike values.
storage_options : dict
If using an fsspec URL to create the store, these will be passed to
the backend implementation. Ignored otherwise.
Expand Down Expand Up @@ -570,7 +578,9 @@ def create_group(
Parameters
----------
store : StoreLike
Store or path to directory in file system or name of zip file.
StoreLike object to open. See the
[storage documentation in the user guide][user-guide-store-like]
for a description of all valid StoreLike values.
path : str, optional
Group path within store.
overwrite : bool, optional
Expand Down Expand Up @@ -672,7 +682,9 @@ def create(
Memory layout to be used within each chunk.
If not specified, the ``array.order`` parameter in the global config will be used.
store : StoreLike or None, default=None
Store or path to directory in file system or name of zip file.
StoreLike object to open. See the
[storage documentation in the user guide][user-guide-store-like]
for a description of all valid StoreLike values.
synchronizer : object, optional
Array synchronizer.
overwrite : bool, optional
Expand Down Expand Up @@ -832,7 +844,9 @@ def create_array(
Parameters
----------
store : StoreLike
Store or path to directory in file system or name of zip file.
StoreLike object to open. See the
[storage documentation in the user guide][user-guide-store-like]
for a description of all valid StoreLike values.
name : str or None, optional
The name of the array within the store. If ``name`` is ``None``, the array will be located
at the root of the store.
Expand Down Expand Up @@ -996,7 +1010,9 @@ def from_array(
Parameters
----------
store : StoreLike
Store or path to directory in file system or name of zip file.
StoreLike object to open. See the
[storage documentation in the user guide][user-guide-store-like]
for a description of all valid StoreLike values.
data : Array | array-like
The array to copy.
write_data : bool, default True
Expand Down Expand Up @@ -1333,7 +1349,9 @@ def open_array(
Parameters
----------
store : StoreLike
Store or path to directory in file system or name of zip file.
StoreLike object to open. See the
[storage documentation in the user guide][user-guide-store-like]
for a description of all valid StoreLike values.
zarr_version : {2, 3, None}, optional
The zarr format to use when saving. Deprecated in favor of zarr_format.
zarr_format : {2, 3, None}, optional
Expand Down
Loading
Loading