Skip to content

Commit 8e3c42b

Browse files
authored
Docs for Zarr storage backends (#791)
1 parent 6a3faeb commit 8e3c42b

File tree

1 file changed

+24
-0
lines changed

1 file changed

+24
-0
lines changed

docs/user-guide/storage.md

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,3 +27,27 @@ To set up a lifecycle rule:
2727
* For **AWS S3**, follow [these instructions](https://lepczynski.it/en/aws_en/automatically-delete-old-files-from-aws-s3/).
2828

2929
If you use this approach then be sure to store persistent data in a separate bucket to the one used for intermediate data.
30+
31+
## Zarr backend
32+
33+
Cubed uses the [`zarr-python`](https://github.com/zarr-developers/zarr-python) library for reading and writing intermediate Zarr data, but it is possible to override this and use another Zarr library by setting the `CUBED_STORAGE_NAME` environment variable.
34+
35+
To use the [`zarrs-python`](https://github.com/zarrs/zarrs-python) Rust implementation, install the `zarrs` Python package and set the `CUBED_STORAGE_NAME` environment variable to `zarrs-python`:
36+
37+
```shell
38+
export CUBED_STORAGE_NAME=zarrs-python
39+
```
40+
41+
To use [Tensorstore](https://google.github.io/tensorstore/), install the `tensorstore` Python package and set the `CUBED_STORAGE_NAME` environment variable to `tensorstore`:
42+
43+
```shell
44+
export CUBED_STORAGE_NAME=tensorstore
45+
```
46+
47+
For `zarr-python` v3 only, you can use Zarr's [`ObjectStore`](https://zarr.readthedocs.io/en/main/api/zarr/storage/index.html#zarr.storage.ObjectStore) which uses the Rust-based [`obstore`](https://developmentseed.org/obstore/latest/), by setting the `storage_options.use_obstore` [configuration](../configuration.md) option to `True`, as illustrated in this YAML file:
48+
49+
```yaml
50+
spec:
51+
storage_options:
52+
use_obstore: True
53+
```

0 commit comments

Comments
 (0)