@@ -854,13 +854,44 @@ please raise an issue on the GitHub issue tracker with any profiling data you
854
854
can provide, as there may be opportunities to optimise further either within
855
855
Zarr or within the mapping interface to the storage.
856
856
857
+ IO with ``fsspec ``
858
+ ~~~~~~~~~~~~~~~~~~
859
+
860
+ As of version 2.5, zarr supports passing URLs directly to `fsspec `_,
861
+ and having it create the "mapping" instance automatically. This means, that
862
+ for all of the backend storage implementations `supported by fsspec `_,
863
+ you can skip importing and configuring the storage explicitly.
864
+ For example::
865
+
866
+ >>> g = zarr.open_group("s3://zarr-demo/store", storage_options={'anon': True})
867
+ >>> g['foo/bar/baz'][:].tobytes()
868
+ b'Hello from the cloud!'
869
+
870
+ The provision of the protocol specifier "s3://" will select the correct backend.
871
+ Notice the kwargs ``storage_options ``, used to pass parameters to that backend.
872
+
873
+ As of version 2.6, write mode and complex URLs are also supported, such that in::
874
+
875
+ >>> g = zarr.open_group("simplecache::s3://zarr-demo/store",
876
+ ... storage_options={"s3": {'anon': True}})
877
+ >>> g['foo/bar/baz'][:].tobytes() # downloads target file
878
+ b'Hello from the cloud!'
879
+ >>> g['foo/bar/baz'][:].tobytes() # uses cached file
880
+ b'Hello from the cloud!'
881
+
882
+ The second invocation here will be much faster. Note that the ``storage_options ``
883
+ have become more complex here, to account for the two parts of the supplied
884
+ URL.
885
+
886
+ .. _fsspec : https://filesystem-spec.readthedocs.io/en/latest/
887
+
888
+ .. _supported by fsspec : https://filesystem-spec.readthedocs.io/en/latest/api.html#built-in-implementations
889
+
857
890
.. _tutorial_copy :
858
891
859
892
Consolidating metadata
860
893
~~~~~~~~~~~~~~~~~~~~~~
861
894
862
- (This is an experimental feature.)
863
-
864
895
Since there is a significant overhead for every connection to a cloud object
865
896
store such as S3, the pattern described in the previous section may incur
866
897
significant latency while scanning the metadata of the array hierarchy, even
0 commit comments