Skip to content
Merged
Changes from 9 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
11ac3d9
Implement asynchronous directory deletion in FsspecStore
carshadi Jan 6, 2025
32d8bc9
Merge branch 'zarr-developers:main' into feat-fsspecstore-bulk-delete
carshadi Jan 6, 2025
bfb4397
Use async batched _rm() for FsspecStore.delete_dir()
carshadi Jan 8, 2025
9812aec
Merge branch 'feat-fsspecstore-bulk-delete' of https://github.com/car…
carshadi Jan 8, 2025
f21f1c2
Suppress allowed exceptions instead of try-except-pass
carshadi Jan 8, 2025
220c28b
Adds note on possibly redundant condition in FsspecStore.delete_dir()
carshadi Jan 8, 2025
ece0f0e
Fix: unpack allowed arguments list
carshadi Jan 8, 2025
a280190
Merge branch 'main' into feat-fsspecstore-bulk-delete
jhamman Jan 10, 2025
bf78caf
Merge branch 'main' into feat-fsspecstore-bulk-delete
jhamman Jan 24, 2025
08dc4f9
Adds tests for FsspecStore.delete_dir
carshadi Jan 24, 2025
6ad9a0a
Update src/zarr/storage/_fsspec.py
carshadi Jan 29, 2025
89e1b5f
Remove supports_listing condition from FsspecStore.delete_dir
carshadi Jan 29, 2025
5e67566
use f-string for url formatting
carshadi Jan 29, 2025
a0214e4
assert `store.fs.asynchronous` instead of `store.fs.async_impl`
carshadi Jan 29, 2025
75ee3a5
Merge remote-tracking branch 'upstream/main' into feat-fsspecstore-bu…
carshadi Jan 29, 2025
aa4a7ff
updates release notes
carshadi Jan 29, 2025
8d44aed
Merge remote-tracking branch 'upstream/main' into feat-fsspecstore-bu…
carshadi Feb 14, 2025
96fb2f6
remove unused import
carshadi Feb 14, 2025
f1d90cc
Explicitly construct wrapped local filesystem for test
carshadi Feb 14, 2025
5c48759
Merge branch 'main' into feat-fsspecstore-bulk-delete
dcherian Feb 14, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions src/zarr/storage/_fsspec.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
from __future__ import annotations

import warnings
import inspect
from contextlib import suppress
from typing import TYPE_CHECKING, Any

from zarr.abc.store import (
Expand Down Expand Up @@ -281,6 +283,23 @@
except self.allowed_exceptions:
pass

async def delete_dir(self, prefix: str) -> None:
# docstring inherited
if not self.supports_deletes:
raise NotImplementedError('This method is only available for stores that support deletes.')

Check warning on line 289 in src/zarr/storage/_fsspec.py

View check run for this annotation

Codecov / codecov/patch

src/zarr/storage/_fsspec.py#L289

Added line #L289 was not covered by tests
if not self.supports_listing:
raise NotImplementedError('This method is only available for stores that support directory listing.')

Check warning on line 291 in src/zarr/storage/_fsspec.py

View check run for this annotation

Codecov / codecov/patch

src/zarr/storage/_fsspec.py#L291

Added line #L291 was not covered by tests
self._check_writable()

path_to_delete = _dereference_path(self.path, prefix)

# this is probably the same condition as `if self.fs.async_impl`
if hasattr(self.fs, "_rm") and inspect.iscoroutinefunction(self.fs._rm):
with suppress(*self.allowed_exceptions):
await self.fs._rm(path_to_delete, recursive=True)
else:
raise NotImplementedError("The store does not support async deletes")

Check warning on line 301 in src/zarr/storage/_fsspec.py

View check run for this annotation

Codecov / codecov/patch

src/zarr/storage/_fsspec.py#L301

Added line #L301 was not covered by tests

async def exists(self, key: str) -> bool:
# docstring inherited
path = _dereference_path(self.path, key)
Expand Down
Loading