-
Notifications
You must be signed in to change notification settings - Fork 71
batch deletions on azure / limited to 256 - Closes #509 #508
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
batch deletions on azure / limited to 256 - Closes #509 #508
Conversation
|
Thanks @alikefia, looks good in principal. Can you follow the other checklist items to lint, add changelog entry, and add a test? |
|
@alikefia Looks like |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## 508-live-tests #508 +/- ##
================================================
- Coverage 93.7% 92.3% -1.4%
================================================
Files 23 23
Lines 1798 1800 +2
================================================
- Hits 1685 1663 -22
- Misses 113 137 +24
🚀 New features to boost your workflow:
|
|
Docs failure is unrelated; pushing to local branch to run live tests. |
…508) (#511) * batch deletions on azure / limited to 256 - Closes #509 (#508) * batch deletions on azure / limited to 256 * test + changelog * avoid batched / compatibility * minimal diff * Update docstrings handler --------- Co-authored-by: Ali Kefia <[email protected]>

One of the implementations of delete folder on azure lists all the blobs and delete them directly. The delete request on azure blobs is limited to 256 objects per request (https://learn.microsoft.com/en-us/python/api/azure-storage-blob/azure.storage.blob.containerclient?view=azure-python#azure-storage-blob-containerclient-delete-blobs)
The proposal is to batch the call
Contributor checklist:
CONTRIBUTING.mdCloses #issueappears in the PR summary (e.g.,Closes #123).HISTORY.mdwith the issue that is addressed and the PR you are submitting. If the top section is not `## UNRELEASED``, then you need to add a new section to the top of the document for your change.