feat(storage): add deleteSourceObjects option to combine/compose method#8444
feat(storage): add deleteSourceObjects option to combine/compose method#8444thiyaguk09 wants to merge 8 commits into
Conversation
…ntroduce ComposeCleanupError for failed source deletions
…te delete options interface
There was a problem hiding this comment.
Code Review
This pull request introduces a new deleteSourceObjects option for the combine operation in the Bucket class, allowing for the automatic deletion of source objects after a successful compose operation. It also adds a ComposeCleanupError class to handle partial failures during this cleanup process. The review feedback highlights critical safety and robustness concerns regarding potential data loss and concurrent deletion issues, suggesting the use of ifGenerationMatch and ignoreNotFound: true in the delete calls, along with corresponding updates to the test suite.
… and ignore missing sources during deletion
| userProject: options.userProject, | ||
| }; | ||
|
|
||
| const generation = source.generation ?? source.metadata?.generation; |
There was a problem hiding this comment.
Why do we only check source.metadata.generation when building the Compose payload, but check both source.generation ?? source.metadata?.generation during the delete phase?
There was a problem hiding this comment.
I updated Compose payload generation checks to match delete cleanup logic for consistency.
Description
This PR implements a client-side workaround for the
deleteSourceObjectsparameter in the GCS Compose API (combinemethod).Key Changes
deleteSourceObjects?: booleantoCombineOptionsand documented it in thecombinemethod JSDocs.deleteSourceObjectsto the JSON request body of the/composerequest (omitted ifundefinedto ensure backward compatibility and clean requests).deleteSourceObjectsis set totrueand the compose operation is successful, programmatically deletes all source files.ComposeCleanupError):ComposeCleanupErrorclass.ComposeCleanupErroris thrown/returned.errors) and attaches the newly composed file (newFile) and rawapiResponseso they are still accessible to the caller.test/bucket.tsverifying all success, failure, and partial cleanup error conditions.Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
Fixes #<issue_number_goes_here> 🦕