Implementing no-overwrite for sync and download #9609
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rakshil14-2
force-pushed
the
no_overwrite_for_sync_operation
branch
2 times, most recently
from
2b8e96b to
8f26bf8
Compare
rakshil14-2
force-pushed
the
no_overwrite_for_sync_operation
branch
from
cf15105 to
8f26bf8
Compare
added 2 commits
July 28, 2025 10:18
Adding Warning handler for no_overwrite
handled error during download streaming streaming
rakshil14-2
force-pushed
the
no_overwrite_for_sync_operation
branch
from
ede77f2 to
db0a335
Compare
hssyoo
requested changes
Jul 30, 2025
hssyoo
reviewed
Aug 4, 2025
hssyoo
requested changes
Aug 4, 2025
hssyoo
approved these changes
Aug 4, 2025
added 2 commits
August 5, 2025 01:58
Updated test case Updated doc comment Added test cases Updating Sync Strategy Updated comments Updated comments Override sync strategy Overridding sync strategy
Added no overwrite for download operation handled error during download streaming streaming Improved upload test cases Updated test cases Added test cases Resolved requested changes NIT change
rakshil14-2
force-pushed
the
no_overwrite_for_sync_operation
branch
from
cb64e3a to
eb56a58
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue : Github Issue #2874
Description of changes:
Customers requested to prevent overwriting of objects. This is done by providing
no-overwriteheader. This PR contains implementation of--no-overwriteflag for 2 operations i.e download and sync operations.To prevent overwriting of the file, before initiating the download, file is begin checked at the local system and if the file is absent downloading is allowed else, operation is terminated.
To implement
--no-overwritea new sync strategy has been introduced named to allow transfer of only those files that are not present at the destination. Since files have already been separated before initiating the operation, so again checking for its existence would be redundant and causes extra api calls hence the argument is removed before the transfer begins. This strategy is independent of size and timestamps. SinceNo-overwritearg is common for all the operation so now it has been moved to Transfer Args which is used bycp,mvandsynccommandsTesting:
Functional testing is performed to validate the object transfer processes. This included both scenarios transferring an object when object with same key is already present in the destination as well as those when the object with same key is not present on destination. These tests were conducted for both object having size less than multipart threshold and one with greater than multipart threshold.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.