Skip to content

Conversation

@sksingh2005
Copy link

Which issue does this PR close?

Closes #7090 .

Rationale for this change

AWS S3 now supports conditional headers (If-Match and If-None-Match) for CopyObject and DeleteObject operations, enabling atomic operations and preventing race conditions in concurrent scenarios. This PR adds support for these headers to OpenDAL's S3 service, bringing it to feature parity with AWS S3's latest capabilities and matching the existing implementation in the Azure Blob service.

What changes are included in this PR?

This PR implements three new conditional operation capabilities for S3:

  1. copy_with_if_not_exists
  • Copy operations that only succeed if the target doesn't exist (using If-None-Match: *)
  1. copy_with_if_match
  • Copy operations that only succeed if the source's ETag matches (using If-Match: <etag>)
  1. delete_with_if_match
  • Delete operations that only succeed if the file's ETag matches (using If-Match: <etag>)

Core Changes:

  • Added copy_with_if_match and delete_with_if_match capability flags
  • Extended OpCopy and OpDelete with if_match fields
  • Added if_match to CopyOptions and DeleteOptions
  • Exposed if_match() methods in FutureCopy and FutureDelete for user-facing API
    S3 Service Implementation:
  • Updated S3 backend to advertise new capabilities
  • Implemented conditional headers in s3_copy_object() and s3_delete_object()
  • Added proper header handling for If-Match and If-None-Match
    Testing:
  • Added 4 new behavior tests covering success and failure scenarios
  • Tests follow existing OpenDAL patterns and are registered in the test suite

Are there any user-facing changes?

Yes, this PR adds new user-facing methods:

// Copy with If-Match header
op.copy_with("source", "target")
    .if_match("\"etag-value\"")
    .await?;
// Delete with If-Match header
op.delete_with("path")
    .if_match("\"etag-value\"")
    .await?;

AI Usage Statement

This PR was implemented with assistance from Claude Sonnet 4.5 through the Antigravity coding assistant. The AI helped with:

  • Test case design based on existing behavior tests

All code was reviewed, tested, and validated by the human contributor.

@sksingh2005 sksingh2005 requested a review from Xuanwo as a code owner December 31, 2025 23:42
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. releases-note/feat The PR implements a new feature or has a title that begins with "feat" labels Dec 31, 2025
@sksingh2005 sksingh2005 requested a review from suyanhanx as a code owner January 3, 2026 11:21
Copy link
Member

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most changes LGTM, thank you for working on this!

copy_with_if_match: true,

list: true,
rename: false,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems we changed list: true to rename: false? Why this change?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My apologies, it seems list: true was accidentally removed during the rename capability update. I have restored it. We need rename: false to force the Copy + Delete polyfill because S3 doesn't support native rename, but list: true is definitely required i guess sir ??

@sksingh2005 sksingh2005 force-pushed the feature/s3 branch 2 times, most recently from edada25 to 768b8b5 Compare January 4, 2026 14:50
@sksingh2005 sksingh2005 requested a review from Xuanwo January 4, 2026 14:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

releases-note/feat The PR implements a new feature or has a title that begins with "feat" size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

new feature: s3 conditional copy & delete

2 participants