Skip to content

Conversation

zoewangg
Copy link
Contributor

@zoewangg zoewangg commented Aug 22, 2025

Motivation and Context

Currently retry is not supported for putObject operation in S3 multipart client if a non-file based AsyncRequestBody is configured. When a retry is attempted, the following error will be thrown. See
#6198

 A retry was attempted, but AsyncRequestBody.split does not support retries.

This pull request adds support retries for individual parts of a multipart upload in S3 multipart client through a new API: BufferedSplittableAsyncRequestBody

Code Example:

AsyncRequestBody originalBody = AsyncRequestBody.fromString("Hello World");

BufferedSplittableAsyncRequestBody retryableBody =
    BufferedSplittableAsyncRequestBody.create(originalBody);

s3Client.putObject(r -> r.bucket(TEST_BUCKET).key(TEST_KEY), retryableBody).join();

Modifications

The changes have been reviewed previously in #6313 and #6346

zoewangg and others added 2 commits August 11, 2025 11:22
…rs the entire content and sup… (#6313)

* Add BufferingAsyncRequestBody that buffers the entire content and supports multiple concurrent subscribers

* Apply suggestions from code review


* Update existing byteBuffersAsyncRequestBody to support multicast
…uestBody that returns an SdkP… (#6346)

* Introduce a new split method in AsyncRequestBody that returns an SdkPublisher of ClosableAsyncRequestBody and use it in s3 multipart client

* Fix build and add more tests

* Refactoring based on API surface are review

* Add more tests and address comments

* Deprecate split, update documentation and add changelog entries
@zoewangg zoewangg added the api-surface-area-approved-by-team Indicate API surface area introduced by this PR has been approved by team label Aug 22, 2025
Copy link

Quality Gate Failed Quality Gate failed

Failed conditions
76.4% Coverage on New Code (required ≥ 80%)
7.9% Duplication on New Code (required ≤ 3%)
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

@davidh44 davidh44 marked this pull request as ready for review August 28, 2025 20:08
@davidh44 davidh44 requested a review from a team as a code owner August 28, 2025 20:08
@davidh44 davidh44 changed the title [DRAFT] Stream based upload retry support Stream based upload retry support Aug 28, 2025
@davidh44 davidh44 added this pull request to the merge queue Aug 28, 2025
Merged via the queue into master with commit 5f87126 Aug 28, 2025
48 of 51 checks passed
Copy link

This pull request has been closed and the conversation has been locked. Comments on closed PRs are hard for our team to see. If you need more assistance, please open a new issue that references this one.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 28, 2025
@davidh44 davidh44 deleted the feature/master/mpu-stream-retry branch August 28, 2025 21:00
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
api-surface-area-approved-by-team Indicate API surface area introduced by this PR has been approved by team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants