Skip to content

Update S3 Pre-signed URL GET design with multipart download support #6334

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions docs/design/core/presignedURL-Get/DecisionLog.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,3 +53,10 @@ The team has decided to implement functionality only for S3 async client and def
2. Remove the consumer builder pattern from Get Request Model.

3. throw UnsupportedOperationException for Multipart S3 Client and S3 CRT Client for now.

## Design Review: 07/31/2025
**Source**: Design : Multipart Download Support for Pre-signed URLs
**Attendees**: John Viegas, Zoe Wang, Dongie Agnir, Bole Yi, Ran Vaknin, Saranya Somepalli, David Ho, Olivier Lepage Applin

### Decision Addressed
Initially considered separate discovery request (bytes=0-0) followed by download, but decided to follow AWS Transfer Manager SEP specification using Range: bytes=0-{partSizeInBytes-1} to download first part AND discover total object size simultaneously from Content-Range header response.
28 changes: 27 additions & 1 deletion docs/design/core/presignedURL-Get/Design.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ The Java SDK team has decided to implement a separate `AsyncPresignedUrlExtensio

The design introduces a new helper API `AsyncPresignedUrlExtension` which can be instantiated via the existing `S3AsyncClient`. This extension provides a clean abstraction layer that preserves SDK benefits while handling the unique requirements of pre-signed URL requests.

This design will implement only the GET /download function for presigned URLs for the S3AsyncClient. The synchronous S3Client implementation is deferred to future work.
This design implements GET/download functionality for presigned URLs for the S3AsyncClient, including multipart download support for large objects. The synchronous S3Client implementation is deferred to future work.



Expand Down Expand Up @@ -48,6 +48,32 @@ AsyncPresignedUrlExtension presignedUrlExtension = s3Client.presignedUrlExtensio
CompletableFuture<GetObjectResponse> response = presignedUrlExtension.getObject(request, AsyncResponseTransformer.toBytes());
```

### Multipart Download Support

For objects with size greater than minimumPartSizeInBytes, the SDK automatically uses multipart downloads with HTTP Range headers when multipart is enabled:

```java
// Enable multipart downloads
S3AsyncClient s3Client = S3AsyncClient.builder()
.multipartEnabled(true)
.build();

// Or with custom configuration
MultipartConfiguration config = MultipartConfiguration.builder()
.minimumPartSizeInBytes(8 * 1024 * 1024) // 8MB parts
.build();

S3AsyncClient multipartClient = S3AsyncClient.builder()
.multipartConfiguration(config)
.build();

// Download automatically uses multipart for objects larger than the configured part size
CompletableFuture<ResponseBytes<GetObjectResponse>> response =
multipartClient.presignedUrlExtension().getObject(request, AsyncResponseTransformer.toBytes());
```

The multipart implementation uses Range headers (e.g., `bytes=0-8388607`) instead of partNumber parameters to preserve presigned URL signatures. The first request downloads the initial part while discovering total object size from the Content-Range header.

### AsyncPresignedUrlExtension Interface

```java
Expand Down
Loading