Skip to content

Add S3 Pre-signed URL GET design document #6203

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Jun 27, 2025
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions docs/design/core/presignedURL-Get/DecisionLog.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@

# S3 Pre-signed URL GET - Decision Log

## Review Meeting: 06/17/2024
**Attendees**: Alban, John, Zoe, Dongie, Bole, Ran, Saranya

### Closed Decisions

1. Create a new PresignedUrlGetObjectResponse specifically for pre-signed URLs, or use the existing GetObjectResponse? Decided to use the existing GetObjectResponse for pre-signed URL operations as the HTTP response from a pre-signed URL GET is same as a standard S3 GetObject response.

2. Use the existing SDK and S3 exceptions or implement specialized exceptions for validation errors like expired URLs? Decided to utilize existing SDK exceptions rather than creating specialized ones for pre-signed URL operations.

3. Provide additional client-side validation with server-side validation as fallback or just rely entirely on server-side validation from S3? No additional client-side validation will be implemented for pre-signed URLs.

### Discussions Addressed

1. Are there alternative methods to skip signing, such as using NoOpSigner(), instead of setting additional Execution attributes? Added the use of NoOpSigner() in the design doc.

2. Does the S3 response include a checksum? If so, should checksum-related support be implemented in this project, or deferred until after Transfer Manager support is added? S3 Response doesn't include checksum.

3. What should we name the Helper API? Options include PresignedURLManager or PresignedUrlExtension. Will be addressed in the Surface API Review.

## Review Meeting: 06/23/2024
**Attendees**: John, Zoe, Dongie, Bole, Ran, Saranya, Alex, David

### Decisions Addressed

1. Should PresignedUrlGetObjectRequest extend S3Request/SdkRequest? Decided to use a standalone request class with minimal parameters (presignedUrl, rangeStart, rangeEnd) to avoid exposing incompatible configurations like credentials and signers. Internally convert to S3Request for ClientHandler compatibility.

2. Replace IS_DISCOVERED_ENDPOINT execution attribute with a more semantically appropriate solution. Decided to introduce new SKIP_ENDPOINT_RESOLUTION execution attribute specifically for presigned URL scenarios where endpoint resolution should be bypassed, as IS_DISCOVERED_ENDPOINT is tied to deprecated endpoint discovery feature.

3. Use separate rangeStart/rangeEnd fields vs single range string parameter. Decided to use separate rangeStart and rangeEnd Long fields for better user experience, as start/end is more intuitive than string parsing.
188 changes: 188 additions & 0 deletions docs/design/core/presignedURL-Get/Design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,188 @@
# Design Document (S3 Pre-signed URL GET)

## Introduction

This design introduces S3 object downloads using pre-signed URLs in AWS SDK Java v2, providing feature parity with [v1](https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/transfer/PresignedUrlDownload.html). Some customers have described a need for downloading S3 objects through pre-signed URLs while maintaining Client side SDK benefits like automatic retries, metrics collection, and typed response objects.

This document proposes how this functionality should be implemented in the Java SDK v2, addressing customer-requested features ([GitHub Issue #2731](https://github.com/aws/aws-sdk-java-v2/issues/2731), [GitHub Issue #181](https://github.com/aws/aws-sdk-java-v2/issues/181)) by reducing complexity and improving usability for temporary access scenarios.

## Design Review

Look at decision log here: [Decision Log Section](DecisionLog.md)

The Java SDK team has decided to implement a separate `PresignedUrlManager`. The team chose the helper API pattern over direct `S3Client` integration to maintain clean separation of concerns while preserving SDK functionality.

## Overview

The design introduces new helper APIs `AsyncPresignedUrlManager` and `PresignedUrlManager` which can be instantiated via the existing `S3AsyncClient` and `S3Client` respectively. These managers provide a clean abstraction layer that preserves SDK benefits while handling the unique requirements of pre-signed URL requests.

This design will implement only the GET /download function for presigned URLs.



## Proposed APIs

The v2 SDK will support a presigned URL manager for both sync and async clients that can leverage pre-signed URL downloads.

### Instantiation
Instantiating from existing client:

```java
// Async Presigned URL Manager
S3AsyncClient s3Client = S3AsyncClient.create();
AsyncPresignedUrlManager presignManager = s3Client.presignedManager();

// Sync Presigned URL Manager
S3Client s3Client = S3Client.create();
PresignedUrlManager presignManager = s3Client.presignedManager();
```

### General Usage Examples

```java
// Create presigned URL request
PresignedUrlGetObjectRequest request = PresignedUrlGetObjectRequest.builder()
.presignedUrl(presignedUrl)
.rangeStart(0L)
.rangeEnd(1024L)
.build();

// Async usage
S3AsyncClient s3Client = S3AsyncClient.create();
AsyncPresignedUrlManager presignManager = s3Client.presignedManager();
CompletableFuture<GetObjectResponse> response = presignManager.getObject(request);

// Sync usage
S3Client s3Client = S3Client.create();
PresignedUrlManager presignManager = s3Client.presignedManager();
GetObjectResponse response = presignManager.getObject(request);
```

### AsyncPresignedUrlManager Interface

```java
/**
* Presigned URL Manager class that implements presigned URL features for an async client.
*/
@SdkPublicApi
@Generated("software.amazon.awssdk:codegen")
public interface AsyncPresignedUrlManager {

/**
* Downloads S3 objects using pre-signed URLs. Bypasses normal authentication
* and endpoint resolution while maintaining SDK benefits like retries and metrics.
*
* @param request the presigned URL request containing URL and optional range parameters.
* @return a CompletableFuture of the corresponding GetObjectResponse.
*/
CompletableFuture<GetObjectResponse> getObject(PresignedUrlGetObjectRequest request);

/**
* Downloads S3 objects using pre-signed URLs with custom response transformation.
*
* @param request the presigned URL request.
* @param responseTransformer custom transformer for processing the response.
* @return a CompletableFuture of the transformed response.
*/
<T> CompletableFuture<T> getObject(PresignedUrlGetObjectRequest request,
AsyncResponseTransformer<GetObjectResponse, T> responseTransformer);

// Additional getObject() overloads for file downloads, byte arrays, etc.
// Standard Builder interface with client() and overrideConfiguration() methods
}
```

### PresignedUrlManager Interface

```java
/**
* Presigned URL Manager class that implements presigned URL features for a sync client.
*/
@SdkPublicApi
@Generated("software.amazon.awssdk:codegen")
public interface PresignedUrlManager {

/**
* Downloads S3 objects using pre-signed URLs. Bypasses normal authentication
* and endpoint resolution while maintaining SDK benefits like retries and metrics.
*
* @param request the presigned URL request containing URL and optional range parameters.
* @return the GetObjectResponse.
*/
GetObjectResponse getObject(PresignedUrlGetObjectRequest request);

/**
* Downloads S3 objects using pre-signed URLs with custom response transformation.
*
* @param request the presigned URL request.
* @param responseTransformer custom transformer for processing the response.
* @return the transformed response.
*/
<T> T getObject(PresignedUrlGetObjectRequest request,
ResponseTransformer<GetObjectResponse, T> responseTransformer);

// Additional getObject() overloads for file downloads, byte arrays, etc.
// Standard Builder interface with client() and overrideConfiguration() methods
}
```

### PresignedUrlGetObjectRequest

```java
/**
* Request object for presigned URL GET operations.
*/
@SdkPublicApi
@Immutable
@ThreadSafe
public final class PresignedUrlGetObjectRequest
implements ToCopyableBuilder<PresignedUrlGetObjectRequest.Builder, PresignedUrlGetObjectRequest> {

private final String presignedUrl;
private final Long rangeStart;
private final Long rangeEnd;

// Standard getters: presignedUrl(), rangeStart(), rangeEnd()
// Standard builder methods: builder(), toBuilder()
// Standard Builder class with presignedUrl(), rangeStart(), rangeEnd() setter methods
}
```

## FAQ

### Why don't we implement presigned URL download/GET feature directly on the S3Client?

Three approaches were considered:

1. **Dedicated PresignedUrlManager (CHOSEN)**: Separate manager accessed via `s3Client.presignedManager()`
- **Pros**: Clean separation, preserves SDK features, follows v2 patterns
- **Cons**: New API surface for users to learn

2. **Direct S3Client Integration**: Add presigned URL methods directly to S3Client
- **Pros**: Familiar interface, direct migration path from v1
- **Cons**: Requires core interceptor changes, complex integration

3. **S3Presigner Extension**: Extend existing S3Presigner to execute URLs
- **Pros**: Logical extension of presigner concept
- **Cons**: Breaks current stateless presigner patterns

**Decision**: Option 1 provides clean separation while preserving SDK benefits and following established v2 utility patterns.cutePresignedGetObject(presignedRequest);

### Why doesn't PresignedUrlGetObjectRequest extend S3Request?

While extending S3Request would provide access to RequestOverrideConfiguration, many of these configurations (like credentials provider, signers) are not supported with presigned URL execution and could cause conflicts. Instead, we use a standalone request with only essential parameters (presignedUrl, rangeStart, rangeEnd). Internally, this gets wrapped in an encapsulated class that extends S3Request for use with ClientHandler.


## References

**GitHub feature requests:**
- [S3 Presigned URL Support #2731](https://github.com/aws/aws-sdk-java-v2/issues/2731)
- [Presigned URL GET Support #181](https://github.com/aws/aws-sdk-java-v2/issues/181)

**AWS Documentation:**
- [S3 Pre-signed URLs](https://docs.aws.amazon.com/AmazonS3/latest/userguide/presigned-urls.html)

**SDK Documentation:**
- [AWS SDK for Java v1 implementation](https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/welcome.html)
- [S3 Client architecture patterns](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html)