Skip to content

Commit b467a60

Browse files
jencymaryjosephjoviegas
jencymaryjoseph
and
joviegas
authored
Add S3 Pre-signed URL GET design document (#6203)
* Add S3 Pre-signed URL GET design document * Update Design.md * Update DecisionLog.md * Updated Design.md followed the format as in https://github.com/aws/aws-sdk-java-v2/tree/master/docs/design/core/batch-manager * Updated DecisionLog.md * Remove images folder * Updated Design.md - removed codegen annotations * Update Design.md - addressed comments --------- Co-authored-by: John Viegas <[email protected]>
1 parent 97009e5 commit b467a60

File tree

2 files changed

+209
-0
lines changed

2 files changed

+209
-0
lines changed

‎docs/design/core/presignedURL-Get/DecisionLog.md

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
2+
# S3 Pre-signed URL GET - Decision Log
3+
4+
## Review Meeting: 06/17/2024
5+
**Attendees**: Alban, John, Zoe, Dongie, Bole, Ran, Saranya
6+
7+
### Closed Decisions
8+
9+
1. Create a new PresignedUrlGetObjectResponse specifically for pre-signed URLs, or use the existing GetObjectResponse? Decided to use the existing GetObjectResponse for pre-signed URL operations as the HTTP response from a pre-signed URL GET is same as a standard S3 GetObject response.
10+
11+
2. Use the existing SDK and S3 exceptions or implement specialized exceptions for validation errors like expired URLs? Decided to utilize existing SDK exceptions rather than creating specialized ones for pre-signed URL operations.
12+
13+
3. Provide additional client-side validation with server-side validation as fallback or just rely entirely on server-side validation from S3? No additional client-side validation will be implemented for pre-signed URLs.
14+
15+
### Discussions Addressed
16+
17+
1. Are there alternative methods to skip signing, such as using NoOpSigner(), instead of setting additional Execution attributes? Added the use of NoOpSigner() in the design doc.
18+
19+
2. Does the S3 response include a checksum? If so, should checksum-related support be implemented in this project, or deferred until after Transfer Manager support is added? S3 Response doesn't include checksum.
20+
21+
3. What should we name the Helper API? Options include PresignedURLManager or PresignedUrlExtension. Will be addressed in the Surface API Review.
22+
23+
## Review Meeting: 06/23/2024
24+
**Attendees**: John, Zoe, Dongie, Bole, Ran, Saranya, Alex, David
25+
26+
### Decisions Addressed
27+
28+
1. Should PresignedUrlGetObjectRequest extend S3Request/SdkRequest? Decided to use a standalone request class with minimal parameters (presignedUrl, rangeStart, rangeEnd) to avoid exposing incompatible configurations like credentials and signers. Internally convert to S3Request for ClientHandler compatibility.
29+
30+
2. Replace IS_DISCOVERED_ENDPOINT execution attribute with a more semantically appropriate solution. Decided to introduce new SKIP_ENDPOINT_RESOLUTION execution attribute specifically for presigned URL scenarios where endpoint resolution should be bypassed, as IS_DISCOVERED_ENDPOINT is tied to deprecated endpoint discovery feature.
31+
32+
3. Use separate rangeStart/rangeEnd fields vs single range string parameter. Decided to use separate rangeStart and rangeEnd Long fields for better user experience, as start/end is more intuitive than string parsing.

‎docs/design/core/presignedURL-Get/Design.md

Lines changed: 177 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,177 @@
1+
# Design Document (S3 Pre-signed URL GET)
2+
3+
## Introduction
4+
5+
This design introduces S3 object downloads using pre-signed URLs in AWS SDK Java v2, providing feature parity with [v1](https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/transfer/PresignedUrlDownload.html). Some customers have described a need for downloading S3 objects through pre-signed URLs while maintaining Client side SDK benefits like automatic retries, metrics collection, and typed response objects.
6+
7+
This document proposes how this functionality should be implemented in the Java SDK v2, addressing customer-requested features ([GitHub Issue #2731](https://github.com/aws/aws-sdk-java-v2/issues/2731), [GitHub Issue #181](https://github.com/aws/aws-sdk-java-v2/issues/181)) by reducing complexity and improving usability for temporary access scenarios.
8+
9+
## Design Review
10+
11+
Look at decision log here: [Decision Log Section](DecisionLog.md)
12+
13+
The Java SDK team has decided to implement a separate `PresignedUrlManager`. The team chose the helper API pattern over direct `S3Client` integration to maintain clean separation of concerns while preserving SDK functionality.
14+
15+
## Overview
16+
17+
The design introduces new helper APIs `AsyncPresignedUrlManager` and `PresignedUrlManager` which can be instantiated via the existing `S3AsyncClient` and `S3Client` respectively. These managers provide a clean abstraction layer that preserves SDK benefits while handling the unique requirements of pre-signed URL requests.
18+
19+
This design will implement only the GET /download function for presigned URLs.
20+
21+
22+
23+
## Proposed APIs
24+
25+
The v2 SDK will support a presigned URL manager for both sync and async clients that can leverage pre-signed URL downloads.
26+
27+
### Instantiation
28+
Instantiating from existing client:
29+
30+
```java
31+
// Async Presigned URL Manager
32+
S3AsyncClient s3Client = S3AsyncClient.create();
33+
AsyncPresignedUrlManager presignManager = s3Client.presignedManager();
34+
35+
// Sync Presigned URL Manager
36+
S3Client s3Client = S3Client.create();
37+
PresignedUrlManager presignManager = s3Client.presignedManager();
38+
```
39+
40+
### General Usage Examples
41+
42+
```java
43+
// Create presigned URL request
44+
PresignedUrlGetObjectRequest request = PresignedUrlGetObjectRequest.builder()
45+
.presignedUrl(presignedUrl)
46+
.rangeStart(0L)
47+
.rangeEnd(1024L)
48+
.build();
49+
50+
// Async usage
51+
S3AsyncClient s3Client = S3AsyncClient.create();
52+
AsyncPresignedUrlManager presignManager = s3Client.presignedManager();
53+
CompletableFuture<GetObjectResponse> response = presignManager.getObject(request);
54+
55+
// Sync usage
56+
S3Client s3Client = S3Client.create();
57+
PresignedUrlManager presignManager = s3Client.presignedManager();
58+
GetObjectResponse response = presignManager.getObject(request);
59+
```
60+
61+
### AsyncPresignedUrlManager Interface
62+
63+
```java
64+
/**
65+
* Interface for presigned URL operations used by Async clients.
66+
*/
67+
@SdkPublicApi
68+
public interface AsyncPresignedUrlManager {
69+
70+
/**
71+
* Downloads S3 objects using pre-signed URLs with custom response transformation.
72+
*
73+
* @param request the presigned URL request.
74+
* @param responseTransformer custom transformer for processing the response.
75+
* @return a CompletableFuture of the transformed response.
76+
*/
77+
<T> CompletableFuture<T> getObject(PresignedUrlGetObjectRequest request,
78+
AsyncResponseTransformer<GetObjectResponse, T> responseTransformer);
79+
80+
// Additional getObject() overloads for file downloads, byte arrays, etc.
81+
// Standard Builder interface with client() and overrideConfiguration() methods
82+
}
83+
```
84+
85+
### PresignedUrlManager Interface
86+
87+
```java
88+
/**
89+
* Interface for presigned URL operations used by Sync clients.
90+
*/
91+
@SdkPublicApi
92+
public interface PresignedUrlManager {
93+
94+
/**
95+
* Downloads S3 objects using pre-signed URLs. Bypasses normal authentication
96+
* and endpoint resolution while maintaining SDK benefits like retries and metrics.
97+
*
98+
* @param request the presigned URL request containing URL and optional range parameters.
99+
* @return the GetObjectResponse.
100+
*/
101+
GetObjectResponse getObject(PresignedUrlGetObjectRequest request);
102+
103+
/**
104+
* Downloads S3 objects using pre-signed URLs with custom response transformation.
105+
*
106+
* @param request the presigned URL request.
107+
* @param responseTransformer custom transformer for processing the response.
108+
* @return the transformed response.
109+
*/
110+
<T> T getObject(PresignedUrlGetObjectRequest request,
111+
ResponseTransformer<GetObjectResponse, T> responseTransformer);
112+
113+
// Additional getObject() overloads for file downloads, byte arrays, etc.
114+
// Standard Builder interface with client() and overrideConfiguration() methods
115+
}
116+
```
117+
118+
### PresignedUrlGetObjectRequest
119+
120+
```java
121+
/**
122+
* Request object for presigned URL GET operations.
123+
*/
124+
@SdkPublicApi
125+
@Immutable
126+
@ThreadSafe
127+
public final class PresignedUrlGetObjectRequest
128+
implements ToCopyableBuilder<PresignedUrlGetObjectRequest.Builder, PresignedUrlGetObjectRequest> {
129+
130+
private final String presignedUrl;
131+
private final Long rangeStart;
132+
private final Long rangeEnd;
133+
134+
// Standard getters: presignedUrl(), rangeStart(), rangeEnd()
135+
// Standard builder methods: builder(), toBuilder()
136+
// Standard Builder class with presignedUrl(), rangeStart(), rangeEnd() setter methods
137+
}
138+
```
139+
140+
## FAQ
141+
142+
### Why don't we implement presigned URL download/GET feature directly on the S3Client?
143+
144+
Three approaches were considered:
145+
146+
1. **Dedicated PresignedUrlManager (CHOSEN)**: Separate manager accessed via `s3Client.presignedManager()`
147+
- **Pros**: Clean separation, preserves SDK features, follows v2 patterns
148+
- **Cons**: New API surface for users to learn
149+
150+
2. **Direct S3Client Integration**: Add presigned URL methods directly to S3Client
151+
- **Pros**: Familiar interface, direct migration path from v1
152+
- **Cons**: Requires core interceptor changes, complex integration, could confuse users by mixing presigned URL APIs with standard service-generated APIs
153+
154+
3. **S3Presigner Extension**: Extend existing S3Presigner to execute URLs
155+
- **Pros**: Logical extension of presigner concept
156+
- **Cons**: Breaks current stateless presigner patterns
157+
158+
**Decision**: Option 1 provides clean separation while preserving SDK benefits and following established v2 utility patterns.cutePresignedGetObject(presignedRequest);
159+
160+
### Why doesn't PresignedUrlGetObjectRequest extend S3Request?
161+
162+
While extending S3Request would provide access to RequestOverrideConfiguration, many of these configurations (like credentials provider, signers) are not supported with presigned URL execution. Instead, we use a standalone request with only essential parameters (presignedUrl, rangeStart, rangeEnd). Internally, this gets wrapped in an encapsulated class that extends S3Request for use with ClientHandler.
163+
164+
165+
## References
166+
167+
**GitHub feature requests:**
168+
- [S3 Presigned URL Support #2731](https://github.com/aws/aws-sdk-java-v2/issues/2731)
169+
- [Presigned URL GET Support #181](https://github.com/aws/aws-sdk-java-v2/issues/181)
170+
171+
**AWS Documentation:**
172+
- [S3 Pre-signed URLs](https://docs.aws.amazon.com/AmazonS3/latest/userguide/presigned-urls.html)
173+
174+
**SDK Documentation:**
175+
- [AWS SDK for Java v1 implementation](https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/welcome.html)
176+
- [S3 Client architecture patterns](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html)
177+

0 commit comments

Comments
 (0)