diff --git a/docs/design/core/presignedURL-Get/DecisionLog.md b/docs/design/core/presignedURL-Get/DecisionLog.md new file mode 100644 index 000000000000..345a3ec9b584 --- /dev/null +++ b/docs/design/core/presignedURL-Get/DecisionLog.md @@ -0,0 +1,32 @@ + +# S3 Pre-signed URL GET - Decision Log + +## Review Meeting: 06/17/2024 +**Attendees**: Alban, John, Zoe, Dongie, Bole, Ran, Saranya + +### Closed Decisions + +1. Create a new PresignedUrlGetObjectResponse specifically for pre-signed URLs, or use the existing GetObjectResponse? Decided to use the existing GetObjectResponse for pre-signed URL operations as the HTTP response from a pre-signed URL GET is same as a standard S3 GetObject response. + +2. Use the existing SDK and S3 exceptions or implement specialized exceptions for validation errors like expired URLs? Decided to utilize existing SDK exceptions rather than creating specialized ones for pre-signed URL operations. + +3. Provide additional client-side validation with server-side validation as fallback or just rely entirely on server-side validation from S3? No additional client-side validation will be implemented for pre-signed URLs. + +### Discussions Addressed + +1. Are there alternative methods to skip signing, such as using NoOpSigner(), instead of setting additional Execution attributes? Added the use of NoOpSigner() in the design doc. + +2. Does the S3 response include a checksum? If so, should checksum-related support be implemented in this project, or deferred until after Transfer Manager support is added? S3 Response doesn't include checksum. + +3. What should we name the Helper API? Options include PresignedURLManager or PresignedUrlExtension. Will be addressed in the Surface API Review. + +## Review Meeting: 06/23/2024 +**Attendees**: John, Zoe, Dongie, Bole, Ran, Saranya, Alex, David + +### Decisions Addressed + +1. Should PresignedUrlGetObjectRequest extend S3Request/SdkRequest? Decided to use a standalone request class with minimal parameters (presignedUrl, rangeStart, rangeEnd) to avoid exposing incompatible configurations like credentials and signers. Internally convert to S3Request for ClientHandler compatibility. + +2. Replace IS_DISCOVERED_ENDPOINT execution attribute with a more semantically appropriate solution. Decided to introduce new SKIP_ENDPOINT_RESOLUTION execution attribute specifically for presigned URL scenarios where endpoint resolution should be bypassed, as IS_DISCOVERED_ENDPOINT is tied to deprecated endpoint discovery feature. + +3. Use separate rangeStart/rangeEnd fields vs single range string parameter. Decided to use separate rangeStart and rangeEnd Long fields for better user experience, as start/end is more intuitive than string parsing. diff --git a/docs/design/core/presignedURL-Get/Design.md b/docs/design/core/presignedURL-Get/Design.md new file mode 100644 index 000000000000..a99d0b8bfccb --- /dev/null +++ b/docs/design/core/presignedURL-Get/Design.md @@ -0,0 +1,177 @@ +# Design Document (S3 Pre-signed URL GET) + +## Introduction + +This design introduces S3 object downloads using pre-signed URLs in AWS SDK Java v2, providing feature parity with [v1](https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/transfer/PresignedUrlDownload.html). Some customers have described a need for downloading S3 objects through pre-signed URLs while maintaining Client side SDK benefits like automatic retries, metrics collection, and typed response objects. + +This document proposes how this functionality should be implemented in the Java SDK v2, addressing customer-requested features ([GitHub Issue #2731](https://github.com/aws/aws-sdk-java-v2/issues/2731), [GitHub Issue #181](https://github.com/aws/aws-sdk-java-v2/issues/181)) by reducing complexity and improving usability for temporary access scenarios. + +## Design Review + +Look at decision log here: [Decision Log Section](DecisionLog.md) + +The Java SDK team has decided to implement a separate `PresignedUrlManager`. The team chose the helper API pattern over direct `S3Client` integration to maintain clean separation of concerns while preserving SDK functionality. + +## Overview + +The design introduces new helper APIs `AsyncPresignedUrlManager` and `PresignedUrlManager` which can be instantiated via the existing `S3AsyncClient` and `S3Client` respectively. These managers provide a clean abstraction layer that preserves SDK benefits while handling the unique requirements of pre-signed URL requests. + +This design will implement only the GET /download function for presigned URLs. + + + +## Proposed APIs + +The v2 SDK will support a presigned URL manager for both sync and async clients that can leverage pre-signed URL downloads. + +### Instantiation +Instantiating from existing client: + +```java +// Async Presigned URL Manager +S3AsyncClient s3Client = S3AsyncClient.create(); +AsyncPresignedUrlManager presignManager = s3Client.presignedManager(); + +// Sync Presigned URL Manager +S3Client s3Client = S3Client.create(); +PresignedUrlManager presignManager = s3Client.presignedManager(); +``` + +### General Usage Examples + +```java +// Create presigned URL request +PresignedUrlGetObjectRequest request = PresignedUrlGetObjectRequest.builder() + .presignedUrl(presignedUrl) + .rangeStart(0L) + .rangeEnd(1024L) + .build(); + +// Async usage +S3AsyncClient s3Client = S3AsyncClient.create(); +AsyncPresignedUrlManager presignManager = s3Client.presignedManager(); +CompletableFuture response = presignManager.getObject(request); + +// Sync usage +S3Client s3Client = S3Client.create(); +PresignedUrlManager presignManager = s3Client.presignedManager(); +GetObjectResponse response = presignManager.getObject(request); +``` + +### AsyncPresignedUrlManager Interface + +```java +/** + * Interface for presigned URL operations used by Async clients. + */ +@SdkPublicApi +public interface AsyncPresignedUrlManager { + + /** + * Downloads S3 objects using pre-signed URLs with custom response transformation. + * + * @param request the presigned URL request. + * @param responseTransformer custom transformer for processing the response. + * @return a CompletableFuture of the transformed response. + */ + CompletableFuture getObject(PresignedUrlGetObjectRequest request, + AsyncResponseTransformer responseTransformer); + + // Additional getObject() overloads for file downloads, byte arrays, etc. + // Standard Builder interface with client() and overrideConfiguration() methods +} +``` + +### PresignedUrlManager Interface + +```java +/** + * Interface for presigned URL operations used by Sync clients. + */ +@SdkPublicApi +public interface PresignedUrlManager { + + /** + * Downloads S3 objects using pre-signed URLs. Bypasses normal authentication + * and endpoint resolution while maintaining SDK benefits like retries and metrics. + * + * @param request the presigned URL request containing URL and optional range parameters. + * @return the GetObjectResponse. + */ + GetObjectResponse getObject(PresignedUrlGetObjectRequest request); + + /** + * Downloads S3 objects using pre-signed URLs with custom response transformation. + * + * @param request the presigned URL request. + * @param responseTransformer custom transformer for processing the response. + * @return the transformed response. + */ + T getObject(PresignedUrlGetObjectRequest request, + ResponseTransformer responseTransformer); + + // Additional getObject() overloads for file downloads, byte arrays, etc. + // Standard Builder interface with client() and overrideConfiguration() methods +} +``` + +### PresignedUrlGetObjectRequest + +```java +/** + * Request object for presigned URL GET operations. + */ +@SdkPublicApi +@Immutable +@ThreadSafe +public final class PresignedUrlGetObjectRequest + implements ToCopyableBuilder { + + private final String presignedUrl; + private final Long rangeStart; + private final Long rangeEnd; + + // Standard getters: presignedUrl(), rangeStart(), rangeEnd() + // Standard builder methods: builder(), toBuilder() + // Standard Builder class with presignedUrl(), rangeStart(), rangeEnd() setter methods +} +``` + +## FAQ + +### Why don't we implement presigned URL download/GET feature directly on the S3Client? + +Three approaches were considered: + +1. **Dedicated PresignedUrlManager (CHOSEN)**: Separate manager accessed via `s3Client.presignedManager()` + - **Pros**: Clean separation, preserves SDK features, follows v2 patterns + - **Cons**: New API surface for users to learn + +2. **Direct S3Client Integration**: Add presigned URL methods directly to S3Client + - **Pros**: Familiar interface, direct migration path from v1 + - **Cons**: Requires core interceptor changes, complex integration, could confuse users by mixing presigned URL APIs with standard service-generated APIs + +3. **S3Presigner Extension**: Extend existing S3Presigner to execute URLs + - **Pros**: Logical extension of presigner concept + - **Cons**: Breaks current stateless presigner patterns + +**Decision**: Option 1 provides clean separation while preserving SDK benefits and following established v2 utility patterns.cutePresignedGetObject(presignedRequest); + +### Why doesn't PresignedUrlGetObjectRequest extend S3Request? + +While extending S3Request would provide access to RequestOverrideConfiguration, many of these configurations (like credentials provider, signers) are not supported with presigned URL execution. Instead, we use a standalone request with only essential parameters (presignedUrl, rangeStart, rangeEnd). Internally, this gets wrapped in an encapsulated class that extends S3Request for use with ClientHandler. + + +## References + +**GitHub feature requests:** +- [S3 Presigned URL Support #2731](https://github.com/aws/aws-sdk-java-v2/issues/2731) +- [Presigned URL GET Support #181](https://github.com/aws/aws-sdk-java-v2/issues/181) + +**AWS Documentation:** +- [S3 Pre-signed URLs](https://docs.aws.amazon.com/AmazonS3/latest/userguide/presigned-urls.html) + +**SDK Documentation:** +- [AWS SDK for Java v1 implementation](https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/welcome.html) +- [S3 Client architecture patterns](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html) +