Use CRT response file in low level CRT S3AsyncClient for getObject#6289
Use CRT response file in low level CRT S3AsyncClient for getObject#6289alextwoods merged 5 commits intomasterfrom
Conversation
...ests/src/test/java/software/amazon/awssdk/archtests/CodingConventionWithSuppressionTest.java
Show resolved
Hide resolved
There was a problem hiding this comment.
Pull Request Overview
This PR implements support for using CRT's response file feature in the low-level CRT S3AsyncClient for getObject operations. This enhancement allows the CRT library to write response data directly to a file, improving memory usage by avoiding passing data chunks through Java.
Key changes:
- Added a new
getObject(GetObjectRequest, Path)method to DefaultS3CrtAsyncClient that uses CRT's direct file writing - Implemented execution attribute plumbing to pass response file path and options through the request pipeline
- Created a new response transformer that acts as a no-op when CRT handles file writing directly
Reviewed Changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| DefaultS3CrtAsyncClient.java | Added getObject method with Path parameter and execution attribute mapping |
| S3CrtAsyncHttpClient.java | Enhanced to handle response file path and options from execution attributes |
| CrtResponseFileResponseTransformer.java | New no-op response transformer for CRT-managed file responses |
| S3InternalSdkHttpExecutionAttribute.java | Added new execution attributes for response file path and options |
| S3CrtAsyncHttpClientTest.java | Added test coverage for response file functionality |
| CrtResponseFileResponseTransformerTest.java | Comprehensive test suite for the new response transformer |
| CodingConventionWithSuppressionTest.java | Updated suppressions to include new transformer class |
| feature-AWSS3-233f74c.json | Changelog entry for the new feature |
.../src/test/java/software/amazon/awssdk/services/s3/internal/crt/S3CrtAsyncHttpClientTest.java
Outdated
Show resolved
Hide resolved
.../src/test/java/software/amazon/awssdk/services/s3/internal/crt/S3CrtAsyncHttpClientTest.java
Outdated
Show resolved
Hide resolved
...java/software/amazon/awssdk/services/s3/internal/crt/CrtResponseFileResponseTransformer.java
Show resolved
Hide resolved
...3/src/main/java/software/amazon/awssdk/services/s3/internal/crt/DefaultS3CrtAsyncClient.java
Show resolved
Hide resolved
...java/software/amazon/awssdk/services/s3/internal/crt/CrtResponseFileResponseTransformer.java
Show resolved
Hide resolved
|
...java/software/amazon/awssdk/services/s3/internal/crt/CrtResponseFileResponseTransformer.java
Show resolved
Hide resolved
|
This pull request has been closed and the conversation has been locked. Comments on closed PRs are hard for our team to see. If you need more assistance, please open a new issue that references this one. |




Use CRT response file in low level CRT S3AsyncClient for getObject.
Motivation and Context
awslabs/aws-crt-java#825 added support for setting the response file and allowing CRT to write directly to the file. That can improve memory usage by avoiding passing the chunks into java where they are then written to the file.
Note that this change does not change the Transfrer manager behavior - TransferManager will still use the Java based in memory transform.
Modifications
This follows the implementation for requestPath (the inverse feature, allowing paths to be set for putObject, see: #4379).
Testing
New and existing s3 tests, including TM + regression tests.
Screenshots (if appropriate)
Types of changes
Checklist
mvn installsucceedsscripts/new-changescript and following the instructions. Commit the new file created by the script in.changes/next-releasewith your changes.License