MIGRATION ISSUE: v3 signed urls with meta data can't be used #7006
Description
Pre-Migration Checklist
- I've read the Migration Guide.
- I've reviewed the upgrading notes and major version differences mentioned in
UPGRADING.md. - I've checked AWS Forums and StackOverflow for similar migration issues.
Which JavaScript Runtime is this issue in?
Node.js (includes AWS Lambda)
AWS Lambda Usage
- Yes, my application is running on AWS Lambda.
- No, my application is not running on AWS Lambda.
Describe the Migration Issue
My backend uses s3 signed urls. Most of them work fine. However, signed URLs that use meta data can't actually be used. Whenever I attempt to use them I get a 403.
Code Comparison
v2
const putParams = {
Bucket: process.env.stage + '-myapp-' + process.env.version + '-images',
Key: imageFileKey,
ContentType: "image/jpg",
ServerSideEncryption: "AES256",
CacheControl: "max-age=86400, private",
Metadata: {
"accountId": accountId,
"deviceName": deviceName,
"identityId": identityId,
"takenAt": imageRecord.takenAt.toString()
},
Expires: 24 * 60 * 60
}
const s3Url = s3.getSignedUrl('putObject', putParams)v3
const putParams = {
Bucket: process.env.stage + '-myapp-' + process.env.version + '-images',
Key: imageFileKey,
ContentType: "image/jpg",
ServerSideEncryption: "AES256",
CacheControl: "max-age=86400, private",
Metadata: {
"accountId": accountId,
"deviceName": deviceName,
"identityId": identityId,
"takenAt": imageRecord.takenAt.toString()
}
}
// Create the command
const putCommand = new PutObjectCommand(putParams);
// Generate presigned URL (expires in 24 hours)
const s3Url = await getSignedUrl(s3Client, putCommand, {
expiresIn: 24 * 60 * 60,
//I've tried both with and without unhoistableHeaders set
unhoistableHeaders: new Set([
"host", //I've tried with and without the host header
"x-amz-meta-accountId", //I've tried lowercasing the headers and with camel case
"x-amz-meta-deviceName",
"x-amz-meta-identityId",
"x-amz-meta-takenAt",
"x-amz-server-side-encryption" //I've tried with and without this
])
});Observed Differences/Errors
When I attempt to use the signed URL from v2, I get a 200 and everything works. When I attempt to use the signed URL from v3, I get a 403.
<?xml version="1.0" encoding="UTF-8"?>
<Error>
<Code>AccessDenied</Code>
<Message>There were headers present in the request which were not signed</Message>
<HeadersNotSigned>x-amz-meta-devicename, x-amz-meta-takenat, x-amz-meta-identityid, x-amz-meta-accountid</HeadersNotSigned>
<RequestId>redacted</RequestId>
<HostId>redacted</HostId>
</Error>Additional Context
Here's my client side code that works with v2 signed urls (but not with v3).
Request request = new Request.Builder()
.url(path)
.addHeader("Content-Type", "image/jpg")
.addHeader("Accept", "text/html,application/json")
.addHeader("x-amz-server-side-encryption", "AES256")
.addHeader("Cache-Control", String.format("max-age=%d, private", 1 * 24 * 60 * 60)) //cache for 1 day
//I've updated the headers to be all lowercase instead of some camelcase and that doesn't make a difference.
.addHeader("x-amz-meta-accountId", settings.getAccountId())
.addHeader("x-amz-meta-deviceName", displayName)
.addHeader("x-amz-meta-identityId", identityId)
.addHeader("x-amz-meta-takenAt", Long.toString(takenAt))
.put(body)
.build();
OkHttpClient client = new OkHttpClient.Builder()
.connectTimeout(30, TimeUnit.SECONDS)
.writeTimeout(60, TimeUnit.SECONDS)
.readTimeout(30, TimeUnit.SECONDS)
.build();
Response response = null;
response = client.newCall(request).execute();
retResponse.status = response.code();
retResponse.body = response.body().string();With v2, it works. With v3, I get a 403.