support unmarshall to convert binary attributes #6599
Description
Describe the feature
It was first introduced here: #2634 (comment)
unmarshall will also not convert binary attributes properly. The DynamoDB stream will provide a base64 encoded presentation of the binary attribute and unmarshall does not base64-decode the value.
But it would be good to add this feature to support unmarshall to convert binary attributes
Reproduction steps:
- Put the JS file in lambda.
import { unmarshall } from "@aws-sdk/util-dynamodb"
export const handler = async (event) => {
for (const record of event.Records) {
if (record.dynamodb?.NewImage) {
const streamItem = record.dynamodb?.NewImage
const item = unmarshall(streamItem)
console.log(item)
}
}
}2: Create a test on lambda, put the following json into test -> test event -> event json and test
{
"Records": [
{
"eventID": "1",
"eventName": "INSERT",
"eventVersion": "1.0",
"eventSource": "aws:dynamodb",
"awsRegion": "us-west-2",
"dynamodb": {
"Keys": {
"Id": {
"N": "101"
}
},
"NewImage": {
"Id": {
"N": "101"
},
"Name": {
"S": "John Doe"
},
"BinaryData": {
"B": "SGVsbG8gV29ybGQ="
}
},
"SequenceNumber": "111",
"SizeBytes": 26,
"StreamViewType": "NEW_AND_OLD_IMAGES"
},
"eventSourceARN": "arn:aws:dynamodb:us-west-2:123456789012:table/YourTableName/stream/2023-05-11T00:00:00.000"
},
{
"eventID": "2",
"eventName": "MODIFY",
"eventVersion": "1.0",
"eventSource": "aws:dynamodb",
"awsRegion": "us-west-2",
"dynamodb": {
"Keys": {
"Id": {
"N": "102"
}
},
"NewImage": {
"Id": {
"N": "102"
},
"Name": {
"S": "Jane Smith"
},
"BinaryData": {
"B": "SGVsbG8gQWdhaW4="
}
},
"SequenceNumber": "222",
"SizeBytes": 28,
"StreamViewType": "NEW_AND_OLD_IMAGES"
},
"eventSourceARN": "arn:aws:dynamodb:us-west-2:123456789012:table/YourTableName/stream/2023-05-11T00:00:00.000"
}
]
}Use Case
unmarshall will also not convert binary attributes properly. The DynamoDB stream will provide a base64 encoded presentation of the binary attribute and unmarshall does not base64-decode the value.
Proposed Solution
I have provided the workaround
import { DynamoDBClient } from "@aws-sdk/client-dynamodb";
import { unmarshall } from "@aws-sdk/util-dynamodb";
const dynamoClient = new DynamoDBClient({ region: "us-west-2" });
function customUnmarshall(item) {
const unmarshalled = unmarshall(item);
for (const [key, value] of Object.entries(unmarshalled)) {
if (item[key] && item[key].B) {
// Decode the base64 string to a UTF-8 string
unmarshalled[key] = Buffer.from(item[key].B, 'base64').toString('utf-8');
}
}
return unmarshalled;
}
export const handler = async (event) => {
for (const record of event.Records) {
if (record.dynamodb?.NewImage) {
const streamItem = record.dynamodb.NewImage;
const item = customUnmarshall(streamItem);
console.log("Unmarshalled item (including decoded binary attributes):", item);
for (const [key, value] of Object.entries(item)) {
if (typeof value === 'string' && item[key] && item[key].B) {
console.log(`Decoded binary attribute ${key}:`, value);
}
}
}
}
};Other Information
No response
Acknowledgements
- I may be able to implement this feature request
- This feature might incur a breaking change
SDK version used
latest
Environment details (OS name and version, etc.)
lambda prod