MIGRATION ISSUE: S3 ListObjectsV2 returns undefined instead of empty array

[vobarian]

Pre-Migration Checklist

• I've read the Migration Guide.
• I've reviewed the upgrading notes and major version differences mentioned in UPGRADING.md.
• I've checked AWS Forums and StackOverflow for similar migration issues.

Which JavaScript Runtime is this issue in?

Node.js (includes AWS Lambda)

AWS Lambda Usage

• Yes, my application is running on AWS Lambda.
• No, my application is not running on AWS Lambda.

Describe the Migration Issue

The S3 ListObjectsV2 result is different when no objects exist for the given prefix. In v2, Contents and CommonPrefixes are empty arrays. In v3, these properties are not present at all in ListObjectsV2CommandOutput, and thus evaluate to undefined. Additionally, in v3, if Delimiter is not passed in the input, CommonPrefixes will be absent in the output, whereas v2 included it as an empty array.

This caused us a production outage. With v2, our code checked for Contents being falsy, but because we could never find a scenario where that actually occurred, we considered it an unexpected error. With v3, the behavior changed unexpectedly.

In my opinion, this is a bad, or at least unconventional API design choice. In most APIs (not AWS specific), if you are searching for any number of matches and getting an array back, and the thing is not found, you get an empty array, rather than the property disappearing. An example is JavaScript's Array#filter method.

I checked all the following documentation and did not see this important difference covered anywhere:

• https://github.com/aws/aws-sdk-js-v3/blob/main/UPGRADING.md
• https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/javascript_s3_code_examples.html
• https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/migrate-s3.html
• https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/client/s3/command/ListObjectsV2Command/
• https://github.com/aws/aws-sdk-js-v3/blob/main/supplemental-docs/CLIENTS.md

I acknowledge that the documentation says the Contents can be undefined, but it does not say under what circumstances exactly that can occur. In v2, it never occurred. Since the SDK is very sloppy about what can actually be undefined, as often things that are declared | undefined actually never can be (and previously this was the case for this operation), it was an unexpected change.

Code Comparison

V2:

  const {Contents} = await client.listObjectsV2(params).promise();
  if (!Contents) {
    throw new Error(
      `Error getting object matching params: ${JSON.stringify(params)}`
    );
  }
  return Contents;

V3:

  const { Contents } = await client.send(new ListObjectsV2Command(params));
  return Contents ?? [];

Observed Differences/Errors

Contents is not present in the output when no objects matching the prefix exist.

Additional Context

No response

[aBurmeseDev]

Hi @vobarian - thanks for reaching out and sorry to hear about inconveniences caused.

The '| undefined' in our type definitions is an intentional design choice for forward compatibility. When AWS services update their APIs to make previously required parameters optional, this type pattern ensures existing client applications continue to work without breaking. This design anticipates potential API changes where requirements become less strict over time.

We actually have it documented here and it appears that it cannot be found easily.

This also allows more flexibility when it comes to AWS service APIs changing over time without needing to update your SDK client version, and encourages defensive checking of API responses. If you acknowledge the risks of not runtime-checking response object structures and would like to skip that, there is a @smithy/types type transform that can be applied to AWS SDK clients.

For usage, see https://github.com/smithy-lang/smithy-typescript/tree/main/packages/types#scenario-removing--undefined-from-input-and-output-structures.

Hope that helps,
John

[vobarian]

Thanks for your reply. The issue I wanted to highlight is primarily not the type declaration, but the behavior change between v2 and v3 SDKs. Both have the Contents property typed with the undefined union, but when the S3 list operation finds no objects, in v2 this was returned as an empty array, and in v3 it is returned as undefined. Hence, when upgrading from v2 to v3, the change in actual behavior was unexpected. I am proposing that the v2 behavior is better and should be restored if possible, since it is generally a more common, safer, and more user-friendly design for an operation to return an empty list when there are no items found.

[aBurmeseDev]

Thank you for your feedback, I'll share pass it to the team for future considerations. As stated above, this's an intentional design decision in v3 for the reason outlined.

The '| undefined' in our type definitions is an intentional design choice for forward compatibility. When AWS services update their APIs to make previously required parameters optional, this type pattern ensures existing client applications continue to work without breaking. This design anticipates potential API changes where requirements become less strict over time.

Closing issue for now.

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.