refactor(sdkv3): migrate S3 Client

[Hweinstock]

Problem

This client is tightly coupled to some implementations, making this migration more difficult than usual. Additionally, v3 handles streams differently than v2. Also, some v2 features in the SDK are completely redone. For example, pre-signing a url: https://aws.amazon.com/blogs/developer/generate-presigned-url-modular-aws-sdk-javascript/.

Solution

• refactor or delete most of the tests in https://github.com/aws/aws-toolkit-vscode/blob/1dbaab43d3533cc35b8bcd705860c7199bc98264/packages/core/src/test/shared/clients/defaultS3Client.test.ts
  • Most of them don't add real value. They simply assert the SDK was called, or that errors propagate. These unit tests should be reserved for testing the logic built on top of these requests, not the requests themselves.
  • For tests that do add value, refactor to avoid stubbing the SDK (Ex. listFiles refactor).
• Throw an explicit error when we reach a code path that is not web-supported.
• Migrate to new pre-signing logic, described here: https://aws.amazon.com/blogs/developer/generate-presigned-url-modular-aws-sdk-javascript/
• Simplify list buckets since region is now added in response by default.
• Unify Bucket type definitions. There originally was the S3 Bucket type exported by the SDK, our own Bucket, and DefaultBucket. Bucket was derived from DefaultBucket, but this made reading type signatures ambiguous at first glance. These are all now under S3Bucket.

Verification

To verify this migration didn't break anything I manually tested the following through the toolkit:

• Create buckets (in console and toolkit) in three different regions, ensure they display in proper regions.
• Create folders (nested as well).
• Upload a file, view and edit the file through explorer.
• Download the file, and verify my changes were saved through S3.
• Generate a presigned url for objects in my bucket.
• Delete a file, bucket, folder.
• Verify copied ARN, name, and Path are correct through selection.

Future Work

• Downloading and uploading a file now supports web streams, which means that GetObject response body is now a StreamingBlobPayloadOutputTypes from @smithy/types. This is a general type for web/node streams, therefore we must cast to Node's Readable type. Ideally we would support working with the abstract stream directly and avoiding the cast. The approach is outlined here: https://docs.aws.amazon.com/code-library/latest/ug/s3_example_s3_Scenario_UsingLargeFiles_section.html.
• E2E tests for uploading and downloading a file.
• Remove mocks from https://github.com/aws/aws-toolkit-vscode/blob/1a313e4029930e1da48558f6c7b84f48036672e9/packages/core/src/test/awsService/s3/commands/uploadFile.test.ts. Mocked tests are extremely expensive to maintain because they inherently rely on implementation details.
• There are three listBuckets-related methods that all do slightly different things. These should live in a single method, listBuckets and it should return an AsyncCollection for proper pagination support.
• Reduce stub usage in the tests to avoid coupling to implementation. The entire test suite for the wrapper stubbed S3 v2 directly, making it very difficult to refactor the tests.

---

• Treat all work as PUBLIC. Private feature/x branches will not be squash-merged at release time.
• Your code changes must meet the guidelines in CONTRIBUTING.md.
• License: I confirm that my contribution is made under the terms of the Apache 2.0 license.

[Hweinstock]

/runIntegrationTests

[Hweinstock]

Integ test failing on master: 99a555a

Duplicate code unrelated to these changes. Refactoring explorer nodes lead to it seeing duplication in the explorer node boilerplate.

[justinmk3]

Wait until after today's release?