[CRITICAL] [lib-storage] Uploading in Safari is in a non-working state

[ffxsam]

Describe the bug

When uploading from Safari directly to S3 (credentials supplied via Cognito), I get this error:

Body Data is unsupported format, expected data to be one of: string | Uint8Array | Buffer | Readable | ReadableStream | Blob

I'm passing a File object like so:

const upload = new Upload({
  client: s3,
  params: {
    Bucket: process.env.VUE_APP_UPLOAD_BUCKET,
    Body: file,
    ContentType: file.type,
    Key: destKey,
    Metadata: metadata,
  }
});

Your environment

SDK version number

@aws-sdk/lib-storage@3.14.0

Is the issue in the browser/Node.js/ReactNative?

Browser

Details of the browser/Node.js/ReactNative version

Safari 14.0.2 and under

Steps to reproduce

(see code snippet above)

Observed behavior

Uploading a file using the Upload class fails with error:
Body Data is unsupported format, expected data to be one of: string | Uint8Array | Buffer | Readable | ReadableStream | Blob

Expected behavior

I would expect Webkit/Safari to follow W3C standards better. 😉 Also, uploads worked totally fine in SDK v2.

Additional context

Pour yourself a coffee and pull up a chair...

So, lib-storage is checking for the existence of data.stream as a function, which is totally legit:

https://github.com/aws/aws-sdk-js-v3/blob/main/lib/lib-storage/src/chunker.ts#L19-L21

File, inheriting from Blob, should implement that. Unfortunately, in Safari, Blob.prototype.stream() does not exist, even though this is a W3C standard drafted way back in May 2019. This wasn't supported in Safari until 14.1.

I'm working around this with the following code, but it's not ideal, as it creates a slight lag for the user as very large files (potentially 70-150MB) are converted to Uint8Array before passing to the Upload constructor:

const fileBuffer = await file.arrayBuffer();
const uploadParams: PutObjectCommandInput = {
  Bucket: process.env.VUE_APP_UPLOAD_BUCKET,
  Body: new Uint8Array(fileBuffer),
  ContentType: file.type,
  Key: destKey,
  Metadata: metadata,
};
const upload = new AWSUpload({
  client: s3,
  params: uploadParams,
});

I don't think there's any ideal workaround at this point. But maybe if this library could handle this scenario instead of putting that on the developer (so they don't wind up wasting time like I did), that would be helpful. Maybe the getChunk function could check for data being an instance of Blob but with data.stream returning undefined, and if so, then call data.arrayBuffer() to parse the data and return it.

If there's a better way I can handle this, I'm open to hearing about it! But if Safari doesn't support Blob streaming, it seems like reading the entire file into memory is the only alternative at this point.

[ffxsam]

Just had a thought:

The v2 AWS SDK didn't have this issue. So I'm not sure what they're doing differently.

[ffxsam]

UPDATE (related issue).

Uploading using this library and Safari 14.1 also breaks (when using the stream) method. The page auto reloads, and I get a strange message at the top of the Safari window: "The webpage was reloaded because a problem occurred." When I read the full arraybuffer into memory and pass a Uint8Array instead, it works fine. So this is something related to Blob#stream again.. maybe.

Here's a reproduction of this upload issue: https://github.com/ffxsam/repro-safari-upload

Clone, run yarn, then yarn serve. You'll obviously need to fill in the proper credentials and S3 bucket name.

[ffxsam]

@AllanZhengYP I think this could be considered critical. Anyone using the v3 SDK with Safari is going to run into this issue, I would imagine.

[ffxsam]

This is becoming more of an issue with our customers. I'd definitely appreciate some help on this! @trivikr @AllanZhengYP @jeskew

[ajredniwja]

@ffxsam apologies for the late reply, I was able to reproduce this previously. Need to investigate more to find an appropriate solution. You are right about that Blob.prototype.stream() missing in safari which originally causes the issue.

[ffxsam]

@ajredniwja Thanks for the response! It looks like Safari 14.1 implements Blob.prototype.stream, but this SDK's use of it causes Safari's upload process to crash and reload the page with the "The webpage was reloaded because a problem occurred" error.

For now, we've added in the old aws-sdk v2 to use S3.upload.

[ajredniwja]

Its unfortunate that you have to downgrade for workaround, I will discuss it with the team and see what a potential solution could be for this.

[ffxsam]

Thanks, Ajwinder!

[ajredniwja]

Hey @ffxsam I was able to run some tests, #2183 (comment) used this code which didn't work in previous version but works in 14.1.1

[ffxsam]

@ajredniwja I don't think that solution will work for me:

const blob = await response.blob();

This looks like it loads the entire blob into memory. We have customers uploading multiple files that could range from 20MB to 100MB. I was already reading the whole file into memory as a workaround, and it bombed some users' browsers.

We really do need streaming to work like it did with aws-sdk's upload method.

[ajredniwja]

I understand, let me try and work with the example you provided.