Allow streaming upload of unknown content length

[aajtodd]

This has been asked for before.

See:

• Allow non-retryable sources when uploading files to S3 aws-sdk-rust#541 (comment)
• Multipart upload: read directly from file aws-sdk-rust#494 (comment)

[waahm7]

Don't we already support it? Customers will have to provide a size_hint upper bound to figure out the part_size but other than that we should support it because of https://github.com/awslabs/aws-s3-transfer-manager-rs/blob/main/aws-s3-transfer-manager/src/operation/upload/service.rs#L213. We should probably add tests for this use-case, though.

[aajtodd]

No precisely because we force them to provide an upper bound size hint they can't use a stream of unknown content length (e.g. it may be useful to provide a stream that does compression as it's read and therefore they wouldn't know the size without compressing first).

[waahm7]

Yeah, but they can just guess the upper bound size. It's kinda same in aws-c-s3 that the upper bound size of unknown content length stream is part_size * 10000. Unless we are planning to do something like bumping the part_size if we are running out of parts dynamically, then customers will have to provide the upper bound either directly or indirectly due to 10000 parts limit.

If someone sets the size_hint to 30 GB but the stream ends at 1 GB, our code will work fine.

[aajtodd]

Well guessing isn't a great user experience but that could be a workaround for now. We should add an e2e test that does compression on the fly as an example of this working.