Files of 2Mb or more are not put with s3

[andrey-ananiev]

Describe the bug

I use this library in Android project. Everything works fine, but if I try to upload a file larger than 2MB, the file fails to send with an HttpException: java.net.SocketTimeoutException: timeout

Regression Issue

• Select this option if this issue appears to be a regression.

Expected behavior

File must uploaded successfully.

Current behavior

I get HttpException java.net.SocketTimeoutException: timeout

Steps to Reproduce

Dependencies:

implementation("aws.sdk.kotlin:s3:1.5.19")

Initializing the s3 client:

val s3Client = S3Client {
            endpointUrl = Url.parse(YandexCloudConfig.host)
            region = YandexCloudConfig.regionName
            credentialsProvider = StaticCredentialsProvider {
                accessKeyId = YandexCloudConfig.idKey
                secretAccessKey = YandexCloudConfig.secretKey
            }
        }

File data loading function:

private suspend fun uploadSimple(
        fileName: String,
        data: ByteArray,
        userEmail: String
    ): Boolean {
        return runCatching {
            s3Client.putObject(PutObjectRequest {
                bucket = YandexCloudConfig.bucketName
                key = "$userEmail/$fileName"
                body = ByteStream.fromBytes(data)
                contentType = "application/octet-stream"
                storageClass = StorageClass.Standard
                acl = ObjectCannedAcl.Private
            })
            Logger.i("File $fileName (${data.size} bytes) uploaded successfully to $userEmail")
            true
        }.onFailure { e ->
            handleUploadError(fileName, userEmail, e, "Simple upload failed")
        }.getOrDefault(false)
    }

Possible Solution

I tried to configure httpEngine but it has no limit on the upload file size. Files of (2MB - 1B) size are uploaded perfectly. I don't understand why larger files don't upload. I was unable to replace the engine.

Context

No response

AWS SDK for Kotlin version

1.5.19

Platform (JVM/JS/Native)

Android

Operating system and version

Android 14

[lauzadis]

Was this working on a previous version of the SDK for you, or never working? By default, S3 has aws-chunked enabled for uploads larger than 1MB. That sounds like it might be causing your problem, you can try disabling it by configuring enableAwsChunked = false on your S3 client. If that fixes your issue, we should investigate why aws-chunked isn't working for you. Could you send some TRACE-level logs of the operation?

[lauzadis]

I saw references to YandexCloud in the code snippet. If you are using a third-party API which emulates S3, it may not support aws-chunked. Disabling it on the client should solve your issue in that case. Let me know!

[andrey-ananiev]

Thank you for your answer. "enableAwsChunked = false" did not solve the problem. But I have made progress in solving it!
Any files are sent successfully if I replace endpointUrl "https://storage.yandexcloud.net" with "http://storage.yandexcloud.net". Apparently the problem is with the SSL certificate?

[lauzadis]

Thanks for the information. It sounds like this may be an issue with your Android device configuration or your 3rd party cloud provider. Are you able to reproduce the issue when using Amazon S3? Either way, TRACE level logging could help us identify the issue

[ianbotsf]

We cannot diagnose this issue without additional logs or a reproduction on Amazon S3. Resolving this issue for now but please feel free to re-open if additional information becomes available.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.