Sending CompleteMultipartUpload S3 POST Request To Wrong Host

[kevingale]

We are using Cyberduck with a custom connection profile to help test a S3 API compatible service we are developing.

If we connect to the root DNS of our service (locals3.com), choose the bucket, choose the destination folder and then upload a large file, Cyberduck performs a multipart upload successfully.

However, if we connect to our bucket DNS (mybucket.locals3.com), choose the destination folder and then upload a large file, Cyberduck uploads all of the parts to our service but then fails when sending the CompleteMultipartUpload POST.

When I look in the Cyberduck debug log I see that it is trying to POST to mybucket.mybucket.locals3.com:

2023-12-11 16:51:00,366 [H4yrFdYN-transfer-2] DEBUG ch.cyberduck.core.s3.RequestEntityRestStorageService - Set endpoint to mybucket.locals3.com
2023-12-11 16:51:00,366 [H4yrFdYN-transfer-2] DEBUG org.jets3t.service.Jets3tProperties - s3service.disable-dns-buckets=false
2023-12-11 16:51:00,366 [H4yrFdYN-transfer-2] DEBUG ch.cyberduck.core.s3.RequestEntityRestStorageService - Set hostname to mybucket.mybucket.locals3.com
2023-12-11 16:51:00,367 [H4yrFdYN-transfer-2] DEBUG org.jets3t.service.impl.rest.httpclient.RestStorageService - Added request parameter: uploadId=b0306123168808bd2c90bd5c518936e6
2023-12-11 16:51:00,367 [H4yrFdYN-transfer-2] DEBUG ch.cyberduck.core.s3.RequestEntityRestStorageService - Set URL to https://mybucket.mybucket.locals3.com:443/AutoUpdate/ApplicationBackgrounds/todelete%20copy/a/b/%E6%97%A5%E6%9C%AC%E8%AA%9E/YearbookCD.zip?uploadId=b0306123168808bd2c90bd5c518936e6
2023-12-11 16:51:00,367 [H4yrFdYN-transfer-2] DEBUG org.jets3t.service.impl.rest.httpclient.RestStorageService - Performing POST request for 'https://mybucket.mybucket.locals3.com:443/AutoUpdate/ApplicationBackgrounds/todelete%20copy/a/b/%E6%97%A5%E6%9C%AC%E8%AA%9E/YearbookCD.zip?uploadId=b0306123168808bd2c90bd5c518936e6', expecting response codes: [200]

(the entire debug log is attached)

The custom connection profile we are using is:

<plist version="1.0">
    <dict>
        <key>Protocol</key>
        <string>s3</string>
        <key>Vendor</key>
        <string>Me</string>
        <key>Description</key>
        <string>Nearly S3</string>
        <key>Default Hostname</key>
        <string>locals3.com</string>
        <key>Authorization</key>
		<string>AWS4HMACSHA256</string>
        <key>Regions</key>
        <array>
            <string>us-east-1</string>
        </array>
    </dict>
</plist>

so... For some reason, Cyberduck doesn't think the bucket is part of host name so adds it to the CompleteMultipartUpload POST. I've checked over our code and custom connection profile but cannot find anything wrong.

Can anybody help us narrow down the problem?

Thanks.

Cyberduck MultipartUpload Failure Log.log

[dkocher]

There is some fuzzy logic to determine if we want to use virtual host style requests by prefixing S3 endpoint host from the connection profile with the bucket name. As a workaround for the bug, try to use path-style requests instead by setting in the profile:

        <key>Properties</key>
        <array>
            <string>s3.bucket.virtualhost.disable=true</string>
        </array>

[kevingale]

Hi.
The workaround fixes the problem.
Thanks.

[dkocher]

Opened #15452 for the bug.