Cross Region Access not working for public S3 buckets without authentication.

[castanhas98]

Describe the bug

We are migrating some workflows from V1 to V2 of the AWS SDK.

One of them concerns accessing a public S3 bucket (samples.dremio.com) without any credentials, using a client set up for a region different from the one the bucket is in (the bucket is in us-west-2).

EDIT: This is an issue because without setting the S3Client with the correct bucket region, we cannot access the bucket. And without being able to access the bucket, we cannot find the correct bucket region.

Say we set up the clients as follows:

• V1: Anonymous credentials provider and region set to us-east-1.
• V2: Anonymous credentials provider, region set to us-east-1, and cross region access enabled.

Calling getBucketLocation on the samples.dremio.com bucket, which is in us-west-2:

• Works for the V1 client.
• Does not work for the V2 client.
• Works for a V2 client if the region is set to the same region as the bucket, us-west-2.

We would think it should be possible to access the bucket across regions if the V1 client allows for it.

As an additional data point, it becomes possible to access the bucket from a V2 client with the following configuration:

• Valid credentials for the samples.dremio.com account.
• Region set to us-east-1.
• Cross region access enabled.

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

A simple example of this is calling getBucketLocation on the bucket, and I would expect for the bucket region to be retrieved correctly, which is what happens with the V1 client.

Current Behavior

I get an exception like the following:

Error connecting to S3: Access Denied (Service: S3, Status Code: 403, Request ID: 7B3DJYB7K8H8SM4P, Extended Request ID: len4RcWY3YiExKPhWAmqJDTa+pJ9KML25a7RCUwunAN734RTXgrIXFn6gol+sHtZW/kUFvWk7GY=) (SDK Attempt Count: 1)

Reproduction Steps

V1 Client Version: 1.12.750
V2 Client Version: 2.30.27

With V2 client, not working:

    try (S3Client s3 =
        S3Client.builder()
            .credentialsProvider(AnonymousCredentialsProvider.create())
            .crossRegionAccessEnabled(true)
            .region(Region.US_EAST_1)
            .build()) {
      System.out.println(
          "Region for samples.dremio.com is: "
              + s3.getBucketLocation(builder -> builder.bucket("samples.dremio.com"))
                  .locationConstraintAsString());
    } catch (Exception e) {
      System.err.println("Error connecting to S3: " + e.getMessage());
    }

With V2 client, working, but setting the correct region (bucket is in US_WEST_2). Output is
Region for samples.dremio.com is: us-west-2:

    try (S3Client s3 =
        S3Client.builder()
            .credentialsProvider(AnonymousCredentialsProvider.create())
            .region(Region.US_WEST_2)
            .build()) {
      System.out.println(
          "Region for samples.dremio.com is: "
              + s3.getBucketLocation(builder -> builder.bucket("samples.dremio.com"))
                  .locationConstraintAsString());
    } catch (Exception e) {
      System.err.println("Error connecting to S3: " + e.getMessage());
    }

With V1 Client, working. Output is Region of samples.dremio.com is: us-west-2:

    final AmazonS3 s3Client =
        AmazonS3ClientBuilder.standard()
            .withCredentials(new AWSStaticCredentialsProvider(new AnonymousAWSCredentials()))
            .withRegion("us-east-1")
            .build();

    System.out.println(
        "Region of samples.dremio.com is: "
            + s3Client.getBucketLocation("samples.dremio.com").toString());

    s3Client.shutdown();

Possible Solution

No response

Additional Information/Context

No response

AWS Java SDK version used

2.30.27

JDK version used

21.0.5

Operating System and version

macOS Sequoia 15.3.2

[steveloughran]

what happens if you try a 2.29.52 instead; 2.30 made some changes incompatible with third party stores

[castanhas98]

Thanks for the quick reply! 2.29.52 still fails. The bucket is hosted in Amazon S3, not in a third-party store, so I am not sure if this would be affected.

[bhoradc]

Hello @castanhas98,

Thanks for reporting the issue. Kindly note that the getBucketLocation API is considered a legacy API in S3, and it doesn't provide the necessary information to the Java V2 SDK to automatically handle cross-region access. The headBucket API is designed to handle cross-region access correctly.

The AWS S3 API documentation explicitly states that the GetBucketLocation operation is provided for backward compatibility only, and that the HeadBucket operation should be used instead for determining a bucket's region.

Using headBucket first and then getBucketLocation may work as a workaround, but the recommended approach is to use headBucket directly whenever you need to determine a bucket's location.

Regards,
Chaitanya

[castanhas98]

Hello @bhoradc,

Thank you for your reply! I missed the backward compatibility notice in the documentation.

That being said, I confirm that headBucket works perfectly for my use case.

Thanks again, I'll close the issue!

Best,
Francisco

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.