Skip to content

S3 CRT Path style not working #2500

@FaZeRs

Description

@FaZeRs

Describe the bug

I have set useVirtualAddressing to false when initializing S3CrtClient, but in logs I see that bucket name is prefixed to endpoint. I'm using Ceph S3 and path looks like https://****/{bucket}/{file_path}

Expected Behavior

Aws::Endpoint::DefaultEndpointProvider [139729996415488] Endpoint rules engine evaluated the endpoint: https://{endpoint}/{bucket}

Current Behavior

Aws::Endpoint::DefaultEndpointProvider [139729996415488] Endpoint rules engine evaluated the endpoint: https:/{bucket}.{endpoint}

Reproduction Steps

#include <iostream>
#include <fstream>
#include <aws/core/Aws.h>
#include <aws/core/utils/memory/stl/AWSStringStream.h>
#include <aws/core/utils/logging/CRTLogSystem.h>
#include <aws/s3-crt/S3CrtClient.h>
#include <aws/s3-crt/model/CreateBucketRequest.h>
#include <aws/s3-crt/model/BucketLocationConstraint.h>
#include <aws/s3-crt/model/DeleteBucketRequest.h>
#include <aws/s3-crt/model/PutObjectRequest.h>
#include <aws/s3-crt/model/GetObjectRequest.h>
#include <aws/s3-crt/model/DeleteObjectRequest.h>
#include <aws/core/utils/UUID.h>

static const char ALLOCATION_TAG[] = "s3-crt-demo";

int main(int argc, char* argv[]) {
  Aws::SDKOptions options;
  options.loggingOptions.logLevel = Aws::Utils::Logging::LogLevel::Trace;
  options.loggingOptions.crt_logger_create_fn = []() {
    return Aws::MakeShared<Aws::Utils::Logging::DefaultCRTLogSystem>(ALLOCATION_TAG,
                                                                     Aws::Utils::Logging::LogLevel::Trace);
  };

  Aws::InitAPI(options);
  {
    Aws::String region = Aws::Region::US_EAST_1;
    Aws::String bucket_name = "****";
    Aws::String object_key = "****";
    const double throughput_target_gbps = 5;
    const uint64_t part_size = 8 * 1024 * 1024; // 8 MB.

    Aws::S3Crt::ClientConfiguration config;
    config.endpointOverride = "****";
    config.throughputTargetGbps = throughput_target_gbps;
    config.partSize = part_size;
    config.enableEndpointDiscovery = true;

    Aws::Auth::AWSCredentials credentials("*", "*");

    Aws::S3Crt::S3CrtClient s3_crt_client(credentials, config,
                                          Aws::Client::AWSAuthV4Signer::PayloadSigningPolicy::Never, false,
                                          Aws::S3Crt::US_EAST_1_REGIONAL_ENDPOINT_OPTION::LEGACY);

    Aws::S3Crt::Model::GetObjectRequest request;
    request.WithBucket(bucket_name).WithKey(object_key);
    auto outcome = s3_crt_client.GetObject(request);
    if (outcome.IsSuccess()) {
      std::cout << "Object content: " << outcome.GetResult().GetBody().rdbuf() << std::endl << std::endl;
    } else {
      std::cout << "GetObject error:\n" << outcome.GetError() << std::endl << std::endl;
    }
  }
  Aws::ShutdownAPI(options);

  return 0;
}

Possible Solution

No response

Additional Information/Context

No response

AWS CPP SDK version used

1.11.79

Compiler and Version used

GCC 12.1.0

Operating System and version

Ubuntu 22.04.2

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugThis issue is a bug.needs-triageThis issue or PR still needs to be triaged.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions