Subject: Issue with Transfer Between AWS S3 Buckets: Missing 'bucketName' and 'region' Attributes

[yassirahla]

Hello EDC Connector Community,

I'm currently working on the sample transfer-05-file-transfer-cloud to transfer a file from one AWS S3 bucket to another. However, I'm encountering an error on the consumer side:

Fatal error while sending TransferRequestMessage to http://localhost:8282/protocol. Error details: {"@type":"dspace:TransferError","dspace:code":"400","dspace:reason":["'bucketName' is a mandatory attribute","'region' is a mandatory attribute"]}

The logs indicate that 'bucketName' and 'region' are mandatory attributes, which I have ensured to include in my setup. Despite this, the error persists. Here's an outline of the steps I've followed for the file transfer process:

On the Provider Side

1. Asset Creation: Successfully created an asset with an AmazonS3 dataAddress, including both bucketName and region.

2. Policy Definition Creation: Defined an access policy without issues.

3. Contract Definition Creation: Successfully created a contract definition that selects the asset.

On the Consumer Side:

1. Catalog Request: I made a catalog request to retrieve the catalog without any issue.

2. Contract Negotiation: Initiated and completed contract negotiation successfully.

3. Contract Agreement Retrieval: Retrieved the contract agreement ID without any issue.

4. Data Transfer: At this stage, attempting to initiate the data transfer results in the mentioned error, even though I specify the bucketName and region in the dataDestination.

I've double-checked my configurations and ensured that all required attributes are correctly provided. Here's a snippet of the transfer initiation request:

{
  "@context": {
    "@vocab": "https://w3id.org/edc/v0.0.1/ns/"
  },
  "counterPartyAddress": "http://localhost:8282/protocol",
  "protocol": "dataspace-protocol-http",
  "connectorId": "consumer",
  "assetId": "asset1",
  "contractId": "<contractId>",
  "dataDestination": {
    "region": "eu-west-3",
    "bucketName": "<bucketName>",
    "type": "AmazonS3"
  }
}

Am I missing something in my configuration or is there a known issue related to this scenario? Any suggestions would be greatly appreciated.

Thank you in advance for your help!

[darxsky]

Hi,
I am going to make a file transfer from one bucket S3 to another. Were you able to resolve the issue?
For the asset creation, which properties should I include in "dataAddress"? Could you show me a json example for creating an asset from a bucket S3?

Thank you in advance.

[Moritz72]

You could look at this tutorial in the MXD repo. It is uses a different EDC version with the TractusX-EDC on top of that. But it should still work the same way.

They use

  "dataAddress": {
    "@type": "DataAddress",
    "type": "AmazonS3",
    "keyName": "alice-test-document.txt",
    "region": "us-east-1",
    "bucketName": "alice-bucket"
  }

and for transfer

  "dataDestination": {
    "type": "AmazonS3",
    "keyName": "alice-test-document.txt",
    "bucketName": "bob-bucket",
    "region": "us-east-1",
    "endpointOverride": "http://bob-minio:9000",
    "accessKeyId": "bobawsclient",
    "secretAccessKey": "bobawssecret"
  }

[darxsky]

Thanks for your support!