Extending SimpleS3

Introduction

SimpleS3 is designed to be easy to extend with new functions. If you want to contribute support for a new function, these are the places you should change:

• Request & RequestMarshal
• Response & ResponseMarshal
• Operations
• Client

In order to contribute, create a fork of this library and make the changes in your own version. Once you are finished, simply create a pull request so that it can be reviewed. For more information on how GitHub collaboration works, see their collaboration guidelines.

SimpleS3.Utility.S3Template

This is a utility that helps you generate the boilerplate code required to extend SimpleS3. You simply open Api.txt in the project, enter the name and type of the S3 function and run the utility. By default, it outputs the finished code files to C:\Temp

The Api.txt file looks like this:

GetBucketAccelerateConfiguration,bucket
GetObject,object

The S3 function name comes from the Amazon S3 documentation and the type can either be bucket or object.

Once you have generated files, you can simply move them into the SimpleS3.Core project. Move the whole folder tree and the files are moved to the right place. The folder structure should look something like this

./Internals/Marshallers/Requests/Objects/GetObjectRequestMarshal.cs
./Internals/Marshallers/Responses/Objects/GetObjectResponseMarshal.cs
./Network/Requests/Objects/GetObjectRequest.cs
./Network/Responses/Objects/GetObjectResponse.cs

Requests

Each S3 function needs something that maps from user input to something we can send to the Amazon S3 server. In our case, this is a Request that holds user input and a RequestMarshal that translates it from user input to on-the-wire format.

Here is an example of a request:

public class GetObjectRequest : BaseRequest, IHasRequestPayer
{
    internal GetObjectRequest() : base(HttpMethod.GET)
    {
    }

    public GetObjectRequest(string bucketName, string objectKey) : this()
    {
        BucketName = bucketName;
        ObjectKey = objectKey;
    }

    public Payer RequestPayer { get; set; }
}

Notice how it inherits from BaseRequest and IHasRequestPayer. All requests need to inherit from BaseRequest, but the interesting part is the IHasRequestPayer interface. SimpleS3 comes with a lot of built-in interfaces that helps with keeping a common name for common properties across requests, so if your request has support for RequestPayer, simply inherit from the IHasRequestPayer interface and you are good to go. A really nice feature is that SimpleS3 automatically detect that you inherit from IHasRequestPayer and maps it, so you don't need to do anything. However, let's say you implement a custom property, then you need a request marshaller.

Request Marshaller

A request marshaller translates your Request to a HTTP request. HTTP works with headers and parameters, so we need to map your request to one or both of those.

Note: If your request only inherts from one of the IHasXXXXX interfaces and have no other properties, you don't need to create a custom request marshal.

Here is an example:

internal class GetObjectRequestMarshal : IRequestMarshal<GetObjectRequest>
{
    public Stream MarshalRequest(GetObjectRequestrequest, IConfig config)
    {
        request.SetHeader(AmzHeaders.XAmzRequestPayer, hasRequestPayer.RequestPayer == Payer.Requester ? "requester" : null);
        return null;
    }
}

As you can see, it sets a header to "Requester" or nothing depending on the value of the RequestPayer value. We return null since we do not have a stream on the request that we need to send.

Response

We need a response that contains the data that comes back from Amazon S3 after we have sent a request. Like requests, we can also inherit from IHasXXXXX interfaces, and if we don't have any custom properties, we don't need to make a response marshaller as well.

Here is an example:

public class GetObjectResponse : BaseResponse, IHasContent, IHasRequestCharged
{
    public ContentReader Content { get; internal set; }
    public bool RequestCharged { get; internal set; }
}

All responses need to inherit from BaseResponse. Notice the IHasContent and IHasRequestCharged interfaces. The IHasContent makes sure we have the Content property that contains a stream, and the IHasRequestCharged makes sure we have the RequestCharged property, that is a boolean to indicate if the request was charged to the requester or not (this is a feature of S3).

In order to map from HTTP to the request above, we need a response marshaller.

Response marshaller

A response marshaller maps parameters, headers and body content to an S3 response. They look like this:

internal class GetObjectResponseMarshal : IResponseMarshal<GetObjectRequest, GetObjectResponse>
{
    public void MarshalResponse(IConfig config, GetObjectRequest request, GetObjectResponse response, IDictionary<string, string> headers, Stream responseStream)
    {
        response.RequestCharged = headers.ContainsKey("x-amz-request-charged");
        response.Content = new ContentReader(responseStream);
    }
}

In this response marshaller, we set the RequestCharged property if the HTTP response contains a header called "x-amz-request-charged" and we set the Content property to the body response steam.

Operations

Once you have a request + marshaller and a response + marshaller, then we need to add the function to the SimpleS3 API. If your request is object based, then you add it to ObjectOperations. If it is bucket based, then you add it to BucketOperations. Operations is the lower API layer in SimpleS3 and all functions must be mapped on this level.

It is really simple, it looks like this:

public interface IObjectOperations
{
    Task<GetObjectResponse> GetObjectAsync(GetObjectRequest request, CancellationToken token = default);
}

It simply takes in the request and gives back a response. Remember to take in a CancellationToken as well.

Client

The clients are part of the higher-level APIs and make it easier for users to send a request without a lot of boilerplate code. It looks like this:

public class S3ObjectClient : IObjectClient
{
    public Task<GetObjectResponse> GetObjectAsync(string bucketName, string objectKey, Action<GetObjectRequest> config = null, CancellationToken token = default)
    {
        GetObjectRequest req = new GetObjectRequest(bucketName, objectKey);
        config?.Invoke(req);

        return ObjectOperations.GetObjectAsync(req, token);
    }
}

Finishing up

That's it, you have now extended SimpleS3 with a new S3 function.