Danube Persistent Storage

A storage service that provides durable message persistence for the Danube messaging system through multiple storage backend implementations.

Usage

TBD

Storage Backend Implementation

You can implement additional storage backends in two ways:

1. Add a new implementation in this repository following the patterns used for MinIO and AWS S3 backends
2. Create a separate repository implementing the storage interface defined in proto/DanubeManagedStorage.proto using your preferred programming language

The gRPC interface makes it easy to implement compatible storage backends in any language while maintaining full compatibility with the Danube messaging system.

Contribution

Please feel free to contribute or report any issues you encountered.

Backend Storage Implementations:

• AWS S3 Storage
• Azure Blob Storage
• GCP Cloud Storage
• MinIO
• Longhorn

Use latest DanubeManagedStorage.proto file

Make sure the proto/DanubeManagedStorage.proto is the latest from Danube project.

If not replace the file and add at the top of the file

option go_package = "github.com/danube-messaging/danube-storage/danube-s3-storage/proto";

right after the package managed_storage;

In order to generate the Go grpc code you need the following packages installed:

go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest

And generate the Go code from the proto file:

protoc --proto_path=./proto --go_out=./proto --go-grpc_out=./proto --go_opt=paths=source_relative      --go-grpc_opt=paths=source_relative proto/DanubeManagedStorage.proto