You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: services/galexie/DEVELOPER_GUIDE.md
+15-5Lines changed: 15 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,4 +1,3 @@
1
-
2
1
# Galexie Developer Guide
3
2
Galexie is a tool to export Stellar network transaction data to cloud storage in a way that is easy to access.
4
3
@@ -15,7 +14,7 @@ The goal of Galexie is to build an easy-to-use tool to export Stellar network le
15
14
To achieve its goals, Galexie uses the following architecture, which consists of the 3 main components:
16
15
- Captive-core to extract raw transaction metadata from the Stellar Network.
17
16
- Export manager to bundles and organizes the ledgers to get them ready for export.
18
-
- The cloud storage plugin writes to the cloud storage. This is specific to the type of cloud storage, GCS in this case.
17
+
- The cloud storage plugin writes to the cloud storage. This supports multiple storage backends including Google Cloud Storage (GCS), Amazon S3, and S3-compatible storage services.
19
18
20
19
21
20

@@ -29,8 +28,19 @@ To achieve its goals, Galexie uses the following architecture, which consists of
29
28
- Objects are compressed before uploading using the [zstd](http://facebook.github.io/zstd/) (zstandard) compression algorithm to optimize network usage and storage needs.
30
29
31
30
## Data Storage
32
-
- An example implementation of `DataStore` for GCS, Google Cloud Storage. This plugin is located in the [support](https://github.com/stellar/go/tree/master/support/datastore) package.
33
-
- Galexie currently implements the interface only for Google Cloud Storage (GCS). The [GCS plugin](https://github.com/stellar/go/blob/master/support/datastore/gcs_datastore.go) uses GCS-specific behaviors like conditional puts, automatic retry, metadata, and CRC checksum.
31
+
Galexie implements a pluggable data storage architecture through the `DataStore` interface. This plugin is located in the [support](https://github.com/stellar/go/tree/master/support/datastore) package and supports multiple storage backends:
32
+
33
+
### Google Cloud Storage (GCS)
34
+
- The [GCS plugin](https://github.com/stellar/go/blob/master/support/datastore/gcs_datastore.go) uses GCS-specific behaviors like conditional puts, automatic retry, metadata, and CRC checksum validation.
35
+
36
+
### Amazon S3 and S3-Compatible Storage
37
+
- The [S3 plugin](https://github.com/stellar/go/blob/master/support/datastore/s3_datastore.go) supports AWS S3 and S3-compatible storage services such as:
38
+
- Amazon S3
39
+
- Cloudflare R2
40
+
- MinIO
41
+
- DigitalOcean Spaces
42
+
- Any other S3-compatible object storage
43
+
- Features include conditional puts using `If-None-Match` headers, metadata storage, CRC32C checksum validation, and automatic retry mechanisms.
34
44
35
45
## Build and Run using Docker
36
46
The Dockerfile contains all the necessary dependencies (e.g., Stellar-core) required to run Galexie.
@@ -59,7 +69,7 @@ $ GALEXIE_INTEGRATION_TESTS_ENABLED=true go test -v -race -run TestGalexieTestSu
59
69
```
60
70
61
71
## Adding support for a new storage type
62
-
Support for different data storage types are encapsulated as 'plugins', which are implementation of `DataStore` interface in a go package. To add a data storage plugin based on a new storage type (e.g. AWS S3), follow these steps:
72
+
Support for different data storage types are encapsulated as 'plugins', which are implementation of `DataStore` interface in a go package. To add a data storage plugin based on a new storage type, follow these steps:
63
73
64
74
- A data storage plugin must implement the [DataStore](https://github.com/stellar/go/blob/master/support/datastore/datastore.go) interface.
65
75
- Add support for new datastore-specific features. Implement any datastore-specific custom logic. Different datastores have different ways of handling
0 commit comments