Skip to content

Commit 451d07d

Browse files
committed
Create 0014-storage-clis-for-blobstore-operations.md
1 parent 8da78dd commit 451d07d

File tree

1 file changed

+66
-0
lines changed

1 file changed

+66
-0
lines changed
Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
# ADR: Introduce Storage CLIs for Blobstore Operations
2+
3+
## Status
4+
5+
🔄 **Under Discussion** – This ADR proposes a shared direction for replacing fog-based blobstore implementations. It has not yet been accepted.
6+
7+
| Provider | Status | Notes |
8+
|----------|--------------------------|---------------------------------------------------------------------------------------------------------|
9+
| Azure | 🚧 PoC in Progress | [PoC](https://github.com/cloudfoundry/cloud_controller_ng/pull/4397) done with `bosh-azure-storage-cli` |
10+
| AWS | 🧭 Open for Contribution | |
11+
| GCP | 🧭 Open for Contribution | |
12+
| Alicloud | 🧭 Open for Contribution | |
13+
14+
15+
## Context
16+
17+
Cloud Controller uses the fog gem family to interface with blobstores like Azure, AWS, GCP, and Alibaba Cloud.
18+
These Ruby gems are largely unmaintained, introducing risks such as:
19+
* Dependency on deprecated SDKs (e.g., Azure SDK for Ruby)
20+
* Blocking Ruby version upgrades
21+
* Potential for unpatched CVEs
22+
23+
Bosh faces similar issues, as it is also written in Ruby and must interact with blobstores. To address this, BOSH introduced standalone CLI tools (e.g., `bosh-azure-storage-cli`, `bosh-s3cli`) which shell out from Ruby to handle all blobstore operations:
24+
- https://github.com/cloudfoundry/bosh-azure-storage-cli
25+
- https://github.com/cloudfoundry/bosh-s3cli
26+
- https://github.com/cloudfoundry/bosh-gcscli
27+
- https://github.com/cloudfoundry/bosh-ali-storage-cli
28+
29+
This approach decouples core logic from Ruby gems and has proven to be robust in production.
30+
These CLIs are implemented in Go and use the respective provider SDKs.
31+
All BOSH storage CLIs currently implement a common interface with the following commands: `put`, `get`, `delete`, `exists`, and `sign`.
32+
33+
A [PoC](https://github.com/cloudfoundry/cloud_controller_ng/pull/4397) has shown that `bosh-azure-storage-cli` can be successfully used in Cloud Controller to push apps.
34+
35+
## Decision
36+
37+
Cloud Controller will introduce support for CLI-based blobstore clients, starting with Azure.
38+
Specifically, we will:
39+
* Add a new blobstore client using `bosh-azure-storage-cli`
40+
* Shell out from Cloud Controller to perform blobstore operations
41+
* Allow opt-in via configuration parameter
42+
* Keep the `fog-azure-rm` backend during the transition
43+
44+
The `bosh-azure-storage-cli` needs to be extended with the following commands:
45+
* `copy`
46+
* `list`
47+
* `properties`
48+
* `ensure-bucket-exists`
49+
50+
Other providers (AWS, GCP, Alibaba) will follow. Each will require equivalent blobstore clients and support for the above commands.
51+
This will eventually allow us to remove all fog-related gems from Cloud Controller.
52+
53+
## Consequences
54+
55+
* Enables removing of `fog-azure-rm` and all other fog related gems
56+
* Reduces long-term maintenance burden and potential security issues
57+
* Allows providers to be migrated independently
58+
* Increases initial complexity during migration phase
59+
* With more consumers, interface changes in the BOSH storage CLIs may require more coordination
60+
61+
## Alternatives Considered
62+
63+
* Replace fog with newer Ruby gems → Maintenance risk persists
64+
* Implement own blobstore client in Ruby → High development and testing effort
65+
66+

0 commit comments

Comments
 (0)