Skip to content

Commit 14184a7

Browse files
kaovilaiclaude
andcommitted
Add Azure and GCP CloudStorage provider implementations
- Add Azure Blob Storage support with container operations - Add Google Cloud Storage support with bucket operations - Update CloudStorage API to support azure and gcp providers - Add config field to CloudStorage spec for provider-specific options - Add comprehensive unit tests for both Azure and GCP implementations - Update CRD manifests with new provider options 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
1 parent 962cc53 commit 14184a7

File tree

13 files changed

+2106
-135
lines changed

13 files changed

+2106
-135
lines changed

api/v1alpha1/cloudstorage_types.go

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,9 @@ type CloudStorageSpec struct {
4444
// provider is the provider of the cloud storage
4545
// +kubebuilder:validation:Enum=aws;azure;gcp
4646
Provider CloudStorageProvider `json:"provider"`
47+
// config is provider-specific configuration options
48+
// +kubebuilder:validation:Optional
49+
Config map[string]string `json:"config,omitempty"`
4750

4851
// https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/storage/[email protected]#section-readme
4952
// azure blob primary endpoint

api/v1alpha1/zz_generated.deepcopy.go

Lines changed: 7 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

bundle/manifests/oadp.openshift.io_cloudstorages.yaml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,11 @@ spec:
3939
type: object
4040
spec:
4141
properties:
42+
config:
43+
additionalProperties:
44+
type: string
45+
description: config is provider-specific configuration options
46+
type: object
4247
creationSecret:
4348
description: creationSecret is the secret that is needed to be used
4449
while creating the bucket.

config/crd/bases/oadp.openshift.io_cloudstorages.yaml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,11 @@ spec:
3939
type: object
4040
spec:
4141
properties:
42+
config:
43+
additionalProperties:
44+
type: string
45+
description: config is provider-specific configuration options
46+
type: object
4247
creationSecret:
4348
description: creationSecret is the secret that is needed to be used
4449
while creating the bucket.

docs/standardized-flow-implementation.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -539,13 +539,15 @@ This enhancement eliminates the need to manually configure the region in the sec
539539
### Azure Workload Identity Implementation
540540

541541
#### Azure Environment Variables
542+
542543
```go
543544
ClientIDEnvKey = "AZURE_CLIENT_ID" // Azure managed identity client ID
544545
TenantIDEnvKey = "AZURE_TENANT_ID" // Azure tenant ID
545546
SubscriptionIDEnvKey = "AZURE_SUBSCRIPTION_ID" // Azure subscription ID
546547
```
547548

548549
#### Azure Prerequisites
550+
549551
1. **OpenShift cluster with Azure Workload Identity enabled**
550552
- Cluster must be installed with manual credentials mode
551553
- Reference: [Installing a cluster on Azure with short-term credentials](https://docs.redhat.com/en/documentation/openshift_container_platform/latest/html-single/installing_on_azure/#installing-azure-with-short-term-creds_installing-azure-customizations)
@@ -585,6 +587,7 @@ The OADP operator automatically patches the Azure credentials secret with the `A
585587
**Important**: The standardized flow only supports the first BSL configuration. Additional BSLs with different resource groups require separate credentials and should not use the standardized flow secret.
586588

587589
#### Azure DPA Configuration
590+
588591
```yaml
589592
apiVersion: oadp.openshift.io/v1alpha1
590593
kind: DataProtectionApplication
@@ -615,6 +618,7 @@ spec:
615618
### GCP Workload Identity Federation Implementation
616619

617620
#### GCP Environment Variables
621+
618622
```go
619623
ProjectNumberEnvKey = "PROJECT_NUMBER" // GCP project number
620624
PoolIDEnvKey = "POOL_ID" // Workload identity pool ID
@@ -623,6 +627,7 @@ ServiceAccountEmailEnvKey = "SERVICE_ACCOUNT_EMAIL" // Service account email to
623627
```
624628

625629
#### GCP Prerequisites
630+
626631
1. **OpenShift cluster with GCP Workload Identity Federation enabled**
627632
- Cluster must be installed with manual credentials mode
628633
- Workload Identity Pool and Provider must be configured
@@ -674,6 +679,7 @@ ServiceAccountEmailEnvKey = "SERVICE_ACCOUNT_EMAIL" // Service account email to
674679
```
675680

676681
#### GCP Secret Creation
682+
677683
The `CreateOrUpdateSTSGCPSecret` function creates a Secret with the required GCP WIF configuration:
678684

679685
```go
@@ -701,11 +707,13 @@ func CreateOrUpdateSTSGCPSecret(setupLog logr.Logger, serviceAccountEmail, proje
701707
```
702708

703709
#### GCP Secret Format
710+
704711
- **Secret Name**: `cloud-credentials-gcp`
705712
- **Secret Key**: `service_account.json`
706713
- **Content**: GCP external account JSON following Google's external account format
707714

708715
#### GCP DPA Configuration
716+
709717
```yaml
710718
apiVersion: oadp.openshift.io/v1alpha1
711719
kind: DataProtectionApplication

go.mod

Lines changed: 48 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -26,34 +26,39 @@ require (
2626
)
2727

2828
require (
29-
cloud.google.com/go/storage v1.50.0
29+
cloud.google.com/go/storage v1.54.0
30+
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.18.0
31+
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.10.0
32+
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.6.1
3033
github.com/aws/aws-sdk-go-v2 v1.30.3
3134
github.com/aws/aws-sdk-go-v2/config v1.26.3
3235
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.15.11
3336
github.com/aws/aws-sdk-go-v2/service/s3 v1.48.0
3437
github.com/deckarep/golang-set/v2 v2.3.0
35-
github.com/google/go-cmp v0.6.0
38+
github.com/google/go-cmp v0.7.0
3639
github.com/hashicorp/go-multierror v1.1.1
3740
github.com/kubernetes-csi/external-snapshotter/client/v6 v6.3.0
3841
github.com/stretchr/testify v1.10.0
3942
github.com/vmware-tanzu/velero v1.14.0
4043
golang.org/x/exp v0.0.0-20230522175609-2e198f4a06a1
41-
google.golang.org/api v0.218.0
44+
google.golang.org/api v0.233.0
4245
k8s.io/klog/v2 v2.130.1
4346
)
4447

4548
require (
46-
cel.dev/expr v0.16.2 // indirect
47-
cloud.google.com/go v0.116.0 // indirect
48-
cloud.google.com/go/auth v0.14.0 // indirect
49-
cloud.google.com/go/auth/oauth2adapt v0.2.7 // indirect
49+
cel.dev/expr v0.20.0 // indirect
50+
cloud.google.com/go v0.121.0 // indirect
51+
cloud.google.com/go/auth v0.16.1 // indirect
52+
cloud.google.com/go/auth/oauth2adapt v0.2.8 // indirect
5053
cloud.google.com/go/compute/metadata v0.6.0 // indirect
51-
cloud.google.com/go/iam v1.2.2 // indirect
52-
cloud.google.com/go/monitoring v1.21.2 // indirect
54+
cloud.google.com/go/iam v1.5.2 // indirect
55+
cloud.google.com/go/monitoring v1.24.0 // indirect
56+
github.com/Azure/azure-sdk-for-go/sdk/internal v1.11.1 // indirect
5357
github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 // indirect
54-
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.25.0 // indirect
55-
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.48.1 // indirect
56-
github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/resourcemapping v0.48.1 // indirect
58+
github.com/AzureAD/microsoft-authentication-library-for-go v1.4.2 // indirect
59+
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.27.0 // indirect
60+
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.51.0 // indirect
61+
github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/resourcemapping v0.51.0 // indirect
5762
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.5.4 // indirect
5863
github.com/aws/aws-sdk-go-v2/credentials v1.17.26 // indirect
5964
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.11 // indirect
@@ -71,17 +76,17 @@ require (
7176
github.com/aws/smithy-go v1.20.3 // indirect
7277
github.com/beorn7/perks v1.0.1 // indirect
7378
github.com/blang/semver/v4 v4.0.0 // indirect
74-
github.com/census-instrumentation/opencensus-proto v0.4.1 // indirect
7579
github.com/cespare/xxhash/v2 v2.3.0 // indirect
76-
github.com/cncf/xds/go v0.0.0-20240905190251-b4127c9b8d78 // indirect
80+
github.com/cncf/xds/go v0.0.0-20250121191232-2f005788dc42 // indirect
7781
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
7882
github.com/emicklei/go-restful/v3 v3.11.0 // indirect
79-
github.com/envoyproxy/go-control-plane v0.13.1 // indirect
80-
github.com/envoyproxy/protoc-gen-validate v1.1.0 // indirect
83+
github.com/envoyproxy/go-control-plane/envoy v1.32.4 // indirect
84+
github.com/envoyproxy/protoc-gen-validate v1.2.1 // indirect
8185
github.com/evanphx/json-patch/v5 v5.9.0 // indirect
8286
github.com/fatih/color v1.18.0 // indirect
8387
github.com/felixge/httpsnoop v1.0.4 // indirect
8488
github.com/fxamacker/cbor/v2 v2.7.0 // indirect
89+
github.com/go-jose/go-jose/v4 v4.0.4 // indirect
8590
github.com/go-logr/stdr v1.2.2 // indirect
8691
github.com/go-logr/zapr v1.3.0 // indirect
8792
github.com/go-openapi/jsonpointer v0.19.6 // indirect
@@ -90,13 +95,14 @@ require (
9095
github.com/go-task/slim-sprig/v3 v3.0.0 // indirect
9196
github.com/gobwas/glob v0.2.3 // indirect
9297
github.com/gogo/protobuf v1.3.2 // indirect
98+
github.com/golang-jwt/jwt/v5 v5.2.2 // indirect
9399
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
94100
github.com/golang/protobuf v1.5.4 // indirect
95101
github.com/google/gnostic-models v0.6.8 // indirect
96102
github.com/google/gofuzz v1.2.0 // indirect
97103
github.com/google/pprof v0.0.0-20241210010833-40e02aabc2ad // indirect
98104
github.com/google/s2a-go v0.1.9 // indirect
99-
github.com/googleapis/enterprise-certificate-proxy v0.3.4 // indirect
105+
github.com/googleapis/enterprise-certificate-proxy v0.3.6 // indirect
100106
github.com/googleapis/gax-go/v2 v2.14.1 // indirect
101107
github.com/gorilla/websocket v1.5.0 // indirect
102108
github.com/hashicorp/errwrap v1.0.0 // indirect
@@ -110,6 +116,7 @@ require (
110116
github.com/json-iterator/go v1.1.12 // indirect
111117
github.com/klauspost/compress v1.17.11 // indirect
112118
github.com/kubernetes-csi/external-snapshotter/client/v7 v7.0.0 // indirect
119+
github.com/kylelemons/godebug v1.1.0 // indirect
113120
github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de // indirect
114121
github.com/mailru/easyjson v0.7.7 // indirect
115122
github.com/mattn/go-colorable v0.1.14 // indirect
@@ -122,6 +129,7 @@ require (
122129
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
123130
github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f // indirect
124131
github.com/oklog/run v1.0.0 // indirect
132+
github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c // indirect
125133
github.com/pkg/errors v0.9.1 // indirect
126134
github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10 // indirect
127135
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
@@ -131,34 +139,35 @@ require (
131139
github.com/prometheus/procfs v0.15.1 // indirect
132140
github.com/spf13/cobra v1.8.1 // indirect
133141
github.com/spf13/pflag v1.0.6-0.20210604193023-d5e0c0615ace // indirect
142+
github.com/spiffe/go-spiffe/v2 v2.5.0 // indirect
134143
github.com/x448/float16 v0.8.4 // indirect
135-
go.opencensus.io v0.24.0 // indirect
144+
github.com/zeebo/errs v1.4.0 // indirect
136145
go.opentelemetry.io/auto/sdk v1.1.0 // indirect
137-
go.opentelemetry.io/contrib/detectors/gcp v1.34.0 // indirect
138-
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.54.0 // indirect
139-
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.59.0 // indirect
140-
go.opentelemetry.io/otel v1.34.0 // indirect
141-
go.opentelemetry.io/otel/metric v1.34.0 // indirect
142-
go.opentelemetry.io/otel/sdk v1.34.0 // indirect
143-
go.opentelemetry.io/otel/sdk/metric v1.34.0 // indirect
144-
go.opentelemetry.io/otel/trace v1.34.0 // indirect
146+
go.opentelemetry.io/contrib/detectors/gcp v1.35.0 // indirect
147+
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.60.0 // indirect
148+
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.60.0 // indirect
149+
go.opentelemetry.io/otel v1.35.0 // indirect
150+
go.opentelemetry.io/otel/metric v1.35.0 // indirect
151+
go.opentelemetry.io/otel/sdk v1.35.0 // indirect
152+
go.opentelemetry.io/otel/sdk/metric v1.35.0 // indirect
153+
go.opentelemetry.io/otel/trace v1.35.0 // indirect
145154
go.uber.org/multierr v1.11.0 // indirect
146155
go.uber.org/zap v1.27.0 // indirect
147-
golang.org/x/crypto v0.35.0 // indirect
148-
golang.org/x/net v0.36.0 // indirect
149-
golang.org/x/oauth2 v0.27.0 // indirect
150-
golang.org/x/sync v0.11.0 // indirect
151-
golang.org/x/sys v0.30.0 // indirect
152-
golang.org/x/term v0.29.0 // indirect
153-
golang.org/x/text v0.22.0 // indirect
154-
golang.org/x/time v0.9.0 // indirect
156+
golang.org/x/crypto v0.38.0 // indirect
157+
golang.org/x/net v0.40.0 // indirect
158+
golang.org/x/oauth2 v0.30.0 // indirect
159+
golang.org/x/sync v0.14.0 // indirect
160+
golang.org/x/sys v0.33.0 // indirect
161+
golang.org/x/term v0.32.0 // indirect
162+
golang.org/x/text v0.25.0 // indirect
163+
golang.org/x/time v0.11.0 // indirect
155164
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d // indirect
156165
gomodules.xyz/jsonpatch/v2 v2.4.0 // indirect
157-
google.golang.org/genproto v0.0.0-20241118233622-e639e219e697 // indirect
158-
google.golang.org/genproto/googleapis/api v0.0.0-20250115164207-1a7da9e5054f // indirect
159-
google.golang.org/genproto/googleapis/rpc v0.0.0-20250115164207-1a7da9e5054f // indirect
160-
google.golang.org/grpc v1.69.4 // indirect
161-
google.golang.org/protobuf v1.36.3 // indirect
166+
google.golang.org/genproto v0.0.0-20250303144028-a0af3efb3deb // indirect
167+
google.golang.org/genproto/googleapis/api v0.0.0-20250505200425-f936aa4a68b2 // indirect
168+
google.golang.org/genproto/googleapis/rpc v0.0.0-20250505200425-f936aa4a68b2 // indirect
169+
google.golang.org/grpc v1.72.0 // indirect
170+
google.golang.org/protobuf v1.36.6 // indirect
162171
gopkg.in/evanphx/json-patch.v4 v4.12.0 // indirect
163172
gopkg.in/inf.v0 v0.9.1 // indirect
164173
gopkg.in/yaml.v2 v2.4.0 // indirect

0 commit comments

Comments
 (0)