Skip to content

Commit 77228b5

Browse files
author
kosta709
committed
dind-volumes - added aws support
1 parent 9b1607f commit 77228b5

File tree

10 files changed

+160
-13
lines changed

10 files changed

+160
-13
lines changed

‎README.md‎

Lines changed: 61 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -75,27 +75,81 @@ rules:
7575

7676
#### Pipeline Storage with docker cache support
7777

78-
###### GKE LocalSSD
79-
**Prerequisite:** [GKE custer with local SSD](https://cloud.google.com/kubernetes-engine/docs/how-to/persistent-volumes/local-ssd)
78+
##### **GKE LocalSSD**
79+
*Prerequisite:* [GKE custer with local SSD](https://cloud.google.com/kubernetes-engine/docs/how-to/persistent-volumes/local-ssd)
80+
81+
*Install venona for using GKE Local SSD:*
8082
```
8183
venonactl install [options] --set-value=Storage.LocalVolumeParentDir=/mnt/disks/ssd0/codefresh-volumes \
8284
--build-node-selector=cloud.google.com/gke-local-ssd=true
8385
```
8486
85-
###### Using GKE Disks
86-
**Prerequisite:** dind-volume-provisioner should have permissions to create/delete/get of google disks
87-
There are 3 options:
87+
##### **GCE Disks**
88+
*Prerequisite:* volume provisioner (dind-volume-provisioner) should have permissions to create/delete/get of google disks
89+
There are 3 options to provide cloud credentials on GCE:
8890
* run venona dind-volume-provisioniner on node with iam role which is allowed to create/delete/get of google disks
8991
* create Google Service Account with ComputeEngine.StorageAdmin, download its key and pass it to venona installed with `--set-file=Storage.GooogleServiceAccount=/path/to/google-service-account.json`
9092
* use [Google Workload Identity](https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity) to assign iam role to `volume-provisioner-venona` service account
9193
92-
**Note**: Builds will be running in single availability zone, so you must to specify AvailabilityZone params
94+
*Note*: Builds will be running in single availability zone, so you must to specify AvailabilityZone params
95+
9396
97+
*Install venona for using GKE Disks:*
9498
```
9599
venonactl install [options] --set-value=Storage.Backend=gcedisk \
96100
--set-value=Storage.AvailabilityZone=us-central1-a \
97101
--build-node-selector=failure-domain.beta.kubernetes.io/zone=us-central1-a \
98-
[--set-file=Storage.GoogleServiceAccount=/path/to/google-service-account.json"]
102+
[--set-file=Storage.GoogleServiceAccount=/path/to/google-service-account.json]
103+
```
104+
105+
##### **Amazon EBS**
106+
107+
*Prerequisite:* volume provisioner (dind-volume-provisioner) should have permissions to create/delete/get of aws ebs
108+
Minimal iam policy for dind-volume-provisioner:
109+
```json
110+
{
111+
"Version": "2012-10-17",
112+
"Statement": [
113+
{
114+
"Effect": "Allow",
115+
"Action": [
116+
"ec2:AttachVolume",
117+
"ec2:CreateSnapshot",
118+
"ec2:CreateTags",
119+
"ec2:CreateVolume",
120+
"ec2:DeleteSnapshot",
121+
"ec2:DeleteTags",
122+
"ec2:DeleteVolume",
123+
"ec2:DescribeInstances",
124+
"ec2:DescribeSnapshots",
125+
"ec2:DescribeTags",
126+
"ec2:DescribeVolumes",
127+
"ec2:DetachVolume"
128+
],
129+
"Resource": "*"
130+
}
131+
]
132+
}
133+
```
134+
135+
There are 3 options to provide cloud credentials on AWS:
136+
* run venona dind-volume-provisioniner on node with the iam role - use `--kube-node-selector=` option
137+
* create AWS IAM User, assign it the permissions above and suppy aws credentials to venona installer `--set-value=Storage.AwsAccessKeyId=ABCDF --set-value=Storage.AwsSecretAccessKey=ZYXWV`
138+
139+
* use [Aws Identity for Service Account](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html) to assign iam role to `volume-provisioner-venona` service account
140+
141+
*Notes*:
142+
- Builds will be running in single availability zone, so you must specify AvailabilityZone parameter `--set-value=Storage.AvailabilityZone=<aws-az>` and build-node-selector `--build-node-selector=failure-domain.beta.kubernetes.io/zone=<aws-az>` in case of multizone cluster
143+
144+
- We support both [in-tree ebs](https://kubernetes.io/docs/concepts/storage/volumes/#awselasticblockstore) (`--set-value=Storage.Backend=ebs`) volumes and ebs-csi(https://github.com/kubernetes-sigs/aws-ebs-csi-driver) (`--set-value=Storage.Backend=ebs-csi`)
145+
146+
*Install Command to run pipelines on ebs volumes*
147+
```
148+
venonactl install [options] --set-value=Storage.Backend=ebs \
149+
--set-value=Storage.AvailabilityZone=us-east-1d \
150+
--build-node-selector=failure-domain.beta.kubernetes.io/zone=us-east-1d \
151+
[--kube-node-selector=kubernetes.io/role=master] \
152+
[--set-value=Storage.AwsAccessKeyId=ABCDF --set-value=Storage.AwsSecretAccessKey=ZYXWV]
99153
```
100154

101155
#### Kubernetes RBAC

‎package.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"name": "venona",
3-
"version": "0.30.6",
3+
"version": "0.31.0",
44
"description": "Codefresh agent to run on Codefresh's runtime environment and execute pipeline",
55
"main": "index.js",
66
"scripts": {

‎venonactl/VERSION‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
0.30.6
1+
0.31.0

‎venonactl/pkg/codefresh/cfapi.go‎

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -178,10 +178,6 @@ func (a *api) Register() (*codefresh.RuntimeEnvironment, error) {
178178
options.NodeSelector = a.buildNodeSelector
179179
}
180180

181-
// options.StorageClass = fmt.Sprintf("%s-%s", a.storageClass, a.clusternamespace)
182-
// if !a.isDefaultStorageClass {
183-
// options.StorageClass = a.storageClass
184-
// }
185181
options.StorageClass = a.storageClass
186182

187183
if len(a.annotations) != 0 {

‎venonactl/pkg/plugins/runtime-environment.go‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,8 @@ func (u *runtimeEnvironmentPlugin) Install(opt *InstallOptions, v Values) (Value
6868
cfOpt.StorageClass = fmt.Sprintf("dind-gcedisk-%s-%s-%s", storageParams["AvailabilityZone"], v["AppName"], v["Namespace"])
6969
case "ebs":
7070
cfOpt.StorageClass = fmt.Sprintf("dind-ebs-%s-%s-%s", storageParams["AvailabilityZone"], v["AppName"], v["Namespace"])
71+
case "ebs-csi":
72+
cfOpt.StorageClass = fmt.Sprintf("dind-ebs-csi-%s-%s-%s", storageParams["AvailabilityZone"], v["AppName"], v["Namespace"])
7173
}
7274
}
7375
}

‎venonactl/pkg/store/store.go‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -109,6 +109,8 @@ func (s *Values) BuildValues() map[string]interface{} {
109109
"LocalVolumeParentDir": "/var/lib/codefresh/dind-volumes",
110110
"AvailabilityZone": "",
111111
"GoogleServiceAccount": "",
112+
"AwsAccessKeyId": "",
113+
"AwsSecretAccessKey": "",
112114
"VolumeProvisioner": map[string]interface{}{
113115
"Image": "codefresh/dind-volume-provisioner:v20",
114116
"NodeSelector": s.KubernetesAPI.NodeSelector,

‎venonactl/pkg/templates/kubernetes/deployment.dind-volume-provisioner.vp.yaml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,20 @@ spec:
4747
env:
4848
- name: PROVISIONER_NAME
4949
value: codefresh.io/dind-volume-provisioner-{{ .AppName }}-{{ .Namespace }}
50+
{{- if .Storage.AwsAccessKeyId }}
51+
- name: AWS_ACCESS_KEY_ID
52+
valueFrom:
53+
secretKeyRef:
54+
name: dind-volume-provisioner-{{ .AppName }}
55+
key: aws_access_key_id
56+
{{- end }}
57+
{{- if .Storage.AwsSecretAccessKey }}
58+
- name: AWS_SECRET_ACCESS_KEY
59+
valueFrom:
60+
secretKeyRef:
61+
name: dind-volume-provisioner-{{ .AppName }}
62+
key: aws_secret_access_key
63+
{{- end }}
5064
{{- if .Storage.GoogleServiceAccount }}
5165
- name: GOOGLE_APPLICATION_CREDENTIALS
5266
value: /etc/dind-volume-provisioner/credentials/google-service-account.json

‎venonactl/pkg/templates/kubernetes/secret.dind-volume-provisioner.vp.yaml‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
{{- if not (eq .Storage.Backend "local") }}
2+
apiVersion: v1
3+
kind: Secret
4+
type: Opaque
5+
metadata:
6+
name: dind-volume-provisioner-{{ .AppName }}
7+
namespace: {{ .Namespace }}
8+
labels:
9+
app: dind-volume-provisioner
10+
data:
11+
{{- if .Storage.GoogleServiceAccount }}
12+
google-service-account.json: {{ .Storage.GoogleServiceAccount | b64enc }}
13+
{{- end }}
14+
{{- if .Storage.AwsAccessKeyId }}
15+
aws_access_key_id: {{ .Storage.AwsAccessKeyId | b64enc }}
16+
{{- end }}
17+
{{- if .Storage.AwsSecretAccessKey }}
18+
aws_secret_access_key: {{ .Storage.AwsSecretAccessKey | b64enc }}
19+
{{- end }}
20+
{{- end }}

‎venonactl/pkg/templates/kubernetes/storageclass.dind-aws-ebs.vp.yaml‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
{{- if or (eq .Storage.Backend "ebs") (eq .Storage.Backend "ebs-csi") }}
2+
---
3+
kind: StorageClass
4+
apiVersion: storage.k8s.io/v1
5+
metadata:
6+
name: dind-{{ .Storage.Backend }}-{{.Storage.AvailabilityZone}}-{{ .AppName }}-{{ .Namespace }}
7+
labels:
8+
app: dind-volume-provisioner
9+
provisioner: codefresh.io/dind-volume-provisioner-{{ .AppName }}-{{ .Namespace }}
10+
parameters:
11+
# ebs or ebs-csi
12+
volumeBackend: {{ .Storage.Backend }}
13+
# gp2 or io1
14+
VolumeType: {{ .Storage.VolumeType | default "gp2" }}
15+
# Valid zone in aws (us-east-1c, ...)
16+
AvailabilityZone: {{ .Storage.AvailabilityZone }}
17+
# ext4 or xfs (default to ext4 )
18+
fsType: {{ .Storage.FsType | default "ext4" }}
19+
{{- end }}

‎venonactl/pkg/templates/kubernetes/templates.go‎

Lines changed: 40 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)