Skip to content

Commit 4b5c93f

Browse files
k8s-ci-robotk8s-ci-robot
authored
Merge pull request kubernetes#3424 from tghartland/magnum-helm-chart
Add magnum support in cluster autoscaler helm chart
2 parents 75a22a3 + 5f08511 commit 4b5c93f

File tree

9 files changed

+193
-19
lines changed

9 files changed

+193
-19
lines changed

‎charts/cluster-autoscaler-chart/Chart.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,4 +17,4 @@ name: cluster-autoscaler-chart
1717
sources:
1818
- https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler
1919
type: application
20-
version: 1.0.5
20+
version: 1.1.0

‎charts/cluster-autoscaler-chart/README.md‎

Lines changed: 32 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -45,10 +45,15 @@ You must provide some minimal configuration, either to specify instance groups o
4545

4646
Either:
4747

48-
- Set `autoDiscovery.clusterName` and tag your autoscaling groups appropriately (`--cloud-provider=aws` only) **or**
49-
- Set at least one ASG as an element in the `autoscalingGroups` array with its three values: `name`, `minSize` and `maxSize`.
48+
- Set `autoDiscovery.clusterName` and provide additional autodiscovery options if necessary **or**
49+
- Set static node group configurations for one or more node groups (using `autoscalingGroups` or `autoscalingGroupsnamePrefix`).
5050

51-
To install the chart with the release name `my-release`:
51+
To create a valid configuration, follow instructions for your cloud provider:
52+
53+
* [AWS](#aws---using-auto-discovery-of-tagged-instance-groups)
54+
* [GCE](#gce)
55+
* [Azure AKS](#azure-aks)
56+
* [OpenStack Magnum](#openstack-magnum)
5257

5358
### AWS - Using auto-discovery of tagged instance groups
5459

@@ -163,6 +168,24 @@ The following parameters are required:
163168
- `azureVMType: "AKS"`
164169
- `azureNodeResourceGroup: "your-aks-cluster-node-resource-group"`
165170
171+
### OpenStack Magnum
172+
173+
`cloudProvider: magnum` must be set, and then one of
174+
175+
- `magnumClusterName=<cluster name or ID>` and `autoscalingGroups` with the names of node groups and min/max node counts
176+
- or `autoDiscovery.clusterName=<cluster name or ID>` with one or more `autoDiscovery.roles`.
177+
178+
Additionally, `cloudConfigPath: "/etc/kubernetes/cloud-config"` must be set as this should be the location
179+
of the cloud-config file on the host.
180+
181+
Example values files can be found [here](../../cluster-autoscaler/cloudprovider/magnum/examples).
182+
183+
Install the chart with
184+
185+
```
186+
$ helm install my-release autoscaler/cluster-autoscaler-chart -f myvalues.yaml
187+
```
188+
166189
## Uninstalling the Chart
167190
168191
To uninstall `my-release`:
@@ -290,9 +313,10 @@ Though enough for the majority of installations, the default PodSecurityPolicy _
290313
| Key | Type | Default | Description |
291314
|-----|------|---------|-------------|
292315
| affinity | object | `{}` | Affinity for pod assignment |
293-
| autoDiscovery.clusterName | string | `nil` | Enable autodiscovery for name in ASG tag (only `cloudProvider=aws`). Must be set for `cloudProvider=gce`, but no MIG tagging required. |
316+
| autoDiscovery.clusterName | string | `nil` | Enable autodiscovery for `cloudProvider=aws`, for groups matching `autoDiscovery.tags`. Enable autodiscovery for `cloudProvider=gce`, but no MIG tagging required. Enable autodiscovery for `cloudProvider=magnum`, for groups matching `autoDiscovery.roles`. |
317+
| autoDiscovery.roles | list | `["worker"]` | Magnum node group roles to match. |
294318
| autoDiscovery.tags | list | `["k8s.io/cluster-autoscaler/enabled","k8s.io/cluster-autoscaler/{{ .Values.autoDiscovery.clusterName }}"]` | ASG tags to match, run through `tpl`. |
295-
| autoscalingGroups | list | `[]` | For AWS. At least one element is required if not using `autoDiscovery`. For example: <pre> - name: asg1<br /> maxSize: 2<br /> minSize: 1 </pre> |
319+
| autoscalingGroups | list | `[]` | For AWS, Azure AKS or Magnum. At least one element is required if not using `autoDiscovery`. For example: <pre> - name: asg1<br /> maxSize: 2<br /> minSize: 1 </pre> |
296320
| autoscalingGroupsnamePrefix | list | `[]` | For GCE. At least one element is required if not using `autoDiscovery`. For example: <pre> - name: ig01<br /> maxSize: 10<br /> minSize: 0 </pre> |
297321
| awsAccessKeyID | string | `""` | AWS access key ID ([if AWS user keys used](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/aws/README.md#using-aws-credentials)) |
298322
| awsRegion | string | `"us-east-1"` | AWS region (required if `cloudProvider=aws`) |
@@ -307,7 +331,7 @@ Though enough for the majority of installations, the default PodSecurityPolicy _
307331
| azureUseManagedIdentityExtension | bool | `false` | Whether to use Azure's managed identity extension for credentials. If using MSI, ensure subscription ID and resource group are set. |
308332
| azureVMType | string | `"AKS"` | Azure VM type. |
309333
| cloudConfigPath | string | `"/etc/gce.conf"` | Configuration file for cloud provider. |
310-
| cloudProvider | string | `"aws"` | The cloud provider where the autoscaler runs. Currently only `gce`, `aws`, and `azure` are supported. `aws` supported for AWS. `gce` for GCE. `azure` for Azure AKS. |
334+
| cloudProvider | string | `"aws"` | The cloud provider where the autoscaler runs. Currently only `gce`, `aws`, `azure` and `magnum` are supported. `aws` supported for AWS. `gce` for GCE. `azure` for Azure AKS. `magnum` for OpenStack Magnum. |
311335
| containerSecurityContext | object | `{}` | [Security context for container](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/) |
312336
| dnsPolicy | string | `"ClusterFirst"` | Defaults to `ClusterFirst`. Valid values are: `ClusterFirstWithHostNet`, `ClusterFirst`, `Default` or `None`. If autoscaler does not depend on cluster DNS, recommended to set this to `Default`. |
313337
| expanderPriorities | object | `{}` | The expanderPriorities is used if `extraArgs.expander` is set to `priority` and expanderPriorities is also set with the priorities. If `extraArgs.expander` is set to `priority`, then expanderPriorities is used to define cluster-autoscaler-priority-expander priorities. See: https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/expander/priority/readme.md |
@@ -319,6 +343,8 @@ Though enough for the majority of installations, the default PodSecurityPolicy _
319343
| image.repository | string | `"us.gcr.io/k8s-artifacts-prod/autoscaling/cluster-autoscaler"` | Image repository |
320344
| image.tag | string | `"v1.18.1"` | Image tag |
321345
| kubeTargetVersionOverride | string | `""` | Allow overridding the `.Capabilities.KubeVersion.GitVersion` check. Useful for `helm template` commands. |
346+
| magnumCABundlePath | string | `"/etc/kubernetes/ca-bundle.crt"` | Path to the host's CA bundle, from `ca-file` in the cloud-config file. |
347+
| magnumClusterName | string | `""` | Cluster name or ID in Magnum. Required if `cloudProvider=magnum` and not setting `autoDiscovery.clusterName`. |
322348
| nameOverride | string | `""` | String to partially override `cluster-autoscaler.fullname` template (will maintain the release name) |
323349
| nodeSelector | object | `{}` | Node labels for pod assignment. Ref: https://kubernetes.io/docs/user-guide/node-selection/. |
324350
| podAnnotations | object | `{}` | Annotations to add to each pod. |

‎charts/cluster-autoscaler-chart/README.md.gotmpl‎

Lines changed: 26 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -45,10 +45,15 @@ You must provide some minimal configuration, either to specify instance groups o
4545

4646
Either:
4747

48-
- Set `autoDiscovery.clusterName` and tag your autoscaling groups appropriately (`--cloud-provider=aws` only) **or**
49-
- Set at least one ASG as an element in the `autoscalingGroups` array with its three values: `name`, `minSize` and `maxSize`.
48+
- Set `autoDiscovery.clusterName` and provide additional autodiscovery options if necessary **or**
49+
- Set static node group configurations for one or more node groups (using `autoscalingGroups` or `autoscalingGroupsnamePrefix`).
5050

51-
To install the chart with the release name `my-release`:
51+
To create a valid configuration, follow instructions for your cloud provider:
52+
53+
* [AWS](#aws---using-auto-discovery-of-tagged-instance-groups)
54+
* [GCE](#gce)
55+
* [Azure AKS](#azure-aks)
56+
* [OpenStack Magnum](#openstack-magnum)
5257

5358
### AWS - Using auto-discovery of tagged instance groups
5459

@@ -163,6 +168,24 @@ The following parameters are required:
163168
- `azureVMType: "AKS"`
164169
- `azureNodeResourceGroup: "your-aks-cluster-node-resource-group"`
165170

171+
### OpenStack Magnum
172+
173+
`cloudProvider: magnum` must be set, and then one of
174+
175+
- `magnumClusterName=<cluster name or ID>` and `autoscalingGroups` with the names of node groups and min/max node counts
176+
- or `autoDiscovery.clusterName=<cluster name or ID>` with one or more `autoDiscovery.roles`.
177+
178+
Additionally, `cloudConfigPath: "/etc/kubernetes/cloud-config"` must be set as this should be the location
179+
of the cloud-config file on the host.
180+
181+
Example values files can be found [here](../../cluster-autoscaler/cloudprovider/magnum/examples).
182+
183+
Install the chart with
184+
185+
```
186+
$ helm install my-release autoscaler/cluster-autoscaler-chart -f myvalues.yaml
187+
```
188+
166189
## Uninstalling the Chart
167190

168191
To uninstall `my-release`:

‎charts/cluster-autoscaler-chart/templates/deployment.yaml‎

Lines changed: 21 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -59,14 +59,20 @@ spec:
5959
- --node-group-auto-discovery=mig:namePrefix={{ .name }},min={{ .minSize }},max={{ .maxSize }}
6060
{{- end }}
6161
{{- end }}
62+
{{- else if eq .Values.cloudProvider "magnum" }}
63+
{{- if .Values.autoDiscovery.clusterName }}
64+
- --cluster-name={{ .Values.autoDiscovery.clusterName }}
65+
- --node-group-auto-discovery=magnum:role={{ tpl (join "," .Values.autoDiscovery.roles) . }}
66+
{{- else }}
67+
- --cluster-name={{ .Values.magnumClusterName }}
68+
{{- end }}
6269
{{- end }}
63-
{{- if eq .Values.cloudProvider "gce" }}
70+
{{- if or (eq .Values.cloudProvider "gce") (eq .Values.cloudProvider "magnum") }}
6471
- --cloud-config={{ .Values.cloudConfigPath }}
65-
{{- end }}
72+
{{- end }}
6673
{{- range $key, $value := .Values.extraArgs }}
6774
- --{{ $key }}={{ $value }}
6875
{{- end }}
69-
7076
env:
7177
{{- if and (eq .Values.cloudProvider "aws") (ne .Values.awsRegion "") }}
7278
- name: AWS_REGION
@@ -167,11 +173,16 @@ spec:
167173
securityContext:
168174
{{ toYaml .Values.containerSecurityContext | nindent 12 | trim }}
169175
{{- end }}
170-
{{- if eq .Values.cloudProvider "gce" }}
176+
{{- if or (eq .Values.cloudProvider "gce") (eq .Values.cloudProvider "magnum") }}
171177
volumeMounts:
172178
- name: cloudconfig
173179
mountPath: {{ .Values.cloudConfigPath }}
174180
readOnly: true
181+
{{- if and (eq .Values.cloudProvider "magnum") (.Values.magnumCABundlePath) }}
182+
- name: ca-bundle
183+
mountPath: {{ .Values.magnumCABundlePath }}
184+
readOnly: true
185+
{{- end }}
175186
{{- end }}
176187
{{- if .Values.affinity }}
177188
affinity:
@@ -188,11 +199,16 @@ spec:
188199
securityContext:
189200
{{ toYaml .Values.securityContext | nindent 8 | trim }}
190201
{{- end }}
191-
{{- if eq .Values.cloudProvider "gce" }}
202+
{{- if or (eq .Values.cloudProvider "gce") (eq .Values.cloudProvider "magnum") }}
192203
volumes:
193204
- name: cloudconfig
194205
hostPath:
195206
path: {{ .Values.cloudConfigPath }}
207+
{{- if and (eq .Values.cloudProvider "magnum") (.Values.magnumCABundlePath) }}
208+
- name: ca-bundle
209+
hostPath:
210+
path: {{ .Values.magnumCABundlePath }}
211+
{{- end }}
196212
{{- end }}
197213
{{- if .Values.image.pullSecrets }}
198214
imagePullSecrets:

‎charts/cluster-autoscaler-chart/values.yaml‎

Lines changed: 20 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,17 +3,25 @@
33
affinity: {}
44

55
autoDiscovery:
6-
# Only cloudProvider `aws` and `gce` are supported by auto-discovery at this time
6+
# cloudProviders `aws`, `gce` and `magnum` are supported by auto-discovery at this time
77
# AWS: Set tags as described in https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/aws/README.md#auto-discovery-setup
8-
# autoDiscovery.clusterName -- Enable autodiscovery for name in ASG tag (only `cloudProvider=aws`). Must be set for `cloudProvider=gce`, but no MIG tagging required.
8+
9+
# autoDiscovery.clusterName -- Enable autodiscovery for `cloudProvider=aws`, for groups matching `autoDiscovery.tags`.
10+
# Enable autodiscovery for `cloudProvider=gce`, but no MIG tagging required.
11+
# Enable autodiscovery for `cloudProvider=magnum`, for groups matching `autoDiscovery.roles`.
912
clusterName: # cluster.local
13+
1014
# autoDiscovery.tags -- ASG tags to match, run through `tpl`.
1115
tags:
1216
- k8s.io/cluster-autoscaler/enabled
1317
- k8s.io/cluster-autoscaler/{{ .Values.autoDiscovery.clusterName }}
1418
# - kubernetes.io/cluster/{{ .Values.autoDiscovery.clusterName }}
1519

16-
# autoscalingGroups -- For AWS. At least one element is required if not using `autoDiscovery`. For example:
20+
# autoDiscovery.roles -- Magnum node group roles to match.
21+
roles:
22+
- worker
23+
24+
# autoscalingGroups -- For AWS, Azure AKS or Magnum. At least one element is required if not using `autoDiscovery`. For example:
1725
# <pre>
1826
# - name: asg1<br />
1927
# maxSize: 2<br />
@@ -84,12 +92,20 @@ azureNodeResourceGroup: ""
8492
# azureUseManagedIdentityExtension -- Whether to use Azure's managed identity extension for credentials. If using MSI, ensure subscription ID and resource group are set.
8593
azureUseManagedIdentityExtension: false
8694

95+
# magnumClusterName -- Cluster name or ID in Magnum.
96+
# Required if `cloudProvider=magnum` and not setting `autoDiscovery.clusterName`.
97+
magnumClusterName: ""
98+
99+
# magnumCABundlePath -- Path to the host's CA bundle, from `ca-file` in the cloud-config file.
100+
magnumCABundlePath: "/etc/kubernetes/ca-bundle.crt"
101+
87102
# cloudConfigPath -- Configuration file for cloud provider.
88103
cloudConfigPath: /etc/gce.conf
89104

90105
# cloudProvider -- The cloud provider where the autoscaler runs.
91-
# Currently only `gce`, `aws`, and `azure` are supported.
106+
# Currently only `gce`, `aws`, `azure` and `magnum` are supported.
92107
# `aws` supported for AWS. `gce` for GCE. `azure` for Azure AKS.
108+
# `magnum` for OpenStack Magnum.
93109
cloudProvider: aws
94110

95111
# containerSecurityContext -- [Security context for container](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/)

‎cluster-autoscaler/cloudprovider/magnum/README.md‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,30 @@ to match your cluster.
4747
| --nodes | Used to select a specific node group to autoscale and constrain its node count. Of the form `min:max:NodeGroupName`. Can be used multiple times. |
4848
| --node-group-auto-discovery | See below. |
4949

50+
#### Deployment with helm
51+
52+
Alternatively, the autoscaler can be deployed with the cluster autoscaler helm chart.
53+
A minimal values.yaml file looks like:
54+
55+
```yaml
56+
cloudProvider: "magnum"
57+
58+
magnumClusterName: "cluster name or ID"
59+
60+
autoscalingGroups:
61+
- name: default-worker
62+
maxSize: 5
63+
minSize: 1
64+
65+
cloudConfigPath: "/etc/kubernetes/cloud-config"
66+
```
67+
68+
For running on the master node and other suggested settings, see
69+
[examples/values-example.yaml](examples/values-example.yaml).
70+
To deploy with node group autodiscovery (for cluster autoscaler v1.19+), see
71+
[examples/values-autodiscovery.yaml](examples/values-autodiscovery.yaml).
72+
73+
5074
## Node group auto discovery
5175
5276
Instead of using `--nodes` to select specific node groups by name,

‎cluster-autoscaler/cloudprovider/magnum/examples/values-autodiscovery.yaml‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
cloudProvider: "magnum"
2+
3+
autoDiscovery:
4+
clusterName: "cluster name or ID"
5+
roles:
6+
- worker
7+
- autoscaling
8+
9+
image:
10+
repository: docker.io/openstackmagnum/cluster-autoscaler
11+
tag: v1.19.0
12+
13+
nodeSelector:
14+
node-role.kubernetes.io/master: ""
15+
16+
tolerations:
17+
- key: CriticalAddonsOnly
18+
value: "True"
19+
effect: NoSchedule
20+
- key: dedicated
21+
value: "master"
22+
effect: NoSchedule
23+
- key: node-role.kubernetes.io/master
24+
effect: NoSchedule
25+
26+
cloudConfigPath: /etc/kubernetes/cloud-config
27+
28+
extraArgs:
29+
v: 2

‎cluster-autoscaler/cloudprovider/magnum/examples/values-example.yaml‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
cloudProvider: "magnum"
2+
3+
magnumClusterName: "cluster name or ID"
4+
5+
autoscalingGroups:
6+
- name: default-worker
7+
maxSize: 5
8+
minSize: 1
9+
10+
image:
11+
repository: docker.io/openstackmagnum/cluster-autoscaler
12+
tag: v1.19.0
13+
14+
nodeSelector:
15+
node-role.kubernetes.io/master: ""
16+
17+
tolerations:
18+
- key: CriticalAddonsOnly
19+
value: "True"
20+
effect: NoSchedule
21+
- key: dedicated
22+
value: "master"
23+
effect: NoSchedule
24+
- key: node-role.kubernetes.io/master
25+
effect: NoSchedule
26+
27+
cloudConfigPath: "/etc/kubernetes/cloud-config"
28+
29+
extraArgs:
30+
v: 2

‎cluster-autoscaler/cloudprovider/magnum/examples/values-minimal.yaml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
cloudProvider: "magnum"
2+
3+
magnumClusterName: "cluster name or ID"
4+
5+
autoscalingGroups:
6+
- name: default-worker
7+
maxSize: 5
8+
minSize: 1
9+
10+
cloudConfigPath: "/etc/kubernetes/cloud-config"

0 commit comments

Comments
 (0)