Skip to content

Commit 7680e06

Browse files
milldrcloudpossebotNuru
authored
Upstream EKS Actions Runner (cloudposse/terraform-aws-components#499)
* upstreaming latest requirements for actions runner * upstreaming latest requirements for actions runner * Added storage option for dind enabled runners * Added storage option for dind enabled runners * Added storage option for dind enabled runners * added optional feature, requires tf v 1.3.x * pr comments * adding webhook driven autoscaling and updating readme * pre-commit fixes * adding webhook driven autoscaling and updating readme * Update modules/eks/actions-runner-controller/README.md Co-authored-by: Nuru <[email protected]> * Update modules/eks/actions-runner-controller/README.md Co-authored-by: Nuru <[email protected]> * README updated for secret steps * README updated for secret steps Co-authored-by: cloudpossebot <[email protected]> Co-authored-by: Nuru <[email protected]>
1 parent 6cc3cb4 commit 7680e06

File tree

6 files changed

+125
-28
lines changed

6 files changed

+125
-28
lines changed

src/README.md

Lines changed: 69 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -28,56 +28,99 @@ components:
2828
name: "actions-runner" # avoids hitting name length limit on IAM role
2929
chart: "actions-runner-controller"
3030
chart_repository: "https://actions-runner-controller.github.io/actions-runner-controller"
31-
chart_version: "0.20.2"
31+
chart_version: "0.21.0"
3232
kubernetes_namespace: "actions-runner-system"
3333
create_namespace: true
34-
resources:
35-
limits:
36-
cpu: 50m
37-
memory: 100Mi
38-
requests:
39-
cpu: 10m
40-
memory: 30Mi
34+
kubeconfig_exec_auth_api_version: "client.authentication.k8s.io/v1beta1"
35+
# Purposely omit resource limits and requests
36+
# https://github.com/actions-runner-controller/actions-runner-controller/blob/91102c8088b6c01645bbb218b3d4552e774672bf/charts/actions-runner-controller/values.yaml#L100-L111
37+
resources: {}
4138
ssm_github_token_path: "/github/acme/github_token"
4239
ssm_github_webhook_secret_token_path: "/github/acme/github_webhook_secret_token"
40+
webhook:
41+
enabled: true
42+
# gha-webhook.use1.auto.core.acme.net
43+
hostname_template: "gha-webhook.%[3]v.%[2]v.%[1]v.acme.net"
44+
github_actions_iam_role_enabled: true
45+
github_actions_iam_role_attributes: [ "eks" ]
46+
github_actions_allowed_repos:
47+
- acme/app
48+
- acme/infrastructure
49+
timeout: 120
4350
runners:
44-
repository-runner:
51+
infrastructure-runner:
4552
type: "repository" # can be either 'organization' or 'repository'
46-
dind_enabled: false # A Docker sidecar container will be deployed
47-
image: summerwind/actions-runner # If dind_enabled=true, set this to 'summerwind/actions-runner-dind'
53+
dind_enabled: false # If `true`, a Docker sidecar container will be deployed
54+
# To run Docker in Docker (dind), change image from summerwind/actions-runner to summerwind/actions-runner-dind
55+
image: summerwind/actions-runner
4856
scope: "acme/infrastructure"
4957
scale_down_delay_seconds: 300
5058
min_replicas: 1
5159
max_replicas: 5
52-
busy_metrics:
60+
busy_metrics:
5361
scale_up_threshold: 0.75
5462
scale_down_threshold: 0.25
5563
scale_up_factor: 2
5664
scale_down_factor: 0.5
5765
resources:
5866
limits:
59-
cpu: 50m
60-
memory: 100Mi
67+
cpu: 200m
68+
memory: 256Mi
6169
requests:
62-
cpu: 10m
63-
memory: 30Mi
64-
webhook_driven_scaling_enabled: false
70+
cpu: 100m
71+
memory: 128Mi
72+
webhook_driven_scaling_enabled: true
6573
pull_driven_scaling_enabled: false
6674
labels:
6775
- "Ubuntu"
68-
- "core-otto"
76+
- "self-hosted"
6977
```
7078
71-
### Creating Github Tokens
79+
### Generating Required Secrets
80+
81+
AWS SSM is used to store and retrieve secrets. Generate the following as required and add each to AWS SSM at your chosen path:
82+
83+
1. A PAT with the scope outlined in [this document](https://github.com/actions-runner-controller/actions-runner-controller#deploying-using-pat-authentication). Save this to the value specified by `ssm_github_token_path`:
84+
85+
```
86+
ssm_github_token_path: "/github/acme/github_token"
87+
```
88+
89+
2. If using the Webhook Driven autoscaling (recommended), generate a random string to use as the Secret when creating the webhook in GitHub.
7290
73-
Ensure that the required tokens are created in AWS SSM.
74-
1. The `github_token` saved under `var.ssm_github_token_path`. The value should be a PAT with the scope outlined in [this document](https://github.com/actions-runner-controller/actions-runner-controller#deploying-using-pat-authentication).
75-
2. If using the Webhook Driven autoscaling (recommended), include the key `github_webhook_secret_token` saved under `var.ssm_github_webhook_secret_token_path`. Set to a random string you will use as the Secret when creating the webhook in GitHub.
7691
Generate the string using 1Password (no special characters, length 45) or by running
7792
```bash
7893
dd if=/dev/random bs=1 count=33 2>/dev/null | base64
7994
```
8095

96+
Store this key in AWS SSM under the same path specified by `ssm_github_webhook_secret_token_path`
97+
```
98+
ssm_github_webhook_secret_token_path: "/github/acme/github_webhook_secret_token"
99+
```
100+
101+
### Using Webhook Driven Autoscaling
102+
103+
To use the Webhook Driven autoscaling, you must also install the GitHub organization-level webhook after deploying the component
104+
(specifically, the webhook server). The URL for the webhook is determined by the `webhook.hostname_template` and where
105+
it is deployed. Recommended URL is `https://gha-webhook.[environment].[stage].[tenant].[service-discovery-domain]`.
106+
107+
As a GitHub organization admin, go to `https://github.com/organizations/[organization]/settings/hooks`, and then:
108+
- Click"Add webhook" and create a new webhook with the following settings:
109+
- Payload URL: copy from Terraform output `webhook_payload_url`
110+
- Content type: `application/json`
111+
- Secret: whatever you configured in the `sops` secret above
112+
- Which events would you like to trigger this webhook:
113+
- Select "Let me select individual events"
114+
- Uncheck everything ("Pushes" is likely the only thing already selected)
115+
- Check "Workflow jobs"
116+
- Ensure that "Active" is checked (should be checked by default)
117+
- Click "Add webhook" at the bottom of the settings page
118+
119+
After the webhook is created, select "edit" for the webhook and go to the "Recent Deliveries" tab and verify that there is a delivery
120+
(of a "ping" event) with a green check mark. If not, verify all the settings and consult
121+
the logs of the `actions-runner-controller-github-webhook-server` pod.
122+
123+
Useful Reference
81124

82125
### Updating CRDs
83126

@@ -99,7 +142,7 @@ Consult [actions-runner-controller](https://github.com/actions-runner-controller
99142

100143
| Name | Version |
101144
|------|---------|
102-
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.0.0 |
145+
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.3.0 |
103146
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | ~> 4.0 |
104147
| <a name="requirement_helm"></a> [helm](#requirement\_helm) | >= 2.0 |
105148
| <a name="requirement_kubernetes"></a> [kubernetes](#requirement\_kubernetes) | >= 2.0 |
@@ -178,7 +221,7 @@ Consult [actions-runner-controller](https://github.com/actions-runner-controller
178221
| <a name="input_regex_replace_chars"></a> [regex\_replace\_chars](#input\_regex\_replace\_chars) | Terraform regular expression (regex) string.<br>Characters matching the regex will be removed from the ID elements.<br>If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. | `string` | `null` | no |
179222
| <a name="input_region"></a> [region](#input\_region) | AWS Region. | `string` | n/a | yes |
180223
| <a name="input_resources"></a> [resources](#input\_resources) | The cpu and memory of the deployment's limits and requests. | <pre>object({<br> limits = object({<br> cpu = string<br> memory = string<br> })<br> requests = object({<br> cpu = string<br> memory = string<br> })<br> })</pre> | n/a | yes |
181-
| <a name="input_runners"></a> [runners](#input\_runners) | Map of Action Runner configurations, with the key being the name of the runner. Please note that the name must be in<br>kebab-case.<br><br>For example:<pre>hcl<br>organization_runner = {<br> type = "organization" # can be either 'organization' or 'repository'<br> dind_enabled: false # A Docker sidecar container will be deployed<br> image: summerwind/actions-runner # If dind_enabled=true, set this to 'summerwind/actions-runner-dind'<br> scope = "ACME" # org name for Organization runners, repo name for Repository runners<br> scale_down_delay_seconds = 300<br> min_replicas = 1<br> max_replicas = 5<br> busy_metrics = {<br> scale_up_threshold = 0.75<br> scale_down_threshold = 0.25<br> scale_up_factor = 2<br> scale_down_factor = 0.5<br> }<br> labels = [<br> "Ubuntu",<br> "mgmt-automation",<br> ]<br>}</pre> | <pre>map(object({<br> type = string<br> scope = string<br> image = string<br> dind_enabled = bool<br> scale_down_delay_seconds = number<br> min_replicas = number<br> max_replicas = number<br> busy_metrics = map(string)<br> webhook_driven_scaling_enabled = bool<br> pull_driven_scaling_enabled = bool<br> labels = list(string)<br> resources = object({<br> limits = object({<br> cpu = string<br> memory = string<br> })<br> requests = object({<br> cpu = string<br> memory = string<br> })<br> })<br> }))</pre> | n/a | yes |
224+
| <a name="input_runners"></a> [runners](#input\_runners) | Map of Action Runner configurations, with the key being the name of the runner. Please note that the name must be in<br>kebab-case.<br><br>For example:<pre>hcl<br>organization_runner = {<br> type = "organization" # can be either 'organization' or 'repository'<br> dind_enabled: false # A Docker sidecar container will be deployed<br> image: summerwind/actions-runner # If dind_enabled=true, set this to 'summerwind/actions-runner-dind'<br> scope = "ACME" # org name for Organization runners, repo name for Repository runners<br> scale_down_delay_seconds = 300<br> min_replicas = 1<br> max_replicas = 5<br> busy_metrics = {<br> scale_up_threshold = 0.75<br> scale_down_threshold = 0.25<br> scale_up_factor = 2<br> scale_down_factor = 0.5<br> }<br> labels = [<br> "Ubuntu",<br> "mgmt-automation",<br> ]<br>}</pre> | <pre>map(object({<br> type = string<br> scope = string<br> image = string<br> dind_enabled = bool<br> scale_down_delay_seconds = number<br> min_replicas = number<br> max_replicas = number<br> busy_metrics = map(string)<br> webhook_driven_scaling_enabled = bool<br> pull_driven_scaling_enabled = bool<br> labels = list(string)<br> storage = optional(string, false)<br> resources = object({<br> limits = object({<br> cpu = string<br> memory = string<br> ephemeral_storage = optional(string, false)<br> })<br> requests = object({<br> cpu = string<br> memory = string<br> })<br> })<br> }))</pre> | n/a | yes |
182225
| <a name="input_s3_bucket_arns"></a> [s3\_bucket\_arns](#input\_s3\_bucket\_arns) | List of ARNs of S3 Buckets to which the runners will have read-write access to. | `list(string)` | `[]` | no |
183226
| <a name="input_ssm_github_token_path"></a> [ssm\_github\_token\_path](#input\_ssm\_github\_token\_path) | The path in SSM to the GitHub token. | `string` | `""` | no |
184227
| <a name="input_ssm_github_webhook_secret_token_path"></a> [ssm\_github\_webhook\_secret\_token\_path](#input\_ssm\_github\_webhook\_secret\_token\_path) | The path in SSM to the GitHub Webhook Secret token. | `string` | `""` | no |
@@ -203,5 +246,7 @@ Consult [actions-runner-controller](https://github.com/actions-runner-controller
203246
- [cloudposse/terraform-aws-components](https://github.com/cloudposse/terraform-aws-components/tree/master/modules/eks/actions-runner-controller) - Cloud Posse's upstream component
204247
- [alb-controller](https://artifacthub.io/packages/helm/aws/aws-load-balancer-controller) - Helm Chart
205248
- [alb-controller](https://github.com/kubernetes-sigs/aws-load-balancer-controller) - AWS Load Balancer Controller
249+
- [actions-runner-controller Webhook Driven Scaling](https://github.com/actions-runner-controller/actions-runner-controller/blob/master/docs/detailed-docs.md#webhook-driven-scaling)
250+
- [actions-runner-controller Chart Values](https://github.com/actions-runner-controller/actions-runner-controller/blob/master/charts/actions-runner-controller/values.yaml)
206251

207252
[<img src="https://cloudposse.com/logo-300x69.svg" height="32" align="right"/>](https://cpco.io/component)

src/charts/actions-runner/templates/runnerdeployment.yaml

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,5 +43,27 @@ spec:
4343
imagePullPolicy: IfNotPresent
4444
serviceAccountName: {{ .Values.service_account_name }}
4545
resources:
46-
{{- toYaml .Values.resources | nindent 8 }}
46+
limits:
47+
cpu: {{ .Values.resources.limits.cpu }}
48+
memory: {{ .Values.resources.limits.memory }}
49+
{{- if and .Values.dind_enabled .Values.resources.limits.ephemeral_storage }}
50+
ephemeral-storage: {{ .Values.resources.limits.ephemeral_storage }}
51+
{{- end }}
52+
requests:
53+
cpu: {{ .Values.resources.requests.cpu }}
54+
memory: {{ .Values.resources.requests.memory }}
55+
{{- if and .Values.dind_enabled .Values.storage }}
56+
dockerVolumeMounts:
57+
- mountPath: /var/lib/docker
58+
name: docker-volume
59+
volumes:
60+
- name: docker-volume
61+
ephemeral:
62+
volumeClaimTemplate:
63+
spec:
64+
accessModes: [ "ReadWriteOnce" ] # Only 1 pod can connect at a time
65+
resources:
66+
requests:
67+
storage: {{ .Values.storage }}
68+
{{- end }}
4769

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
type: "repository" # can be either 'organization' or 'repository'
2+
dind_enabled: true # If `true`, a Docker sidecar container will be deployed
3+
# To run Docker in Docker (dind), change image from summerwind/actions-runner to summerwind/actions-runner-dind
4+
image: summerwind/actions-runner-dind
5+
scope: "example/app"
6+
scale_down_delay_seconds: 300
7+
min_replicas: 1
8+
max_replicas: 2
9+
busy_metrics:
10+
scale_up_threshold: 0.75
11+
scale_down_threshold: 0.25
12+
scale_up_factor: 2
13+
scale_down_factor: 0.5
14+
resources:
15+
limits:
16+
cpu: 1.5
17+
memory: 4Gi
18+
ephemeral_storage: "10Gi"
19+
requests:
20+
cpu: 0.5
21+
memory: 1Gi
22+
storage: "10Gi"
23+
webhook_driven_scaling_enabled: false
24+
pull_driven_scaling_enabled: false
25+
labels:
26+
- "Ubuntu"
27+
- "core-example"

src/main.tf

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -192,6 +192,7 @@ module "actions_runner" {
192192
dind_enabled = each.value.dind_enabled
193193
service_account_role_arn = module.actions_runner_controller.service_account_role_arn
194194
resources = each.value.resources
195+
storage = each.value.storage
195196
labels = each.value.labels
196197
scale_down_delay_seconds = each.value.scale_down_delay_seconds
197198
min_replicas = each.value.min_replicas

src/variables.tf

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -158,10 +158,12 @@ variable "runners" {
158158
webhook_driven_scaling_enabled = bool
159159
pull_driven_scaling_enabled = bool
160160
labels = list(string)
161+
storage = optional(string, false)
161162
resources = object({
162163
limits = object({
163-
cpu = string
164-
memory = string
164+
cpu = string
165+
memory = string
166+
ephemeral_storage = optional(string, false)
165167
})
166168
requests = object({
167169
cpu = string

src/versions.tf

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
terraform {
2-
required_version = ">= 1.0.0"
2+
required_version = ">= 1.3.0"
33

44
required_providers {
55
aws = {

0 commit comments

Comments
 (0)