Skip to content

Commit c78201d

Browse files
committed
Update readme, explain offline runners
1 parent 1d555ea commit c78201d

File tree

1 file changed

+16
-13
lines changed

1 file changed

+16
-13
lines changed

README.md

Lines changed: 16 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,10 @@
11
# Terraform module for scalable self hosted GitHub action runners
22

3-
> WIP: Module is in development
4-
53
This [Terraform](https://www.terraform.io/) modules create the required infra structure needed to host [GitHub Action](https://github.com/features/actions) self hosted runners on [AWS spot instances](https://aws.amazon.com/ec2/spot/). All logic required to handle the lifecycle for an action runners is implemented in AWS Lambda functions.
64

75
## Motivation
86

9-
GitHub Actions `self hosted` runners provides you with a flexible option to run your CI workloads on compute of your choice. Currently there is no option provided to automate the creation and scaling of action runners. This module takes care of creating the AWS infra structure to host action runners on spot instances. And provides lambda modules to orchestrate the lifecycle of the action runners.
7+
GitHub Actions `self hosted` runners provides you with a flexible option to run your CI workloads on compute of your choice. Currently there is no option provided to automate the creation and scaling of action runners. This module takes care of creating the AWS infra structure to host action runners on spot instances. And provides lambda modules to orchestrate the life cycle of the action runners.
108

119
Lambda is chosen as runtime for two major reasons. First it allows to create small components with minimal access to AWS and GitHub. Secondly it provides a scalable setup for minimal costs that works on repo level and scales to organization level. The lambdas will create Linux based EC2 instances with Docker to serve CI workloads that can run on Linux and/or Docker. The main goal is here to support Docker based workloads.
1210

@@ -37,12 +35,15 @@ Besides these permissions, the lambdas also need permission to CloudWatch (for l
3735
Examples are provided in [the example directory](examples/). Please ensure you have installed the following tools.
3836

3937
- Terraform, or [tfenv](https://github.com/tfutils/tfenv).
40-
- Bash shell or compatible.
41-
- TODO: building lambda ?
42-
- AWS cli
38+
- Bash shell or compatible
39+
- Docker (optional, to build lambda's without node).
40+
- AWS cli (optional)
41+
- Node and yarn (for lambda development).
4342

4443
The module support two main scenarios for creating runners. On repository level a runner will be dedicated to only one repository, no other repository can use the runner. On organization level you can use the runner(s) for all the repositories within the organization. See https://help.github.com/en/actions/hosting-your-own-runners/about-self-hosted-runners for more information. Before starting the deployment you have to choose one option.
4544

45+
GitHub workflows will fail immediately if there is no action runner available to host your builds. Since this module supports to scale from 0 and up your builds will fail in case there is no active runner available. So we recommend to create an offline runner with the matching label by default `self-hosted`. Create this runner by just following the GitHub instruction on you local machine. You can stop the process after the step of running the `config.sh`. This offline runner will will ensure you builds will not fail if a runner becomes available within 24 hours.
46+
4647
The setup consists of running Terraform to create all AWS resources and configure the GitHub App. The Terraform module requires configuration from the GitHub App and the GitHub app requires output from Terraform. Therefore you should first create the GitHub App, configure the basics. Then run Terraform and finalize the configuration of the GitHub App afterwards.
4748

4849
### Setup GitHub App (part 1)
@@ -88,14 +89,15 @@ output "files" {
8889
}
8990
```
9091

91-
Next run `terraform init && terraform apply` as result the lambdas will be download to the same directory.
92+
Next run `terraform init && terraform apply` as result the lambdas will be download to the same directory. Alternatively you can download the zip artifacts with any other tool of you favour.
9293

94+
For local development you can build all the lambda's at once using `.ci/build.sh` or per lambda using `yarn`, check the`package.json` for the commands.
9395

9496
Next create a second terraform workspace and initiate the module, see the examples for more details.
9597

9698
```terraform
9799
module "runners" {
98-
source = "git::https://github.com/philips-labs/terraform-aws-github-runner/"
100+
source = "git::https://github.com/philips-labs/terraform-aws-github-runner.git?ref=master"
99101
100102
aws_region = "eu-west-1"
101103
vpc_id = "vpc-123"
@@ -107,8 +109,8 @@ module "runners" {
107109
key_base64 = "base64string"
108110
id = "1"
109111
client_id = "c-123"
110-
client_secret = "secret"
111-
webhook_secret = "secret"
112+
client_secret = "client_secret"
113+
webhook_secret = "webhook_secret"
112114
}
113115
114116
webhook_lambda_zip = "lambdas-download/webhook.zip"
@@ -125,7 +127,7 @@ terraform init
125127
terrafrom apply
126128
```
127129

128-
Check the terraform output for the API gateway url, which you need in the next step. The lambda for syncing the GitHub distribution will be executed by a trigger via Cloud Watch. To ensure the binary is cached, trigger the `runner-binaries-syncer` manually. The payload does not matter.
130+
Check the terraform output for the API gateway url (endpoint), which you need in the next step. The lambda for syncing the GitHub distribution will be executed by a trigger via Cloud Watch. To ensure the binary is cached, trigger the `runner-binaries-syncer` manually. The payload does not matter. (e.g. `aws lambda invoke --function-name <environment>-syncer respone.json`)
129131

130132
### Setup GitHub App (part 2)
131133

@@ -136,9 +138,12 @@ Go back to the GitHub App and update the following settings.
136138
3. Provide the webhook secret.
137139
4. Enable the `Check run` event for the webhook.
138140

141+
You are now ready to run action workloads on self hosted runner, remember builds will fail if there is no runner available with matching labels.
139142

140143
## Examples
141144

145+
TODO
146+
142147
## Inputs
143148

144149
| Name | Description | Type | Default | Required |
@@ -169,8 +174,6 @@ Go back to the GitHub App and update the following settings.
169174
| runners | n/a |
170175
| webhook | n/a |
171176

172-
173-
174177
## Philips Forest
175178

176179
This module is part of the Philips Forest.

0 commit comments

Comments
 (0)