Skip to content

Commit f3d2eb7

Browse files
committed
feat: debian runner docs
1 parent eac8006 commit f3d2eb7

File tree

1 file changed

+62
-0
lines changed

1 file changed

+62
-0
lines changed
Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
---
2+
title: Self-hosted Debian Github Runner for ocflib
3+
---
4+
5+
## Origin
6+
Why self-host a github runner if Github gives you free ones if your repo is public?
7+
8+
In a nutshell, repositories like ocflib has CI tests that depends on having access to OCF LDAP (only accessible from within OCF network) and OCF groups/users (UNIX groups). If we want to keep using github's free runner, we have to configure some kind of external LDAP access only for the runner, and also configure test UNIX groups info prior to running each CI/CD.
9+
10+
Self-hosted github runner would run under ocf network, on our k8s cluster. We also have a lot more control over it.
11+
12+
[words from storce]
13+
Why did I write a new runner on top of the Nix runner? For a few (personal) reasons:
14+
1. There is a significant lack of documentation around how Nix runner actuall works.
15+
2. I am a big believer of OCF k8s, and have the runner be centralized along with other applications on k8s makes me happy :D.
16+
3. I am not entirely comfortable with Nix (yet). Spinning up a Debian runner based off the existing OCF Docker image (that has things like LDAP pre-configured) was very simple to do.
17+
18+
## User Manual
19+
### How to add new repositories to the runner
20+
This action can only be performed by a superuser (ocfroot/owner of the OCF github organization). In order to avoid creating security vulnerabilities, it is very important for you to follow ALL OF THE STEPS below when adding a new repository, :
21+
1. Thoroughly audit the files under `.github/workflow/` of the target repository. Make sure that there are no malicious code. Ideally, set up branch protection for the repository.
22+
2. Go the github.com/ocf -> Settings -> Actions -> Runner groups -> Default, select the new repository by clicking the gear icon next to "x selected repository"
23+
3. Under the target repository's action files (`.github/workflow/*.yml`), make sure that there are fork PR protections under your desired job (the conditional `if: github.repository == 'ocf/repo_name'`).
24+
4. Again, under your desired job, write `runs-on: self-hosted`
25+
Like:
26+
```
27+
name: __name__
28+
29+
on:
30+
push:
31+
branches: [master]
32+
33+
jobs:
34+
__job_name__:
35+
if: github.repository == 'ocf/repo_name'
36+
name: __name__
37+
runs-on: self-hosted
38+
steps:
39+
...
40+
```
41+
42+
43+
## Architecture
44+
The runner can be found at github:ocf/github-runner
45+
46+
The runner runs on our kubernetes cluster under the namespace github-runner. It uses the transpire CI/CD pipeline.
47+
48+
The Dockerfile and the comments in it should be largely self-explatory. A lot of the set up steps are just copy-pasted from Github's documentation. Some points, however are worth noting:
49+
50+
1. `setup_user.sh` initializes the runner with some UNIX groups/users (like the basic ocfstaff/guser etc.) so that some tests that runs on it (like those from ocflib) wouldn't fail. I couldn't find a smarter way to import them. Yes, this means that if some tests were to require new groups and users, we would need to manually change the script.
51+
2. `start.sh` starts the runner. It uses Github Personal Access Token (PAT). We use PAT since the default runner intialization token provided by Github is ephemeral, which makes it unsuitable for an environment like k8s. The PAT is an org-scoped github PAT called `self-hosted-runner` that allows users to read and write to self-hosted runners and org-level administration. The PAT is injected as an environmental variable into the container.
52+
53+
While the org-scoped github PAT gives us the flexibility of using the runner for all OCF repos, we can't just let any actions or repos run on it. For example, a malicious action can execute `echo $GITHUB_PAT` and leak the PAT in action logs. There are security concerns with runnners like this. Here is how they are addressed:
54+
55+
1. While the PAT is org-scoped, the runner is assigned to the Default runner group (see ocf org Github admin page). The runner is only allowed to pick up jobs from authorized repos under the group. So an org owner (ocfroot) would have to approve any new repo.
56+
2. Another vulnerability comes from fork pull request action triggers. A malicious user can open a PR and trick the runner into executing malicious code. To prevent this, the github workflow configuration file must have `if: github.repository == 'ocf/repo_name'` so that actions can only be triggered from the main repository, not a fork.
57+
58+
59+
60+
61+
62+

0 commit comments

Comments
 (0)