|
| 1 | +In the previous unit, you explored how to choose and manage GitHub-hosted and self-hosted runners at a high level. In this unit, you'll learn how to configure, organize, monitor, and secure self-hosted runners for enterprise-scale use. This includes: |
| 2 | + |
| 3 | +- Setting up runners with proxies and custom labels |
| 4 | +- Managing runner groups |
| 5 | +- Monitoring runner health and performance |
| 6 | +- Configuring secure access using labels and IP allowlists |
| 7 | + |
| 8 | +## Configure self-hosted runners for enterprise use |
| 9 | + |
| 10 | +Self-hosted runners in GitHub Actions provide greater flexibility and control for enterprises that require **customized environments, network access, and security hardening**. This guide covers best practices for configuring self-hosted runners, including **proxies, labels, and networking** considerations. |
| 11 | + |
| 12 | +### 1. Setting up a self-hosted runner |
| 13 | + |
| 14 | +#### Step 1: Create and register a self-hosted runner |
| 15 | +1. Navigate to **GitHub Enterprise → Settings → Actions → Runners**. |
| 16 | +2. Click **New Runner** and select the desired OS (**Linux, Windows, or macOS**). |
| 17 | +3. Follow the provided commands to install and configure the runner on your machine. |
| 18 | + |
| 19 | +#### Step 2: Install & start the runner |
| 20 | +Run the following commands based on your OS: |
| 21 | + |
| 22 | +**Linux/macOS** |
| 23 | +```sh |
| 24 | +./config.sh --url https://github.com/<org-name> --token <generated-token> |
| 25 | +./run.sh |
| 26 | +``` |
| 27 | + |
| 28 | +**Windows (PowerShell)** |
| 29 | +``` |
| 30 | +.\config.cmd --url https://github.com/<org-name> --token <generated-token> |
| 31 | +.\run.cmd |
| 32 | +``` |
| 33 | + |
| 34 | +### 2. Configuring proxies for self-hosted runners |
| 35 | +Enterprises often operate behind corporate firewalls and proxies that restrict internet access. To allow self-hosted runners to communicate with GitHub, configure proxy settings as follows: |
| 36 | + |
| 37 | +##### Linux/macOS: Configure proxy |
| 38 | +Edit the environment file to define proxy settings: |
| 39 | +``` |
| 40 | +export http_proxy=http://proxy.company.com:8080 |
| 41 | +export https_proxy=http://proxy.company.com:8080 |
| 42 | +export no_proxy=localhost,127.0.0.1 |
| 43 | +``` |
| 44 | +Apply the settings: |
| 45 | + |
| 46 | +``` |
| 47 | +source ~/.bashrc |
| 48 | +``` |
| 49 | +#### Windows: Configure proxy |
| 50 | + |
| 51 | +Use the following PowerShell commands: |
| 52 | + |
| 53 | +``` |
| 54 | +[System.Environment]::SetEnvironmentVariable("HTTP_PROXY", "http://proxy.company.com:8080", "Machine") |
| 55 | +[System.Environment]::SetEnvironmentVariable("HTTPS_PROXY", "http://proxy.company.com:8080", "Machine") |
| 56 | +``` |
| 57 | +### 3.Using Labels for runner management |
| 58 | + |
| 59 | +Labels help organize and route jobs to specific self-hosted runners based on OS, hardware, or project requirements. |
| 60 | + |
| 61 | +:::image type="content" source="../media/github-runner-label.png" alt-text="Screenshot of runner groups screen with default label."::: |
| 62 | + |
| 63 | +#### Assigning labels to a runner |
| 64 | + |
| 65 | +When configuring a runner, you can assign custom labels: |
| 66 | + |
| 67 | +``` |
| 68 | +./config.sh --url https://github.com/<org-name> --token <generated-token> --labels "high-memory,gpu" |
| 69 | +``` |
| 70 | + |
| 71 | +#### Targeting a specific runner in a workflow |
| 72 | +To run a job on a specific runner with labels, update the workflow .yml: |
| 73 | +``` |
| 74 | +jobs: |
| 75 | + build: |
| 76 | + runs-on: [self-hosted, high-memory] |
| 77 | + steps: |
| 78 | + - name: Checkout repository |
| 79 | + uses: actions/checkout@v3 |
| 80 | +``` |
| 81 | + |
| 82 | +### 4.Networking considerations |
| 83 | +#### Allowlist GitHub IPs |
| 84 | + |
| 85 | +GitHub-hosted runners operate on dynamic IPs, but self-hosted runners need firewall rules to allow access. Retrieve the latest GitHub IP ranges: |
| 86 | +``` |
| 87 | +curl -s https://api.github.com/meta | jq .actions |
| 88 | +``` |
| 89 | +Allow these IPs in your firewall settings to ensure connectivity. |
| 90 | + |
| 91 | +#### Private network & VPN access |
| 92 | +For enterprise workloads requiring access to private systems, configure the runner to connect via VPN or an internal network. |
| 93 | + |
| 94 | +### 5. Security best practices for enterprise runners |
| 95 | +Restrict runners to trusted workflows: Prevent untrusted code from executing on self-hosted runners. |
| 96 | +Use ephemeral runners: Automatically remove runners after jobs to prevent persistent threats. |
| 97 | +Monitor runner activity: Log all runner actions and audit access. |
| 98 | +Apply OS security patches: Regularly update and secure the runner machine. |
| 99 | + |
| 100 | +### Manage self-hosted runners using groups |
| 101 | +Runner groups allow organizations to **manage access, control workload distribution, and enforce security policies** for self-hosted runners in GitHub Actions. This guide covers how to **create, manage, and move runners between groups** effectively. |
| 102 | + |
| 103 | +### 1. Understanding runner groups |
| 104 | +Runner groups help **organize and control** self-hosted runners within a **GitHub Enterprise or Organization**. They allow: |
| 105 | +- Restricting which repositories can use specific runners. |
| 106 | +- Controlling runner availability for different teams or workloads. |
| 107 | +- Managing permissions for **specific branches, workflows, or environments**. |
| 108 | + |
| 109 | +#### Runner group availability |
| 110 | +| GitHub Plan | Runner Groups Available? | |
| 111 | +|-------------|--------------------------| |
| 112 | +| GitHub Free | ❌ Not Available | |
| 113 | +| GitHub Pro | ❌ Not Available | |
| 114 | +| GitHub Team | ✅ Available | |
| 115 | +| GitHub Enterprise | ✅ Available | |
| 116 | + |
| 117 | +### 2. Creating a runner group |
| 118 | + |
| 119 | +1. Go to **GitHub → Organization Settings → Actions → Runners**. |
| 120 | +2. Click **"New group"** under **Self-Hosted Runners**. |
| 121 | +3. Provide a **name** for the group (e.g., "Linux-Runners", "High-Memory"). |
| 122 | +4. Choose **who can access the group** (entire organization or specific repositories). |
| 123 | +5. Click **Save**. |
| 124 | + |
| 125 | +:::image type="content" source="../media/github-create-runner-group.png" alt-text="Screenshot of the create runner group form screen."::: |
| 126 | + |
| 127 | +### 3. Adding Runners to a Group |
| 128 | +Once the group is created, you can **add runners manually or during registration**. |
| 129 | + |
| 130 | +##### Option 1: Assign during registration |
| 131 | +When configuring a new runner, specify the group: |
| 132 | +```sh |
| 133 | +./config.sh --url https://github.com/<org-name> --token <generated-token> --runnergroup "Linux-Runners" |
| 134 | +``` |
| 135 | +#### Option 2: Move an existing runner |
| 136 | +1. Navigate to **GitHub → Organization Settings → Actions → Runners.** |
| 137 | +2. Locate the **runner** and click **Edit.** |
| 138 | +3. Select a **new runner group** and save changes. |
| 139 | + |
| 140 | +### 4. Managing access and permissions |
| 141 | +#### Restricting runner group access |
| 142 | +**Organization-level runners:** Restrict usage to specific repositories. |
| 143 | +**Repository-level runners:** Only selected workflows can access the runner. |
| 144 | + |
| 145 | +Example: Restrict access to a specific repository |
| 146 | +1. Navigate to **Runner Group Settings.** |
| 147 | +2. Under **Repository Access**, select **"Only select repositories".** |
| 148 | +3. Add the repositories that are allowed to use the runner group. |
| 149 | + |
| 150 | +### 5. Moving runners between groups |
| 151 | +To **reassign a runner** from one group to another: |
| 152 | + |
| 153 | +1. Go to **GitHub → Organization Settings → Actions → Runners.** |
| 154 | +2. Click on the **runner name.** |
| 155 | +3. Select **Change group** → **Choose a new group.** |
| 156 | +4. Click **Save.** |
| 157 | +Alternatively, unregister and re-register the runner in a different group: |
| 158 | +``` |
| 159 | +./config.sh remove |
| 160 | +./config.sh --url https://github.com/<org-name> --token <generated-token> --runnergroup "New-Group" |
| 161 | +``` |
| 162 | + |
| 163 | +- Create separate groups for different OS types (e.g., Windows, Linux, macOS). |
| 164 | +- Use labels to further classify runners (e.g., GPU, high-memory, ARM). |
| 165 | +- Limit runner access to trusted repositories only. |
| 166 | +- Regularly audit and update runner groups based on team requirements. |
| 167 | +- Monitor runner usage and performance to optimize CI/CD workloads. |
| 168 | + |
| 169 | +## Monitor, troubleshoot, and update self-hosted runners** |
| 170 | +Managing self-hosted runners effectively requires **continuous monitoring, proactive troubleshooting, and regular updates**. This guide covers best practices and GitHub-recommended methods for ensuring **high availability, security, and performance** of self-hosted runners. |
| 171 | + |
| 172 | +### 1. Monitoring self-hosted runners |
| 173 | + |
| 174 | +#### Checking runner status |
| 175 | +To monitor runner availability: |
| 176 | +1. Navigate to **GitHub → Organization Settings → Actions → Runners**. |
| 177 | +2. Review the status: |
| 178 | + - ✅ **Idle** → Ready for workflows. |
| 179 | + - 🔄 **Active** → Currently running a job. |
| 180 | + - ❌ **Offline** → Runner is down or disconnected. |
| 181 | + |
| 182 | +#### Using GitHub API to fetch runner status |
| 183 | +You can programmatically check the status of self-hosted runners: |
| 184 | +```sh |
| 185 | +curl -H "Authorization: token <your_github_token>" \ |
| 186 | + -H "Accept: application/vnd.github.v3+json" \ |
| 187 | + https://api.github.com/orgs/<org-name>/actions/runners |
| 188 | +``` |
| 189 | +Response example: |
| 190 | +``` |
| 191 | +{ |
| 192 | + "total_count": 2, |
| 193 | + "runners": [ |
| 194 | + { |
| 195 | + "id": 1, |
| 196 | + "name": "runner-1", |
| 197 | + "status": "online" |
| 198 | + }, |
| 199 | + { |
| 200 | + "id": 2, |
| 201 | + "name": "runner-2", |
| 202 | + "status": "offline" |
| 203 | + } |
| 204 | + ] |
| 205 | +} |
| 206 | +``` |
| 207 | +### Logging and metrics |
| 208 | + |
| 209 | +- **System Logs:** Check logs in the `_diag/` directory within the runner installation folder. |
| 210 | +- **GitHub Actions Workflow Logs:** Navigate to Actions → Workflow Run → Logs to see runner execution details. |
| 211 | +- **Monitoring via Prometheus/Grafana:** Configure Prometheus exporters to track CPU, memory, and job execution time. |
| 212 | + |
| 213 | +### 2. Troubleshooting self-hosted runners |
| 214 | + |
| 215 | +#### Common issues and fixes |
| 216 | + |
| 217 | +| Issue | Possible Cause | Fix | |
| 218 | +|------------------------------|---------------------------------------|------------------------------------------| |
| 219 | +| Runner shows Offline | Network issue, token expired, or runner crashed | Restart runner: `./run.sh` | |
| 220 | +| Job stuck in Queued state | No available runners with required labels | Add runners or update labels | |
| 221 | +| Job fails with permission errors | Incorrect runner permissions | Ensure runner has the correct access | |
| 222 | +| Workflow execution is slow | High CPU/memory usage | Monitor system metrics & scale runners | |
| 223 | + |
| 224 | +#### Restarting a runner |
| 225 | +If a runner is stuck or not responding, restart it: |
| 226 | + |
| 227 | +```sh |
| 228 | +./svc.sh stop |
| 229 | +./svc.sh start |
| 230 | +``` |
| 231 | +For systemd-based Linux runners: |
| 232 | +``` |
| 233 | +sudo systemctl restart actions.runner.<org-name>.<runner-name>.service |
| 234 | +``` |
| 235 | +#### Checking logs for errors |
| 236 | +**Runner logs:** `<runner_dir>/_diag/Runner_<timestamp>.log` |
| 237 | +**GitHub Actions logs:** Check workflow execution logs in the GitHub UI. |
| 238 | + |
| 239 | +### 3. Updating self-hosted runners |
| 240 | +#### Checking for Runner Updates |
| 241 | +GitHub periodically updates runner binaries. To check for updates: |
| 242 | + |
| 243 | +```sh |
| 244 | +./config.sh --version |
| 245 | +``` |
| 246 | + |
| 247 | +You can also check runner versions via API: |
| 248 | +``` |
| 249 | +curl -H "Authorization: token <your_github_token>" \ |
| 250 | + -H "Accept: application/vnd.github.v3+json" \ |
| 251 | + https://api.github.com/repos/actions/runner/releases/latest |
| 252 | +``` |
| 253 | +#### Updating the runner |
| 254 | +**1. Manual update** |
| 255 | +Stop the runner |
| 256 | +``` |
| 257 | +./svc.sh stop |
| 258 | +``` |
| 259 | +**2. Download the latest runner:** |
| 260 | +``` |
| 261 | +curl -o actions-runner-linux-x64.tar.gz -L \ |
| 262 | + https://github.com/actions/runner/releases/latest/download/actions-runner-linux-x64.tar.gz |
| 263 | +``` |
| 264 | +**3.Extract and reconfigure:** |
| 265 | +``` |
| 266 | +tar xzf ./actions-runner-linux-x64.tar.gz |
| 267 | +./config.sh --url https://github.com/<org-name> --token <generated-token> |
| 268 | +./svc.sh install |
| 269 | +./svc.sh start |
| 270 | +``` |
| 271 | + |
| 272 | +#### Automated updates with GitHub actions |
| 273 | +To automatically check for and update runners: |
| 274 | +``` |
| 275 | +name: Update Runners |
| 276 | +
|
| 277 | +on: |
| 278 | + schedule: |
| 279 | + - cron: '0 3 * * 1' # Runs every Monday at 3 AM |
| 280 | +
|
| 281 | +jobs: |
| 282 | + update-runners: |
| 283 | + runs-on: self-hosted |
| 284 | + steps: |
| 285 | + - name: Download and update runner |
| 286 | + run: | |
| 287 | + ./svc.sh stop |
| 288 | + curl -o actions-runner.tar.gz -L \ |
| 289 | + https://github.com/actions/runner/releases/latest/download/actions-runner-linux-x64.tar.gz |
| 290 | + tar xzf actions-runner.tar.gz |
| 291 | + ./config.sh --url https://github.com/<org-name> --token <generated-token> |
| 292 | + ./svc.sh start |
| 293 | +``` |
| 294 | + |
0 commit comments