k3s Homelab System

A Terraform configuration for a k3s-based homelab.

Explore the docs >

Tech Stack · Installation · Usage · Future Plans

Table of Contents

1. About the Project
2. Tech Stack
3. Prerequisites
4. Installation
5. VPN Setup
6. Usage
7. Modules
8. Future Plans
9. Contributing

About the Project

This project sets up a lightweight Kubernetes cluster using k3s. It runs on ARM64 hardware. The example uses a Raspberry Pi 4B. Terraform deploys services like Vaultwarden, Syncthing, Grafana, and NGINX Ingress. The setup includes a single-node cluster. It focuses on home use.

Tech Stack

Category	Technologies
Infrastructure
Services
VPN

Prerequisites

Install these on your local machine:

• Terraform.
• kubectl.
• SSH client.

For the server hardware:

• ARM64 device, such as Raspberry Pi 4B.
• SD card (e.g., 16GB).

Installation

Ssh key generation

From local machine, generate SSH key:

ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

Copy .pub file content in the next step.

OS Setup

Flash the image to the SD card with Raspberry Pi Imager.

Choose Raspberry Pi OS Lite (64-bit) This example uses the headless version.

In the imager, go to Edit settings > General. Set hostname (e.g., cirrus.local) Set username and password (e.g., username: ibis) Go to > Services tab. Set Allow public-key authentication only. Set authorized_keys via Add Key. Paste .pub file's content here.

Feel free to set other settings.

Insert the SD card. Connect to network. Power on the device.

Now you can ssh into the device via:

ssh -i ~/.ssh/ibis-cirrus ibis@cirrus

Update the system:

sudo apt update -y && sudo apt upgrade -y

Set hostname (e.g., cirrus):

sudo hostnamectl set-hostname cirrus

Reboot:

sudo reboot now

Local Machine Hostname Setup

For now, manual name resolution is needed. Get the device's ip address. (e.g., 192.168.1.13) Add the following to /etc/hosts:

192.168.1.13 cirrus
192.168.1.13 vaultwarden.cirrus
192.168.1.13 syncthing.cirrus
192.168.1.13 grafana.cirrus

k3s Setup

On the device, enable kernel parameters:

sudo sed -i 's/$/ cgroup_memory=1 cgroup_enable=memory/' /boot/firmware/cmdline.txt
sudo reboot now

Install iptables:

sudo apt install iptables -y

Install k3s. Disable Traefik:

curl -sfL https://get.k3s.io | sh -s - --disable traefik

Verify:

sudo kubectl get nodes

kubectl Access

On local machine, copy kubeconfig:

ssh -i ~/.ssh/ibis-cirrus ibis@cirrus  "sudo cat /etc/rancher/k3s/k3s.yaml" > ~/.kube/config

Update server URL:

sed -i 's|server: https://127.0.0.1:6443|server: https://cirrus:6443|g' ~/.kube/config

Vaultwarden

Certificate

Generate self-signed certificate on server:

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=cirrus"

Use in .tfvars file.

Admin Token

Generate random token:

openssl rand -base64 48

Use in .tfvars file.

Grafana Storage

K3s' Persistent Volume managed directory would not have the correct permissions. The best solution is to create directory on server manually:

sudo mkdir -p /mnt/grafana-data
sudo chmod 777 /mnt/grafana-data

Terraform Setup

Clone repository:

git clone https://github.com/bramlak/k3s-homelab-system.git
cd k3s-homelab-system/k3s

Copy variables file:

cp terraform.tfvars.example terraform.tfvars

Edit terraform.tfvars. Set values like admin token.

Initialize:

terraform init

VPN Setup

Install Docker on server:

curl -fsSL https://get.docker.com | sh
sudo usermod -aG docker $USER
newgrp docker

Enable IP forwarding:

sudo sysctl -w net.ipv4.ip_forward=1

Make permanent. Edit /etc/sysctl.conf. Uncomment net.ipv4.ip_forward=1. Apply:

sudo sysctl -p

Use wireguard-docker-compose.yml from repository.

Get puid and pgid values:

id -u
id -g

Tailor wireguard-docker-compose.yml via changing puid and pgid.

Allow ipv4 packet forwarding via editing /etc/sysctl.conf: uncomment: net.ipv4.ip_forward=1

sudo sysctl -p

Start WireGuard:

docker compose -f wireguard-docker-compose.yml up -d

DDNS Setup

Register domain at ydns.eu (e.g., ibis.ydns.eu).

Clone updater:

git clone https://github.com/ydns/bash-updater.git

Edit updater.sh. Add credentials.

Set crontab:

crontab -e

Add:

*/5 * * * * ~/bash-updater/updater.sh >> ~/bash-updater/updater.log 2>&1

Port Forwarding

Configure router. Example for ZTE F660RV1:

Go to Application > Port Forwarding.

Add rule:

• Name: cirrus-wg
• Protocol: UDP
• WAN Ports: 51820
• LAN IP: Server IP
• LAN Ports: 51820

WireGuard Client

Copy client config from server: ~/wireguard/config/peer1/peer1.conf.

Add to client device.

Usage

Deploy:

terraform apply

Check pods:

kubectl get pods --all-namespaces

Destroy:

terraform destroy

Modules

• NGINX Ingress: Custom ingress controller.
• Vaultwarden: Password manager. Needs admin token.
• Syncthing: File syncronisation solution.
• Grafana: Monitoring dashboard.

Future Plans

• streamline install process.
• Automate storage permissions.
• Add more services.
• Add primary VPN into Terraform. Keeping docker-compose wireguard for secondary emergency VPN server.
• Solve DNS resolutions.

Contributing

Fork the repository. Create a branch. Commit changes. Push. Open pull request.

Top contributors: