Skip to content

Commit 7dcef6d

Browse files
nemanjamnemanjam
committed
add hashnode, devto md articles
1 parent a100a72 commit 7dcef6d

File tree

2 files changed

+734
-0
lines changed

2 files changed

+734
-0
lines changed

‎src/content/post/2025/05-29-traefik-load-balancer/_resources/index-devto.md

Lines changed: 367 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,367 @@
1+
## Introduction
2+
3+
This article is a continuation of [Expose home server with Rathole tunnel and Traefik](https://nemanjamitic.com/blog/2025-04-29-rathole-traefik-home-server) article, which explains how to permanently host websites from home by bypassing CGNAT. That setup works well for exposing a single home server (like a Raspberry Pi, server PC, or virtual machine), but it has a limitation: it requires one VPS (or at least one public network interface) per home server. This is because the Rathole server exclusively uses ports `80` and `443`.
4+
5+
But it doesn't have to be like this. We can reuse a single Rathole server for many tunnels and home servers, we just need a tool to load balance their traffic, as long as our VPS's network interface provides enough bandwidth for our websites and services.
6+
7+
This article explains how to achieve that using Traefik HTTP and TCP routers.
8+
9+
## Prerequisites
10+
11+
- A working Rathole tunnel setup from the previous article (including a VPS and a domain name)
12+
- More than one home server (Raspberry Pi, server PC, virtual machine, or LXC container)
13+
14+
## Architecture overview
15+
16+
### The problem
17+
18+
The main problem here is that we can't bind more than one port to ports `80` and `443`, respectively. Only one service can listen on a given port at the same time. So something like this doesn't exist:
19+
20+
```yml
21+
# docker-compose.yml
22+
23+
services:
24+
rathole:
25+
image: rapiz1/rathole:v0.5.0
26+
container_name: rathole
27+
command: --server /config/rathole.server.toml
28+
restart: unless-stopped
29+
ports:
30+
# host:container
31+
- 2333:2333
32+
- 80:5080,5081 # non existent syntax, can't bind two ports to a single port
33+
- 443:5443,5444 # same
34+
volumes:
35+
- ./rathole.server.toml:/config/rathole.server.toml:ro
36+
```
37+
38+
Neither the operating system nor Docker provides load balancing functionality out of the box, we need to handle it ourselves.
39+
40+
### The solution
41+
42+
We need to introduce a tool for load balancing traffic between tunnels. We will use Traefik, since we already use it with the Rathole client.
43+
44+
For each home server, we need 2 tunnels: one for HTTP and another for HTTPS traffic:
45+
46+
1. The tunnel for HTTP traffic will use the Traefik HTTP router as usual.
47+
2. The tunnel for HTTPS traffic is a bit more interesting and challenging. For it, we will use the Traefik TCP router running in passthrough mode, since we don't want to terminate HTTPS traffic on the VPS. Instead, we want to delegate certificate resolution to the existing Traefik instance running on the client side to preserve the current setup and architecture.
48+
49+
![Traefik load balancer architecture diagram](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/492hjhuusn1yf3atcgua.png)
50+
51+
**Reminder:**
52+
53+
I already wrote about the advantage of resolving SSL certificates locally on the home server in the [Architecture overview](https://nemanjamitic.com/blog/2025-04-29-rathole-traefik-home-server#architecture-overview) section of the previous article, but here is a quick recap:
54+
55+
1. The home server contains its entire configuration
56+
2. The home server is tunnel-agnostic and reusable
57+
3. No coupling between the tunnel server and client, no need to maintain state or version
58+
4. Decoupled debugging
59+
5. Improved security, an additional encryption layer further down the tunnel
60+
61+
## Traefik load balancer and Rathole server
62+
63+
Since we passthrough encrypted HTTPS traffic, Traefik can't read the subdomain from an HTTP request as usual. Instead, we will run the Traefik router in TCP mode, using the [HostSNIRegexp](https://doc.traefik.io/traefik/v2.9/routing/routers/#rule_1) matcher. This will run the router on layer 4 (TCP) instead of the usual layer 7 (HTTP).
64+
65+
For more in-depth info on how this works, you can read here: [Server Name Indication (SNI)](https://en.wikipedia.org/wiki/Server_Name_Indication).
66+
67+
Now that we understand the principle, we can get to the practical implementation.
68+
69+
### Traefik HTTP and TCP routers
70+
71+
Below is the complete `docker-compose.yml` that defines the Traefik TCP router and the Rathole server with 2 HTTP/HTTPS tunnel pairs for 2 home servers: `pi` (OrangePi) and `local` (MiniPC), in my case.
72+
73+
```yml
74+
# docker-compose.yml
75+
76+
version: '3.8'
77+
78+
services:
79+
traefik:
80+
image: traefik:v2.9.8
81+
container_name: traefik
82+
restart: unless-stopped
83+
command:
84+
- --providers.docker=true
85+
- --entrypoints.web.address=:80
86+
- --entrypoints.websecure.address=:443
87+
- --entrypoints.traefik.address=:8080
88+
- --api.dashboard=true
89+
- --api.insecure=false
90+
- --log.level=DEBUG
91+
- --accesslog=true
92+
ports:
93+
- 80:80
94+
- 443:443
95+
- 8080:8080
96+
volumes:
97+
- /var/run/docker.sock:/var/run/docker.sock:ro
98+
networks:
99+
- proxy
100+
labels:
101+
# Enable the dashboard at http://traefik.amd2.nemanjamitic.com
102+
# http for simplicity, no acme.json file
103+
- traefik.enable=true
104+
- 'traefik.http.routers.traefik.rule=Host(`traefik.amd2.${SITE_HOSTNAME}`)'
105+
- traefik.http.routers.traefik.entrypoints=web
106+
- traefik.http.routers.traefik.service=api@internal
107+
- traefik.http.routers.traefik.middlewares=auth
108+
- 'traefik.http.middlewares.auth.basicauth.users=${TRAEFIK_AUTH}'
109+
110+
rathole:
111+
image: rapiz1/rathole:v0.5.0
112+
container_name: rathole
113+
command: --server /config/rathole.server.toml
114+
restart: unless-stopped
115+
ports:
116+
- 2333:2333
117+
volumes:
118+
- ./rathole.server.toml:/config/rathole.server.toml:ro
119+
networks:
120+
- proxy
121+
122+
labels:
123+
### HTTP port 80 - HTTP routers ###
124+
125+
# pi.nemanjamitic.com, www.pi.nemanjamitic.com, *.pi.nemanjamitic.com, www.*.pi.nemanjamitic.com
126+
127+
# Route *.pi.nemanjamitic.com -> 5080
128+
- 'traefik.http.routers.rathole-pi.rule=HostRegexp(`pi.${SITE_HOSTNAME}`, `www.pi.${SITE_HOSTNAME}`, `{subdomain:[a-z0-9-]+}.pi.${SITE_HOSTNAME}`, `www.{subdomain:[a-z0-9-]+}.pi.${SITE_HOSTNAME}`)'
129+
- traefik.http.routers.rathole-pi.entrypoints=web
130+
- traefik.http.routers.rathole-pi.service=rathole-pi
131+
- traefik.http.services.rathole-pi.loadbalancer.server.port=5080
132+
133+
# Route *.local.nemanjamitic.com -> 5081
134+
- 'traefik.http.routers.rathole-local.rule=HostRegexp(`local.${SITE_HOSTNAME}`, `www.local.${SITE_HOSTNAME}`, `{subdomain:[a-z0-9-]+}.local.${SITE_HOSTNAME}`, `www.{subdomain:[a-z0-9-]+}.local.${SITE_HOSTNAME}`)'
135+
- traefik.http.routers.rathole-local.entrypoints=web
136+
- traefik.http.routers.rathole-local.service=rathole-local
137+
- traefik.http.services.rathole-local.loadbalancer.server.port=5081
138+
139+
### HTTPS port 443 with TLS passthrough - TCP routers ###
140+
141+
# Route *.pi.nemanjamitic.com -> 5443
142+
- 'traefik.tcp.routers.rathole-pi-secure.rule=HostSNIRegexp(`pi.${SITE_HOSTNAME}`, `www.pi.${SITE_HOSTNAME}`, `{subdomain:[a-z0-9-]+}.pi.${SITE_HOSTNAME}`, `www.{subdomain:[a-z0-9-]+}.pi.${SITE_HOSTNAME}`)'
143+
- traefik.tcp.routers.rathole-pi-secure.entrypoints=websecure
144+
- traefik.tcp.routers.rathole-pi-secure.tls.passthrough=true
145+
- traefik.tcp.routers.rathole-pi-secure.service=rathole-pi-secure
146+
- traefik.tcp.services.rathole-pi-secure.loadbalancer.server.port=5443
147+
148+
# Route *.local.nemanjamitic.com -> 5444
149+
- 'traefik.tcp.routers.rathole-local-secure.rule=HostSNIRegexp(`local.${SITE_HOSTNAME}`, `www.local.${SITE_HOSTNAME}`, `{subdomain:[a-z0-9-]+}.local.${SITE_HOSTNAME}`, `www.{subdomain:[a-z0-9-]+}.local.${SITE_HOSTNAME}`)'
150+
- traefik.tcp.routers.rathole-local-secure.entrypoints=websecure
151+
- traefik.tcp.routers.rathole-local-secure.tls.passthrough=true
152+
- traefik.tcp.routers.rathole-local-secure.service=rathole-local-secure
153+
- traefik.tcp.services.rathole-local-secure.loadbalancer.server.port=5444
154+
155+
networks:
156+
proxy:
157+
external: true
158+
```
159+
160+
Let's start with the most important part: the `labels` on the `rathole` container that define load balancing on the two tunnels.
161+
162+
First, we define two HTTP routers using the `HostRegexp()` matcher. It takes HTTP traffic from the entrypoint on port `80` and load balances it between two tunnels on ports `5080` and `5081`.
163+
164+
The second pair of labels defines a TCP router that takes traffic from the HTTPS entrypoint on port `443`, passes it through without decrypting, and load balances it between tunnels on ports `5443` and `5444`. Note that with the `HostSNIRegexp()` matcher, you can't include escaped dots (`.`) in the regex, you must repeat the entire domain sequence to handle the `www` variant of the domain.
165+
166+
Also note that we use separate regex variants to match the root subdomain explicitly, e.g. `pi.nemanjamitic.com` and `www.pi.nemanjamitic.com` for both HTTP and TCP routers.
167+
168+
That's it, this is the main load balancing logic definition.
169+
170+
**Note:** Because we use `HostRegexp()` and `HostSNIRegexp()` on the server, you will need to use `Host()` and `HostSNI()` matchers **for the Traefik running on the client side of the tunnel**, or you will get `404` errors without additional configuration. Regex matchers on both the server and client sides seem to be too loose.
171+
172+
### Rathole server config
173+
174+
Now it's just left to write the config for the Rathole server that defines 2×2 tunnels. Just make sure to use **a different token and port** for each tunnel.
175+
176+
```toml
177+
# rathole.server.toml
178+
179+
[server]
180+
bind_addr = "0.0.0.0:2333"
181+
182+
[server.transport]
183+
type = "noise"
184+
185+
[server.transport.noise]
186+
local_private_key = "private_key"
187+
188+
# separated based on token, also can NOT use same ports
189+
190+
# pi
191+
[server.services.pi-traefik-http]
192+
token = "secret_token_1"
193+
bind_addr = "0.0.0.0:5080"
194+
195+
[server.services.pi-traefik-https]
196+
token = "secret_token_1"
197+
bind_addr = "0.0.0.0:5443"
198+
199+
# local
200+
[server.services.local-traefik-http]
201+
token = "secret_token_2"
202+
bind_addr = "0.0.0.0:5081"
203+
204+
[server.services.local-traefik-https]
205+
token = "secret_token_2"
206+
bind_addr = "0.0.0.0:5444"
207+
```
208+
209+
**Reminder:** You just need to open port `2333` in the VPS firewall for the Rathole control channel and not for the ports `5080`, `5081`, `5443`, or `5444`, because they are used by Rathole internally.
210+
211+
### Traefik dashboard
212+
213+
Additionally, for the sake of debugging, we expose the Traefik dashboard using `labels` on the `traefik` container. To simplify the configuration and avoid handling the `acme.json` file, we expose it using HTTP.
214+
215+
**Warning:** When setting the dashboard hashed password via the `TRAEFIK_AUTH` environment variable, make sure to escape the `$` characters properly or authentication will break. To do that, you need to use both double quotes `"..."` and the escape slash '`\`', as shown in the example below:
216+
217+
```bash
218+
# install apache2-utils
219+
sudo apt install apache2-utils
220+
221+
# hash the password
222+
htpasswd -nb admin yourpassword
223+
```
224+
225+
```bash
226+
# .env
227+
228+
# use BOTH "..." and \$ to escape $ properly
229+
230+
# this will work correctly
231+
TRAEFIK_AUTH="admin:\$asd1\$E3lsdAo\$3Mertp57JJ4LVU.HRR0"
232+
233+
# this will break
234+
TRAEFIK_AUTH="admin:$asd1$E3lsdAo$3Mertp57JJ4LVU.HRR0"
235+
236+
# this will also break
237+
TRAEFIK_AUTH=admin:\$asd1\$E3lsdAo\$3Mertp57JJ4LVU.HRR0
238+
```
239+
240+
## Rathole client
241+
242+
The client part of the tunnel is almost the same as for a single home server. The only thing to keep in mind is to bind the specific client only to the tunnels that are meant for it, and not to all tunnels. Kind of obvious and self-explanatory, but just in case, let's be very clear and explicit.
243+
244+
Here, we define the `rathole.client.toml` Rathole client config to bind the `pi` home server to its HTTP `pi-traefik-http` and HTTPS `pi-traefik-https` tunnels.
245+
246+
```toml
247+
# rathole.client.toml
248+
249+
[client]
250+
remote_addr = "123.123.123.123:2333"
251+
252+
[client.transport]
253+
type = "noise"
254+
255+
[client.transport.noise]
256+
remote_public_key = "public_key"
257+
258+
# single client per tunnels pair
259+
260+
# pi
261+
[client.services.pi-traefik-http]
262+
token = "secret_token_1"
263+
local_addr = "traefik:80"
264+
265+
[client.services.pi-traefik-https]
266+
token = "secret_token_1"
267+
local_addr = "traefik:443"
268+
```
269+
270+
Similarly, here we define the `rathole.client.toml` config to bind the `local` home server to it's HTTP `local-traefik-http` and HTTPS `local-traefik-https` tunnels.
271+
272+
```toml
273+
# rathole.client.toml
274+
275+
[client]
276+
remote_addr = "123.123.123.123:2333"
277+
278+
[client.transport]
279+
type = "noise"
280+
281+
[client.transport.noise]
282+
remote_public_key = "public_key"
283+
284+
# single client per tunnels pair
285+
286+
# local
287+
[client.services.local-traefik-http]
288+
token = "secret_token_2"
289+
local_addr = "traefik:80"
290+
291+
[client.services.local-traefik-https]
292+
token = "secret_token_2"
293+
local_addr = "traefik:443"
294+
```
295+
296+
`docker-compose.yml` for the Rathole client and Traefik is exactly the same as it was for a single home server. I am repeating it here just for the sake of completeness.
297+
298+
```yml
299+
# docker-compose.yml
300+
301+
version: '3.8'
302+
303+
services:
304+
rathole:
305+
image: rapiz1/rathole:v0.5.0
306+
container_name: rathole
307+
command: --client /config/rathole.client.toml
308+
restart: unless-stopped
309+
volumes:
310+
- ./rathole.client.toml:/config/rathole.client.toml:ro
311+
networks:
312+
- proxy
313+
314+
traefik:
315+
image: 'traefik:v2.9.8'
316+
container_name: traefik
317+
restart: unless-stopped
318+
depends_on:
319+
- rathole
320+
command:
321+
# moved from static conf to pass email as env var
322+
- '--certificatesresolvers.letsencrypt.acme.email=${TRAEFIK_LETSENCRYPT_EMAIL}'
323+
security_opt:
324+
- no-new-privileges:true
325+
networks:
326+
- proxy
327+
# rathole will pass traffic through proxy network directly on 80 and 443
328+
# defined in rathole.client.toml
329+
environment:
330+
- TRAEFIK_AUTH=${TRAEFIK_AUTH}
331+
volumes:
332+
- /etc/localtime:/etc/localtime:ro
333+
- /var/run/docker.sock:/var/run/docker.sock:ro
334+
- ./traefik-data/traefik.yml:/traefik.yml:ro
335+
- ./traefik-data/acme.json:/acme.json
336+
- ./traefik-data/configurations:/configurations
337+
labels:
338+
- 'traefik.enable=true'
339+
- 'traefik.docker.network=proxy'
340+
- 'traefik.http.routers.traefik-secure.entrypoints=websecure'
341+
- 'traefik.http.routers.traefik-secure.rule=Host(`traefik.${SITE_HOSTNAME}`)'
342+
- 'traefik.http.routers.traefik-secure.middlewares=user-auth@file'
343+
- 'traefik.http.routers.traefik-secure.service=api@internal'
344+
345+
networks:
346+
proxy:
347+
external: true
348+
```
349+
350+
## Completed code
351+
352+
- **Traefik load balancer and Rathole server:** https://github.com/nemanjam/rathole-server
353+
- **Rathole client and local Traefik:** https://github.com/nemanjam/traefik-proxy/tree/main/core
354+
355+
## Conclusion
356+
357+
You can use this setup to expose as many home servers as you want, in a cost-effective and practical way, as long as your VPS has enough network bandwidth to support their traffic. It can bring your homelab to another level.
358+
359+
What tool and method did you use to expose your home servers to the internet? Do you like this approach, are you willing to give it a try? Let me know in the comments.
360+
361+
Happy self-hosting.
362+
363+
## References
364+
365+
- Traefik `v2.9` `HostRegexp` reference: https://doc.traefik.io/traefik/v2.9/routing/routers/#rule
366+
- Traefik `v2.9` `HostSNIRegexp` reference: https://doc.traefik.io/traefik/v2.9/routing/routers/#rule_1
367+
- TLS Server Name Indication (SNI), Wikipedia https://en.wikipedia.org/wiki/Server_Name_Indication

0 commit comments

Comments
 (0)