Skip to content

Commit d7d38cf

Browse files
gsanchiettigsanchietti
committed
feat(ns-ha): handle wg interfaces, ipsec interfaces, routes
1 parent 358c9b6 commit d7d38cf

File tree

8 files changed

+295
-1
lines changed

8 files changed

+295
-1
lines changed

‎packages/ns-ha/Makefile‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,11 @@ define Package/ns-ha/install
4141
$(INSTALL_DIR) $(1)/usr/libexec
4242
$(INSTALL_DIR) $(1)/etc/keepalived/scripts/
4343
$(INSTALL_BIN) ./files/keepalived-config $(1)/usr/sbin
44+
$(INSTALL_BIN) ./files/ns-ha-disable $(1)/usr/sbin
45+
$(INSTALL_BIN) ./files/ns-ha-enable $(1)/usr/sbin
46+
$(INSTALL_BIN) ./files/ns-ha-export $(1)/usr/sbin
47+
$(INSTALL_BIN) ./files/ns-ha-import $(1)/usr/sbin
48+
$(INSTALL_DATA) ./files/400-network $(1)/etc/hotplug.d/keepalived
4449
$(INSTALL_DATA) ./files/500-nathelpers $(1)/etc/hotplug.d/keepalived
4550
$(INSTALL_DATA) ./files/500-netmap $(1)/etc/hotplug.d/keepalived
4651
$(INSTALL_DATA) ./files/560-mac-binding $(1)/etc/hotplug.d/keepalived

‎packages/ns-ha/README.md‎

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,48 @@ This package is a set of scripts to configure a high availability firewall.
44
Configured with keepalived, it will provide a failover mechanism between two nodes.
55

66
Requirements:
7+
- 2 nodes with similar hardware
78
- nodes must be connected to the same LAN
89
- nodes must have a dedicated interface for the HA configuration
910
- nodes must have only one WAN interface configured with DHCP
1011

12+
Limitations:
13+
14+
- WAN must be configured in DHCP
15+
- extra packages like NUT are not supported
16+
- rsyslog configuration is not synced: if you need to send logs to a remote server, you must use the controller
17+
- hotspot is not supported since it requires a new registration when the master node goes down because the MAC address associated to the hotspot interface will be different
18+
19+
The following features are supported:
20+
21+
- Firewall rules, including port forwarding
22+
- DHCP and DNS server
23+
- SSH server (dropbear)
24+
- OpenVPN RoadWarrior and tunnels
25+
- IPsec tunnels (strongwan)
26+
- WireGuard tunnels
27+
- Static routes
28+
- QoS (qosify)
29+
- Multi-WAN (mwan3)
30+
- DPI rules
31+
- Netifyd informatics configuration
32+
- Threat shield IP (banip)
33+
- Threat shield DNS (adblock)
34+
- Reverse proxy (nginx)
35+
- ACME certificates
36+
- Users and objects database
37+
- Netmap
38+
- Flashstart
39+
- SNMP server (snmpd)
40+
- NAT helpers
41+
- Dynamic DNS (ddns)
42+
- SMTP client (msmtp)
43+
- Backup encryption password
44+
- Controller connection and subscription (ns-plug)
45+
- Active connections tracking (conntrackd) - NOT tested
46+
47+
## Configuration
48+
1149
The setup process will configure all the following:
1250
- create a new firewall zone `ha`
1351
- configure the HA interface, the one dedicated for the HA traffic
@@ -56,3 +94,41 @@ In this example:
5694
/etc/init.d/firewall restart
5795
/etc/init.d/keepalived restart
5896
```
97+
98+
## How it works
99+
100+
The HA is always composed by two nodes: one is the master and the other is the backup.
101+
All configuration must be node always on the master node.
102+
The configuration is then automatically synchronized to the backup node.
103+
104+
The keepalived configuration uses a special crafted rsync script named `/etc/keepalived/scripts/ns-rsync.sh`.
105+
106+
The script is executed on the primary node, when it is master, at regular intervals and it will:
107+
- export WireGuard interfaces, IPsec interfaces and routes to a special directory named `/etc/ha`
108+
- synchronize all files listed inside by `sysupgrade -l` and all files added with the `add_sync_file` option from scripts inside `/etc/hotplug.d/keepalived` directory;
109+
files are synchronized to backup node inside the directory `/usr/share/keepalived/rsync/`
110+
111+
The hotplug `keepalived` event is used to inform the system about changes in the keepalived status.
112+
113+
The event is triggered with an `ACTION` parameter that can be:
114+
115+
- `NOTIFY_SYNC`: the script is executed on the backup node, after a sync has been done and a listed file is changed
116+
During this phase all directories (like `/etc/openvpn` and `/etc/ha`) are synched to the original position.
117+
Also WireGuard interfaces, IPsec interfaces and routes are imported from the `/etc/ha` directory but in disabled state.
118+
119+
- `NOTIFY_MASTER`: the script can be executed both on the master and on the backup node:
120+
- on the master node, after keepalived is started: this is the normal startup state
121+
- on the backup node, after an switchover has been done: this is the failover state;
122+
all WireGuard interfaces, IPsec interfaces and routes previously imported from the `/etc/ha` are enabled if they were enabled on the master node
123+
124+
- `NOTIFY_BACKUP`: the script is executed on the backup node, after keepalived is started or if the master returns up after a downtime
125+
All non required services are disabled, including WireGuard interfaces, IPsec interfaces and routes.
126+
127+
The backup node keeps the configuration in sync with the master node, but most services, including crontabs, are disabled.
128+
The following cronjobs are disabled on the backup node and enabled on the master node:
129+
130+
- subscription heartbeat
131+
- subscription inventory
132+
- phonehome
133+
- remote reports to the controller
134+
- remote backup

‎packages/ns-ha/files/400-network‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
#!/bin/sh
2+
3+
. /lib/functions/keepalived/hotplug.sh
4+
5+
set_service_name network_files
6+
7+
if [ "$ACTION" == "NOTIFY_MASTER" ]; then
8+
if [ "$(/usr/libexec/rpcd/ns.ha call status | jq .role)" == "backup" ]; then
9+
/usr/sbin/ns-ha-enable
10+
fi
11+
elif [ "$ACTION" == "NOTIFY_SYNC" ]; then
12+
home=$(get_rsync_user_home)
13+
rsync -avr $home/etc/ha/ /etc/ha/
14+
/usr/sbin/ns-ha-import
15+
elif [ "$ACTION" == "NOTIFY_BACKUP" ]; then
16+
/usr/sbin/ns-ha-disable
17+
fi
18+
19+
keepalived_hotplug

‎packages/ns-ha/files/ns-ha-disable‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
#!/usr/bin/python3
2+
3+
#
4+
# Copyright (C) 2025 Nethesis S.r.l.
5+
# SPDX-License-Identifier: GPL-2.0-only
6+
#
7+
8+
import os
9+
import json
10+
import subprocess
11+
from euci import EUci
12+
13+
out_dir = "/etc/ha"
14+
15+
def disable_interfaces(file):
16+
u = EUci()
17+
with open(os.path.join(out_dir, file), 'r') as f:
18+
interfaces = json.load(f)
19+
for interface in interfaces.keys():
20+
u.set('network', interface, 'disabled', '1')
21+
u.commit('network')
22+
23+
def disable_routes():
24+
u = EUci()
25+
with open(os.path.join(out_dir, 'routes'), 'r') as f:
26+
routes = json.load(f)
27+
for route in routes.keys():
28+
u.set('network', route, 'disabled', '1')
29+
u.commit('network')
30+
31+
if __name__ == "__main__":
32+
disable_interfaces('wg_interfaces')
33+
disable_interfaces('ipsec_interfaces')
34+
disable_routes()
35+
subprocess.run(["/sbin/reload_config"], capture_output=True)

‎packages/ns-ha/files/ns-ha-enable‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
#!/usr/bin/python3
2+
3+
#
4+
# Copyright (C) 2025 Nethesis S.r.l.
5+
# SPDX-License-Identifier: GPL-2.0-only
6+
#
7+
8+
import os
9+
import json
10+
import subprocess
11+
from euci import EUci
12+
13+
out_dir = "/etc/ha"
14+
15+
def enable_interfaces(file):
16+
u = EUci()
17+
with open(os.path.join(out_dir, file), 'r') as f:
18+
interfaces = json.load(f)
19+
for interface, options in interfaces.items():
20+
if options.get('disabled', '0') == '0':
21+
u.set('network', interface, 'disabled', '0')
22+
u.commit('network')
23+
24+
def enable_routes():
25+
u = EUci()
26+
with open(os.path.join(out_dir, 'routes'), 'r') as f:
27+
routes = json.load(f)
28+
for route, options in routes.items():
29+
if options.get('disabled', '0') == '0':
30+
u.set('network', route, 'disabled', '0')
31+
u.commit('network')
32+
33+
if __name__ == "__main__":
34+
enable_interfaces('wg_interfaces')
35+
enable_interfaces('ipsec_interfaces')
36+
enable_routes()
37+
subprocess.run(["/sbin/reload_config"], capture_output=True)

‎packages/ns-ha/files/ns-ha-export‎

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
#!/usr/bin/python3
2+
3+
#
4+
# Copyright (C) 2025 Nethesis S.r.l.
5+
# SPDX-License-Identifier: GPL-2.0-only
6+
#
7+
8+
# Export the folloing network configuration to /etc/ha:
9+
# - routes
10+
# - ipsec interfaces
11+
# - wireguard interfaces
12+
# - wireguard peers
13+
# This configuration will be imported as disabled on the backup node
14+
15+
import os
16+
import json
17+
from euci import EUci
18+
from nethsec import utils
19+
20+
out_dir = "/etc/ha"
21+
22+
def export_routes():
23+
routes = {}
24+
u = EUci()
25+
for route in utils.get_all_by_type(u, 'network', 'route'):
26+
routes[route] = u.get_all('network', route)
27+
28+
with open(os.path.join(out_dir, 'routes'), 'w') as f:
29+
json.dump(routes, f)
30+
31+
def export_ipsec_interfaces():
32+
ipsec_interfaces = {}
33+
u = EUci()
34+
for interface in utils.get_all_by_type(u, 'network', 'interface'):
35+
if interface.startswith('ipsec'):
36+
ipsec_interfaces[interface] = u.get_all('network', interface)
37+
38+
with open(os.path.join(out_dir, 'ipsec_interfaces'), 'w') as f:
39+
json.dump(ipsec_interfaces, f)
40+
41+
def export_wireguard_interfaces():
42+
wireguard_interfaces = {}
43+
u = EUci()
44+
for interface in utils.get_all_by_type(u, 'network', 'interface'):
45+
if interface.startswith('wg'):
46+
wireguard_interfaces[interface] = u.get_all('network', interface)
47+
48+
with open(os.path.join(out_dir, 'wg_interfaces'), 'w') as f:
49+
json.dump(wireguard_interfaces, f)
50+
51+
def export_wireguard_peers():
52+
wireguard_peers = {}
53+
u = EUci()
54+
for section in u.get_all('network'):
55+
if u.get('network', section).startswith('wireguard_'):
56+
wireguard_peers[section] = u.get_all('network', section)
57+
58+
with open(os.path.join(out_dir, 'wg_peers'), 'w') as f:
59+
json.dump(wireguard_peers, f)
60+
61+
62+
if __name__ == '__main__':
63+
os.makedirs(out_dir, exist_ok=True)
64+
export_routes()
65+
export_ipsec_interfaces()
66+
export_wireguard_interfaces()
67+
export_wireguard_peers()

‎packages/ns-ha/files/ns-ha-import‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
#!/usr/bin/python3
2+
3+
#
4+
# Copyright (C) 2025 Nethesis S.r.l.
5+
# SPDX-License-Identifier: GPL-2.0-only
6+
#
7+
8+
# Import the network configuration exported by the master node but in a disabled state
9+
10+
import os
11+
import json
12+
from euci import EUci
13+
14+
out_dir = "/etc/ha"
15+
16+
def import_interfaces(file):
17+
u = EUci()
18+
with open(os.path.join(out_dir, file), 'r') as f:
19+
interfaces = json.load(f)
20+
for interface, options in interfaces.items():
21+
u.set('network', interface, 'interface')
22+
for opt in options:
23+
u.set('network', interface, opt, options[opt])
24+
u.set('network', interface, 'disabled', '1')
25+
u.commit('network')
26+
27+
def import_wireguard_peers():
28+
u = EUci()
29+
with open(os.path.join(out_dir, 'wg_peers'), 'r') as f:
30+
peers = json.load(f)
31+
for section, options in peers.items():
32+
stype = "wireguard_"+section.split("_")[0]
33+
u.set('network', section, stype)
34+
for opt in options:
35+
u.set('network', section, opt, options[opt])
36+
u.commit('network')
37+
38+
def import_routes():
39+
u = EUci()
40+
with open(os.path.join(out_dir, 'routes'), 'r') as f:
41+
routes = json.load(f)
42+
for section, options in routes.items():
43+
u.set('network', section, 'route')
44+
for opt in options:
45+
u.set('network', section, opt, options[opt])
46+
u.set('network', section, 'disabled', '1')
47+
u.commit('network')
48+
49+
if __name__ == "__main__":
50+
import_interfaces('wg_interfaces')
51+
import_wireguard_peers()
52+
import_interfaces('ipsec_interfaces')
53+
import_routes()

‎packages/ns-ha/files/ns-rsync.sh‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@ ha_sync_send() {
3333
local address ssh_key ssh_port sync_list sync_dir sync_file count
3434
local ssh_options ssh_remote dirs_list files_list
3535
local changelog="/tmp/changelog"
36+
local ha_export="/etc/ha"
3637

3738
config_get address "$cfg" address
3839
[ -z "$address" ] && return 0
@@ -80,7 +81,7 @@ ha_sync_send() {
8081
fi
8182

8283
# shellcheck disable=SC2086
83-
rsync -a --relative ${files_list} ${changelog} -e "ssh $ssh_options" --rsync-path="sudo rsync" "$ssh_remote":"$sync_dir" || {
84+
rsync -a --relative ${files_list} ${ha_export} ${changelog} -e "ssh $ssh_options" --rsync-path="sudo rsync" "$ssh_remote":"$sync_dir" || {
8485
log_err "rsync transfer failed for $address"
8586
update_last_sync_time "$cfg"
8687
update_last_sync_status "$cfg" "Rsync Transfer Failed"
@@ -151,6 +152,7 @@ main() {
151152
return 1
152153
fi
153154

155+
/usr/sbin/ns-ha-export
154156
config_load keepalived
155157
config_foreach ha_sync vrrp_instance
156158

0 commit comments

Comments
 (0)