|
| 1 | +# Failover Mechanisms |
| 2 | + |
| 3 | +The project has a few failure points listed below that we explicitly want to control instead and try to recover from. |
| 4 | + |
| 5 | +Our goal if something breaks, is to try to recover automatically where possible, but if recovery is not possible or the failure is critical, the user must lose internet so they are forced to debug and restart Slips manually instead of staying connected without Slips protection. |
| 6 | + |
| 7 | + |
| 8 | +All failure points are handled by the ```failover_handler.sh``` script located in ```StratosphereLinuxIPS/rpi_scripts/```. |
| 9 | + |
| 10 | +## Prerequisites |
| 11 | + |
| 12 | +- Raspberry Pi with docker installed |
| 13 | +- StratosphereLinuxIPS cloned. (or just the rpi_scripts/ directory) |
| 14 | +- Root access to the Raspberry Pi. |
| 15 | +- A [running access point](https://stratospherelinuxips.readthedocs.io/en/develop/immune/installing_slips_in_the_rpi.html#protect-your-local-network-with-slips-on-the-rpi). |
| 16 | + |
| 17 | + |
| 18 | +## How to use |
| 19 | + |
| 20 | +Run the following command from Slips main directory as root: |
| 21 | + |
| 22 | +```bash |
| 23 | +sudo ./rpi_scripts/failover_handler.sh <wifi_interface>,<ethernet_interface> |
| 24 | +``` |
| 25 | + |
| 26 | +**Where** |
| 27 | + |
| 28 | +- ```<wifi_interface>``` is the name of the wifi interface used by the access point (e.g. ```wlan0```, etc). |
| 29 | + |
| 30 | +- ```<ethernet_interface>``` is the name of the ethernet interface connected to the router (e.g. ```eth0```, etc). |
| 31 | + |
| 32 | +**You should see output similar to the image below:** |
| 33 | + |
| 34 | + |
| 35 | + |
| 36 | + |
| 37 | +**Output:** |
| 38 | + |
| 39 | +The script will |
| 40 | +- Log Slips docker container status, used command, and any errors to ```slips_container.log``` for debugging purposes. This file should be checked in case you notice any issues with the AP or Slips. |
| 41 | +- Start Slips and iptables watcher services through systemd so they start automatically on reboot and on failure. |
| 42 | +- Start slips inside a docker container monitring your ethernet and wifi interfaces. |
| 43 | +- Mount your local ```StratosphereLinuxIPS/output``` to ```/StratosphereLinuxIPS/output``` inside the started Slips container so any output generated by Slips will be available on the host machine. |
| 44 | + |
| 45 | + |
| 46 | + |
| 47 | +## How Failovers are Handled |
| 48 | + |
| 49 | +Slips in the Raspberry Pi has 3 main failure points that we want to handle: |
| 50 | +1. The access point dies |
| 51 | +2. Slips dies |
| 52 | +3. The Raspberry Pi reboots |
| 53 | + |
| 54 | +### The access point dies |
| 55 | + |
| 56 | +If the AP dies, clients get disconnected. There's nothing to handle here. Slips keeps monitoring the ethernet interface and when the AP is back up, clients can reconnect and Slips continues protecting them. |
| 57 | + |
| 58 | + |
| 59 | +### Slips dies |
| 60 | + |
| 61 | +Failovers when Slips dies consist of: |
| 62 | + |
| 63 | +* **Iptables firewall rules persistence**: The iptables firewall rules are saved periodically by the systemd unit that watches for iptables changes and saves them using ```netfilter-persistent``` whenever a change is detected. |
| 64 | +* **Shutting down the AP process**: If Slips crashes we do not want the AP to keep running without Slips protection so we intentionally shut down the access point for the user to notice, debug and restart Slips manually instead of staying connected without Slips protection. |
| 65 | +* **Restarting Slips automatically through systemd:** Slips restarts automatically through systemd on failure and on reboot. |
| 66 | +* **Logging Slips container status to a file for debugging**: Slips container logs are places in ```slips_container.log``` for debugging purposes. |
| 67 | + |
| 68 | + |
| 69 | + |
| 70 | +### The Raspberry Pi reboots |
| 71 | + |
| 72 | +Failovers consist of: |
| 73 | + |
| 74 | +* **Iptables firewall rules persistence** |
| 75 | +* **Automatic start of Slips service through systemd** |
| 76 | + |
| 77 | + |
| 78 | +When the Pi reboots, we want Slips to start automatically, and we want the iptables rules added by Slips to persist. The automatic restart is handled by systemd through the generated ```slips.service``` file, and firewall persistence is handled using the custom iptables watcher through the generated ```iptables-watcher.service```. |
| 79 | + |
| 80 | + |
| 81 | +Both units are generated and started and added to the user's ```/etc/systemd/system``` by the ```failover_handler.sh``` script. |
| 82 | + |
| 83 | +--- |
| 84 | + |
| 85 | +## File Descriptions |
| 86 | + |
| 87 | +All the files involved in failover mechanisms are placed in ```StratosphereLinuxIPS/rpi_scripts/``` and are described in the table below: |
| 88 | + |
| 89 | + |
| 90 | +| File | What it does | |
| 91 | +|-------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| |
| 92 | +| failover_handler.sh | The central orchestrator that checks AP status, ensures root access, prepares directories, sets up iptables persistence, builds the Slips runner script, generates the systemd unit, and enforces all failover behavior. This is the piece that links every component together and decides how the system should react when something breaks. | |
| 93 | +| iptables_autosave/check-iptables-hash.sh | Keeps track of the hash of the current iptables rules and triggers a save when change is detected. | |
| 94 | +| iptables_autosave/iptables-watcher.service | The systemd service that runs check-iptables-hash.sh (because we can't run the script directly by the timer), this is triggered by the iptables-watcher.timer every 10s to check for iptables changes. | |
| 95 | +| iptables_autosave/iptables-watcher.timer | A systemd timer that periodically runs iptables-watcher.service so iptables rule changes are captured and saved automatically. | |
| 96 | +| slips_container.log | A runtime log collecting Docker container output, commands, and status. useful for investigating restarts, failures, or unexpected behavior. | |
| 97 | +| slips-runner-template.sh | The script that launches Slips container and launches slips inside of it in a tmux. This runner keeps the container up as long as Slips is running. | |
| 98 | +| slips.service.template | The systemd unit that starts slips on reboot and on failure, it runs the slips-runner-template.sh. | |
| 99 | + |
| 100 | + |
| 101 | +--- |
0 commit comments