From 0a4012fe55ef8b363cc79a8cbf9b0955754ee15b Mon Sep 17 00:00:00 2001 From: ukkopahis <95980324+ukkopahis@users.noreply.github.com> Date: Sun, 12 Dec 2021 11:09:19 +0200 Subject: [PATCH 01/24] Grafana instructions for adding influxdb datasource Using localhost in the url doesn't work, so took me a while to figure out the correct URL. --- docs/Containers/Grafana.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/docs/Containers/Grafana.md b/docs/Containers/Grafana.md index 824dd8e8..85e64bb6 100644 --- a/docs/Containers/Grafana.md +++ b/docs/Containers/Grafana.md @@ -15,6 +15,16 @@ The default *~/IOTstack/services/grafana/grafana.env* contains this line: Uncomment that line and change the right hand side to [your own timezone](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones). +## Adding InfluxDB datasource + +Select Data Sources -> Add data source -> InfluxDB. + +Set options: + +* HTTP / URL: `http://influxdb:8086` +* InfluxDB Details / Database: `telegraf` +* InfluxDB Details / User: `nodered` +* InfluxDB Details / Password: `nodered` ## Security From f897bf1409026dca8d64e156cf0f81eb51ec0df3 Mon Sep 17 00:00:00 2001 From: ukkopahis <95980324+ukkopahis@users.noreply.github.com> Date: Thu, 16 Dec 2021 03:34:14 +0200 Subject: [PATCH 02/24] Pi-hole: docs to setup DNS for esphome devices --- docs/Containers/Pi-hole.md | 70 +++++++++++++++++++++++++++++++++++--- 1 file changed, 66 insertions(+), 4 deletions(-) diff --git a/docs/Containers/Pi-hole.md b/docs/Containers/Pi-hole.md index 7842f79b..038c7b84 100644 --- a/docs/Containers/Pi-hole.md +++ b/docs/Containers/Pi-hole.md @@ -1,8 +1,70 @@ # Pi-hole -Pi-hole is a fantastic utility to reduce ads +Pi-hole is a fantastic utility to reduce ads. -The interface can be found on `"your_ip":8089/admin` +The interface can be found on `http://"your_ip":8089/admin` -Default password is `pihole`. This can be changed in the `~/IOTstack/services/pihole/pihole.env` file +Default password is `IOtSt4ckP1Hol3`. This can be changed with: +``` +docker exec pihole pihole -a -p myNewPassword +``` -To enable your router to use the pihole container edit your DNS settings on your router to point to your Pi's IP address \ No newline at end of file +## Pi-hole as DNS server + +In order for the Pi-hole to work, it needs to be defined as the DNS server. +This can either be done manually to each device or you can define it as a +DNS-nameserver for the whole LAN. + +Assuming your pihole hostname is `raspberrypi` and has the IP `192.168.1.10`: + +1. Go to the Pi-hole web interface: `http://raspberrypi.local:8089/admin` and Login + * From Left menu: Select Local DNS -> DNS Records + * Enter Domain: `raspberrypi.home.arpa` and IP Address: `192.168.1.10`. Press Add. +2. Go to your DHCP server, usually this is your Wireless Access Point / WLAN Router web interface. + * Find where DNS servers are defined + * Change all DNS fields to `192.168.1.10`. +3. All local machines have to be rebooted or have their DHCP leases released. Without this they will continue to use the old DNS setting from an old DHCP lease for quite some time. + +Now you can use `raspberrypi.home.arpa` as the domain name for the Raspberry Pi in your whole local network. + +For the Raspberry Pi itself to also use the Pi-hole DNS server, run: +```bash +echo "name_servers=127.0.0.1" | sudo tee -a /etc/resolvconf.conf +echo "name_servers_append=8.8.8.8" | sudo tee -a /etc/resolvconf.conf +echo "resolv_conf_local_only=NO" | sudo tee -a /etc/resolvconf.conf +sudo resolvconf -u # Ignore "Too few arguments."-complaint +``` +Quick explanation: resolv_conf_local_only is disabled and a public nameserver is added, so that in case the Pi-hole container is stopped, the Raspberry won't lose DNS functionality. It will just fallback to 8.8.8.8. + +### Testing & Troubleshooting + +Install dig: +``` +apt install dnsutils +``` + +Test that pi-hole is correctly configured (should respond 192.168.1.10): +``` +dig raspberrypi.home.arpa @192.168.1.10 +``` + +To test on your desktop if your network configuration is correct, and an ESP will resolve its DNS queries correctly, restart your desktop machine to ensure DNS changes are updated and then use: +``` +dig raspberrypi.home.arpa +``` +This should produce the same result as the previous command. + +If this fails to resolve the IP, check that the server in the response is `192.168.1.10`. +If it's `127.0.0.xx` check `/etc/resolv.conf` begins with `nameserver 192.168.1.10`. + +## Microcontrollers + +If you want to avoid hardcoding your Raspberry Pi IP to your ESPhome devices, +you need a DNS server that will do the resolving. This can be done using the +Pi-hole container as described above. + +## Why .home.arpa? + +Instead of `.home.arpa` - which is the real standard, but a mouthful - you may use `.internal`. +Using `.local` would technically also work, but it should be reserved only for mDNS use. + +Note: There is a special case for resolving `*.local` addresses. If you do a `ping raspberrypi.local` on your desktop linux or the RPI, it will first try using mDNS/bonjour to resolve the IP address raspberrypi.local. If this fails it will then ask the DNS server. Esphome devices can't use mDNS to resolve an IP address. You need a proper DNS server to respond to queries made by an ESP. As such, `dig raspberrypi.local` will fail, simulating ESPhome device behavior. This is as intended, and you should use raspberrypi.home.arpa as the address on your ESP-device. From c06c08af8c82b7d7422c7327f67e0eda38f8ba8c Mon Sep 17 00:00:00 2001 From: ukkopahis <95980324+ukkopahis@users.noreply.github.com> Date: Fri, 17 Dec 2021 21:29:30 +0200 Subject: [PATCH 03/24] Fix docs on how to update containers --- docs/Updating-the-Project.md | 30 +++++++++++++++++++++++++++--- 1 file changed, 27 insertions(+), 3 deletions(-) diff --git a/docs/Updating-the-Project.md b/docs/Updating-the-Project.md index 5d0aeff0..5b495e66 100644 --- a/docs/Updating-the-Project.md +++ b/docs/Updating-the-Project.md @@ -1,9 +1,33 @@ # Updating the project -**If you ran the git checkout -- 'git ls-files -m' as suggested in the old wiki entry then please check your duck.sh because it removed your domain and token** - Periodically updates are made to project which include new or modified container template, changes to backups or additional features. As these are released your local copy of this project will become out of date. This section deals with how to bring your project to the latest published state. +Quick instructions: + +1. backup your current settings: `cp docker-compose.yml docker-compose.yml.bak` +2. check `git status` for any local changes you may have made to project files. Save and preserve your changes by doing a commit: `git commit -a -m "local customization"`. Or revert them using: `git checkout -- path/to/changed_file`. +3. update project files from github: `git pull origin master -r` +4. get latest images from the web: `docker-compose pull` +5. rebuild localy created images from new Dockerfiles: `docker-compose build --pull --no-cache` +6. update running containers to latest: `docker-compose up --build -d` + +*Troubleshooting:* if a container fails to restart after update +* try restarting the whole stack: `docker-compose restart` +* backup your stack settings: `cp docker-compose.yml docker-compose.yml.bak` +* Check log output of the failing service: `docker-compose logs *service-name*` + * try googling and fixing problems in docker-compose.yml manually. +* try recreating the failing service definition using menu.sh: + * `./menu.sh`, select Build Stack, unselect the failing service, press enter + to build, and then exit. + * `./menu.sh`, select Build Stack, select the service back again, press enter + to build, and then exit. + * Try starting now: `docker-compose up -d` +* Go to the IOTStack Discord and ask for help. + +## Details, partly outdated + +**If you ran the git checkout -- 'git ls-files -m' as suggested in the old wiki entry then please check your duck.sh because it removed your domain and token** + Git offers build in functionality to fetch the latest changes. `git pull origin master` will fetch the latest changes from GitHub without overwriting files that you have modified yourself. If you have done a local commit then your project may to handle a merge conflict. @@ -18,4 +42,4 @@ With the new latest version of the project you can now use the menu to build you ![image](https://user-images.githubusercontent.com/46672225/68646024-8fee2f80-0522-11ea-8b6e-f1d439a5be7f.png) -After your stack had been rebuild you can run `docker-compose up -d` to pull in the latest changes. If you have not update your images in a while consider running the `./scripts/update.sh` to get the latest version of the image from Docker hub as well \ No newline at end of file +After your stack had been rebuild you can run `docker-compose up -d` to pull in the latest changes. If you have not update your images in a while consider running the `./scripts/update.sh` to get the latest version of the image from Docker hub as well From 9e97ee3d320e0a4cf46c29f8265cc1ba8fb0b8a0 Mon Sep 17 00:00:00 2001 From: ukkopahis <95980324+ukkopahis@users.noreply.github.com> Date: Sat, 11 Dec 2021 21:00:34 +0200 Subject: [PATCH 04/24] docs: fix syntax and cleanup --- docs/Backup-and-Restore.md | 6 +++ docs/Containers/MariaDB.md | 3 ++ docs/Home.md | 75 ------------------------------------ docs/Updating-the-Project.md | 21 +++++----- 4 files changed, 21 insertions(+), 84 deletions(-) delete mode 100644 docs/Home.md diff --git a/docs/Backup-and-Restore.md b/docs/Backup-and-Restore.md index 3dae4ab1..48de688a 100755 --- a/docs/Backup-and-Restore.md +++ b/docs/Backup-and-Restore.md @@ -8,6 +8,7 @@ The backup command can be executed from IOTstack's menu, or from a cronjob. To ensure that all your data is saved correctly, the stack should be brought down. This is mainly due to databases potentially being in a state that could cause data loss. There are 2 ways to run backups: + * From the menu: `Backup and Restore` > `Run backup` * Running the following command: `bash ./scripts/backup.sh` @@ -21,6 +22,7 @@ The current directory of bash must be in IOTstack's directory, to ensure that it ``` ./scripts/backup.sh {TYPE=3} {USER=$(whoami)} ``` + * Types: * 1 = Backup with Date * A tarball file will be created that contains the date and time the backup was started, in the filename. @@ -33,10 +35,12 @@ The current directory of bash must be in IOTstack's directory, to ensure that it If this parameter is not supplied when run as root, the script will ask for the username as input Backups: + * You can find the backups in the ./backups/ folder. With rolling being in ./backups/rolling/ and date backups in ./backups/backup/ * Log files can also be found in the ./backups/logs/ directory. ### Examples: + * `./scripts/backup.sh` * `./scripts/backup.sh 3` @@ -52,6 +56,7 @@ This will only produce a backup in the rollowing folder and change all the permi ## Restore There are 2 ways to run a restore: + * From the menu: `Backup and Restore` > `Restore from backup` * Running the following command: `bash ./scripts/restore.sh` @@ -64,6 +69,7 @@ There are 2 ways to run a restore: ./scripts/restore.sh {FILENAME=backup.tar.gz} {noask} ``` The restore script takes 2 arguments: + * Filename: The name of the backup file. The file must be present in the `./backups/` directory, or a subfolder in it. That means it should be moved from `./backups/backup` to `./backups/`, or that you need to specify the `backup` portion of the directory (see examples) * NoAsk: If a second parameter is present, is acts as setting the no ask flag to true. diff --git a/docs/Containers/MariaDB.md b/docs/Containers/MariaDB.md index 6f89b82b..90a3739d 100644 --- a/docs/Containers/MariaDB.md +++ b/docs/Containers/MariaDB.md @@ -1,3 +1,4 @@ +# MariaDB ## Source * [Docker hub](https://hub.docker.com/r/linuxserver/mariadb/) @@ -59,6 +60,8 @@ You can open a terminal session within the mariadb container via: $ docker exec -it mariadb bash ``` +To connect to the database: `mysql -uroot -p` + To close the terminal session, either: * type "exit" and press return; or diff --git a/docs/Home.md b/docs/Home.md deleted file mode 100644 index d21ab031..00000000 --- a/docs/Home.md +++ /dev/null @@ -1,75 +0,0 @@ -# Wiki - -The README is moving to the Wiki, It's easier to add content and example to the Wiki vs the README.md - -* [Getting Started](https://sensorsiot.github.io/IOTstack/Containers/Getting-Started) -* [Updating the project](https://sensorsiot.github.io/IOTstack/Containers/Updating-the-Project) -* [How the script works](https://sensorsiot.github.io/IOTstack/Containers/How-the-script-works) -* [Understanding Containers](https://sensorsiot.github.io/IOTstack/Containers/Understanding-Containers) - -*** - -# Docker - -* [Commands](https://sensorsiot.github.io/IOTstack/Containers/Docker-commands) -* [Docker Networks](https://sensorsiot.github.io/IOTstack/Containers/Networking) - -*** - -# Containers -* [Portainer](https://sensorsiot.github.io/IOTstack/Containers/Portainer) -* [Portainer-ce](https://sensorsiot.github.io/IOTstack/Containers/Portainer-ce) -* [Portainer Agent](https://sensorsiot.github.io/IOTstack/Containers/Portainer-agent) -* [Node-RED](https://sensorsiot.github.io/IOTstack/Containers/Node-RED) -* [Grafana](https://sensorsiot.github.io/IOTstack/Containers/Grafana) -* [Mosquitto](https://sensorsiot.github.io/IOTstack/Containers/Mosquitto) -* [PostgreSQL](https://sensorsiot.github.io/IOTstack/Containers/PostgreSQL) -* [Adminer](https://sensorsiot.github.io/IOTstack/Containers/Adminer) -* [openHAB](https://sensorsiot.github.io/IOTstack/Containers/openHAB) -* [Home Assistant](https://sensorsiot.github.io/IOTstack/Containers/Home-Assistant) -* [Pi-Hole](https://sensorsiot.github.io/IOTstack/Containers/Pi-hole) -* [zigbee2MQTT](https://sensorsiot.github.io/IOTstack/Containers/Zigbee2MQTT) -* [Plex](https://sensorsiot.github.io/IOTstack/Containers/Plex) -* [TasmoAdmin](https://sensorsiot.github.io/IOTstack/Containers/TasmoAdmin) -* [RTL_433](https://sensorsiot.github.io/IOTstack/Containers/RTL_433-docker) -* [EspruinoHub (testing)](https://sensorsiot.github.io/IOTstack/Containers/EspruinoHub) -* [Next-Cloud](https://sensorsiot.github.io/IOTstack/Containers/NextCloud) -* [MariaDB](https://sensorsiot.github.io/IOTstack/Containers/MariaDB) -* [MotionEye](https://sensorsiot.github.io/IOTstack/Containers/MotionEye) -* [Blynk Server](https://sensorsiot.github.io/IOTstack/Containers/Blynk_server) -* [diyHue](https://sensorsiot.github.io/IOTstack/Containers/diyHue) -* [Python](https://sensorsiot.github.io/IOTstack/Containers/Python) -* [Heimdall](https://sensorsiot.github.io/IOTstack/Containers/Heimdall) -* [DashMachine](https://sensorsiot.github.io/IOTstack/Containers/DashMachine) -* [Homer](https://sensorsiot.github.io/IOTstack/Containers/Homer) -* [Custom containers](https://sensorsiot.github.io/IOTstack/Containers/Custom) - -*** - -# Native installs - -* [RTL_433](https://sensorsiot.github.io/IOTstack/Containers/Native-RTL_433) -* [RPIEasy](https://sensorsiot.github.io/IOTstack/Containers/RPIEasy_native) - -*** - -# Backups - -* [Docker backups](https://sensorsiot.github.io/IOTstack/Containers/Backups) -* Recovery (coming soon) -*** - -# Remote Access - -* [VPN and Dynamic DNS](https://sensorsiot.github.io/IOTstack/Containers/Accessing-your-Device-from-the-internet) -* [x2go](https://sensorsiot.github.io/IOTstack/Containers/x2go) - -*** - -# Miscellaneous - -* [log2ram](https://sensorsiot.github.io/IOTstack/Containers/Misc) -* [Dropbox-Uploader](https://sensorsiot.github.io/IOTstack/Containers/Misc) - -*** - diff --git a/docs/Updating-the-Project.md b/docs/Updating-the-Project.md index 5b495e66..93654c18 100644 --- a/docs/Updating-the-Project.md +++ b/docs/Updating-the-Project.md @@ -2,7 +2,7 @@ Periodically updates are made to project which include new or modified container template, changes to backups or additional features. As these are released your local copy of this project will become out of date. This section deals with how to bring your project to the latest published state. -Quick instructions: +## Quick instructions 1. backup your current settings: `cp docker-compose.yml docker-compose.yml.bak` 2. check `git status` for any local changes you may have made to project files. Save and preserve your changes by doing a commit: `git commit -a -m "local customization"`. Or revert them using: `git checkout -- path/to/changed_file`. @@ -12,21 +12,24 @@ Quick instructions: 6. update running containers to latest: `docker-compose up --build -d` *Troubleshooting:* if a container fails to restart after update + * try restarting the whole stack: `docker-compose restart` * backup your stack settings: `cp docker-compose.yml docker-compose.yml.bak` * Check log output of the failing service: `docker-compose logs *service-name*` - * try googling and fixing problems in docker-compose.yml manually. + * try googling and fixing problems in docker-compose.yml manually. * try recreating the failing service definition using menu.sh: - * `./menu.sh`, select Build Stack, unselect the failing service, press enter - to build, and then exit. - * `./menu.sh`, select Build Stack, select the service back again, press enter - to build, and then exit. - * Try starting now: `docker-compose up -d` -* Go to the IOTStack Discord and ask for help. + 1. `./menu.sh`, select Build Stack, unselect the failing service, press + enter to build, and then exit. + 2. `./menu.sh`, select Build Stack, select the service back again, press + enter to build, and then exit. + 3. Try starting now: `docker-compose up -d` +* Go to the [IOTStack Discord](https://discord.gg/ZpKHnks) and describe your + problem. We're happy to help. ## Details, partly outdated -**If you ran the git checkout -- 'git ls-files -m' as suggested in the old wiki entry then please check your duck.sh because it removed your domain and token** +!!! warning + If you ran `git checkout -- 'git ls-files -m'` as suggested in the old wiki entry then please check your duck.sh because it removed your domain and token Git offers build in functionality to fetch the latest changes. From 2459ef997b761cacc9e0a549dba11f131d92f7a1 Mon Sep 17 00:00:00 2001 From: ukkopahis <95980324+ukkopahis@users.noreply.github.com> Date: Sat, 15 Jan 2022 04:15:22 +0200 Subject: [PATCH 05/24] docs: move developer documentation to subfolder * links fixed and changed to relative. Absolute links are not supported by mkdocs: https://www.mkdocs.org/user-guide/writing-your-docs/#linking-to-pages --- .../{ => Developers}/BuildStack-RandomPassword.md | 0 docs/{ => Developers}/BuildStack-Services.md | 0 docs/{ => Developers}/Contributing-Services.md | 15 ++++++++------- docs/{ => Developers}/Menu-System.md | 0 docs/PostBuild-Script.md | 2 +- docs/index.md | 2 ++ 6 files changed, 11 insertions(+), 8 deletions(-) rename docs/{ => Developers}/BuildStack-RandomPassword.md (100%) rename docs/{ => Developers}/BuildStack-Services.md (100%) rename docs/{ => Developers}/Contributing-Services.md (73%) rename docs/{ => Developers}/Menu-System.md (100%) diff --git a/docs/BuildStack-RandomPassword.md b/docs/Developers/BuildStack-RandomPassword.md similarity index 100% rename from docs/BuildStack-RandomPassword.md rename to docs/Developers/BuildStack-RandomPassword.md diff --git a/docs/BuildStack-Services.md b/docs/Developers/BuildStack-Services.md similarity index 100% rename from docs/BuildStack-Services.md rename to docs/Developers/BuildStack-Services.md diff --git a/docs/Contributing-Services.md b/docs/Developers/Contributing-Services.md similarity index 73% rename from docs/Contributing-Services.md rename to docs/Developers/Contributing-Services.md index 4f8c4ec7..7fe80dc1 100755 --- a/docs/Contributing-Services.md +++ b/docs/Developers/Contributing-Services.md @@ -9,8 +9,8 @@ Services will grow over time, we may split up the buildstack menu into subsectio * `build.py` file is correct * Service allows for changing external WUI port from Build Stack's options menu if service uses a HTTP/S port * Use a default password, or allow the user to generate a random password for the service for initial installation. If the service asks to setup an account this can be ignored. -* Ensure [Default Configs](https://sensorsiot.github.io/IOTstack/Default-Configs) is updated with WUI port and username/password. -* Must detect port confilicts with other services on [BuildStack](https://sensorsiot.github.io/IOTstack/Menu-System) Menu. +* Ensure [Default Configs](../Default-Configs.md) is updated with WUI port and username/password. +* Must detect port confilicts with other services on [BuildStack](Menu-System.md) Menu. * `Pre` and `Post` hooks work with no errors. * Does not require user to edit config files in order to get the service running. * Ensure that your service can be backed up and restored without errors or data loss. @@ -21,8 +21,9 @@ Services will grow over time, we may split up the buildstack menu into subsectio If your new service is approved and merged then congratulations! Please watch the Issues page on github over the next few days and weeks to see if any users have questions or issues with your new service. Links: -* [Default configs](https://sensorsiot.github.io/IOTstack/Default-Configs) -* [Password configuration for Services](https://sensorsiot.github.io/IOTstack/BuildStack-RandomPassword) -* [Build Stack Menu System](https://sensorsiot.github.io/IOTstack/Menu-System) -* [Coding a new service](https://sensorsiot.github.io/IOTstack/BuildStack-Services) -* [IOTstack issues](https://github.com/SensorsIot/IOTstack/issues) \ No newline at end of file + +* [Default configs](../Default-Configs.md) +* [Password configuration for Services](BuildStack-RandomPassword.md) +* [Build Stack Menu System](Menu-System.md) +* [Coding a new service](BuildStack-Services.md) +* [IOTstack issues](htps://github.com/SensorsIot/IOTstack/issues) diff --git a/docs/Menu-System.md b/docs/Developers/Menu-System.md similarity index 100% rename from docs/Menu-System.md rename to docs/Developers/Menu-System.md diff --git a/docs/PostBuild-Script.md b/docs/PostBuild-Script.md index 2bc8430e..2cda0f3c 100755 --- a/docs/PostBuild-Script.md +++ b/docs/PostBuild-Script.md @@ -2,7 +2,7 @@ The postbuild bash script allows for executing arbitrary execution of bash commands after the stack has been build. ## How to use -Place a file in the main directory called `postbuild.sh`. When the buildstack [build logic](https://sensorsiot.github.io/IOTstack/Menu-System) finishes, it'll execute the `postbuild.sh` script, passing in each service selected from the buildstack menu as a parameter. This script is run each time the buildstack logic runs. +Place a file in the main directory called `postbuild.sh`. When the buildstack [build logic](Developers/Menu-System.md) finishes, it'll execute the `postbuild.sh` script, passing in each service selected from the buildstack menu as a parameter. This script is run each time the buildstack logic runs. ## Updates The `postbuild.sh` file has been added to gitignore, so it won't be updated by IOTstack when IOTstack is updated. It has also been added to the backup script so that it will be backed up with your personal IOTstack backups. diff --git a/docs/index.md b/docs/index.md index 96f90300..f880d0fb 100644 --- a/docs/index.md +++ b/docs/index.md @@ -9,3 +9,5 @@ Welcome to the IOTstack Wiki: * If you are looking for information on a specific container, click on the "Containers" folder at the bottom of the list. * If you are just getting started with IOTstack, see [Getting Started](./Getting-Started.md). + +* To contribute see [Contributing Services](Developers/Contributing-Services.md) From 067995b44bb1703b860e8246b12b833b167211bb Mon Sep 17 00:00:00 2001 From: Willem Dekker Date: Sun, 12 Jul 2020 11:25:13 +0200 Subject: [PATCH 06/24] docs: add dark and light theme Basically contains most changes from: https://github.com/SensorsIot/IOTstack/pull/338 * Dark and light modes * Always expand left-navigation folders, improving usability --- .github/workflows/main.yml | 14 +++++++------- mkdocs.yml | 27 +++++++++++++++++++++++---- 2 files changed, 30 insertions(+), 11 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index d8887048..3c27211b 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -9,10 +9,10 @@ jobs: name: Deploy docs runs-on: ubuntu-latest steps: - - name: Checkout master - uses: actions/checkout@v1 - - - name: Deploy docs - uses: mhausenblas/mkdocs-deploy-gh-pages@master - env: - PERSONAL_TOKEN: ${{ secrets.PERSONAL_TOKEN }} + - uses: actions/checkout@v2 + - uses: actions/setup-python@v2 + with: + python-version: 3.x + - run: pip install mkdocs-material + - run: pip install mkdocs-git-revision-date-localized-plugin + - run: mkdocs gh-deploy --force diff --git a/mkdocs.yml b/mkdocs.yml index 3bd986db..57195c7b 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -1,10 +1,29 @@ site_name: IOTstack site_description: 'Docker stack for getting started on IOT on the Raspberry PI' + + +# Repository +repo_url: https://github.com/SensorsIot/IOTstack +repo_name: SensorsIot/IOTstack +edit_uri: "https://github.com/SensorsIot/IOTstack/edit/master/" + theme: name: material -plugins: - - search -# - awesome-pages + palette: + - scheme: default + toggle: + icon: material/weather-sunny + name: Switch to dark mode + - scheme: slate + toggle: + icon: material/weather-night + name: Switch to light mode + features: + - tabs + - navigation.expand + plugins: + - search + - git-revision-date-localized markdown_extensions: + - admonition - pymdownx.superfences -repo_url: https://github.com/SensorsIot/IOTstack From a81573f243a82b906104b3f966075a07556abbca Mon Sep 17 00:00:00 2001 From: ukkopahis <95980324+ukkopahis@users.noreply.github.com> Date: Sat, 15 Jan 2022 13:03:07 +0200 Subject: [PATCH 07/24] docs: fix unsupported absolute links * Only relative links withing pages are supported: https://www.mkdocs.org/user-guide/writing-your-docs/#linking-to-pages As an additional benefit, when running: $ mkdocs serve it will issues a warnings for invalid relative links --- docs/Accessing-your-Device-from-the-internet.md | 2 +- docs/Containers/NextCloud.md | 2 +- docs/Containers/WireGuard.md | 4 ++-- docs/Custom.md | 2 +- docs/Getting-Started.md | 2 +- docs/New-Menu-Release-Notes.md | 4 ++-- 6 files changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/Accessing-your-Device-from-the-internet.md b/docs/Accessing-your-Device-from-the-internet.md index 405d09ee..8ffd315f 100644 --- a/docs/Accessing-your-Device-from-the-internet.md +++ b/docs/Accessing-your-Device-from-the-internet.md @@ -152,7 +152,7 @@ $ cat /dev/null >~/Logs/duck.log ### WireGuard -WireGuard is supplied as part of IOTstack. See [WireGuard documentation](https://sensorsiot.github.io/IOTstack/Containers/WireGuard.html). +WireGuard is supplied as part of IOTstack. See [WireGuard documentation](Containers/WireGuard.md). ### PiVPN diff --git a/docs/Containers/NextCloud.md b/docs/Containers/NextCloud.md index 72c4448f..d26ff469 100644 --- a/docs/Containers/NextCloud.md +++ b/docs/Containers/NextCloud.md @@ -265,7 +265,7 @@ Nextcloud traffic is not encrypted. Do **not** expose it to the web by opening a A script , or "agent", to assess the health of the MariaDB container has been added to the *local image* via the *Dockerfile*. In other words, the script is specific to IOTstack. -Because it is an instance of MariaDB, Nextcloud_DB inherits the health-check agent. See the [IOTstack MariaDB](https://sensorsiot.github.io/IOTstack/Containers/MariaDB/) documentation for more information. +Because it is an instance of MariaDB, Nextcloud_DB inherits the health-check agent. See the [IOTstack MariaDB](MariaDB.md) documentation for more information. ## Keeping Nextcloud up-to-date diff --git a/docs/Containers/WireGuard.md b/docs/Containers/WireGuard.md index 53d0991a..86a1aa93 100644 --- a/docs/Containers/WireGuard.md +++ b/docs/Containers/WireGuard.md @@ -29,7 +29,7 @@ $ sudo apt upgrade -y Before you can use WireGuard (or any VPN solution), you need a mechanism for your remote clients to reach your home router. You have two choices: 1. Obtain a permanent IP address for your home router from your Internet Service Provider (ISP). Approach your ISP if you wish to pursue this option. It generally involves additional charges. -2. Use a Dynamic DNS service. See IOTstack documentation [Accessing your device from the internet](https://sensorsiot.github.io/IOTstack/Accessing-your-Device-from-the-internet.html). The rest of this documentation assumes you have chosen this option. +2. Use a Dynamic DNS service. See IOTstack documentation [Accessing your device from the internet](../Accessing-your-Device-from-the-internet.md). The rest of this documentation assumes you have chosen this option. ### Step 3: Understand the Service Definition @@ -226,7 +226,7 @@ Of the two, the first is generally the simpler and means you don't have to re-ru #### Method 2: Configure WireGuard using `compose-override.yml` -The [Custom services and overriding default settings for IOTstack](https://sensorsiot.github.io/IOTstack/Custom/) page describes how to use an override file to allow the menu to incorporate your custom configurations into the final `docker-compose.yml` file. +The [Custom services and overriding default settings for IOTstack](../Custom.md) page describes how to use an override file to allow the menu to incorporate your custom configurations into the final `docker-compose.yml` file. You will need to create the `compose-override.yml` **before** running the menu to build your stack. If you have already built your stack, you'll have to rebuild it after creating `compose-override.yml`. diff --git a/docs/Custom.md b/docs/Custom.md index 8879f9ad..f6277d78 100644 --- a/docs/Custom.md +++ b/docs/Custom.md @@ -125,7 +125,7 @@ services: environment: ``` -This will remove the default environment variables set in the template, and tell docker-compose to use the variables specified in your file. It is not mandatory that the *.env file be placed in the service's service directory, but is strongly suggested. Keep in mind the [PostBuild Script](https://sensorsiot.github.io/IOTstack/PostBuild-Script) functionality to automatically copy your *.env files into their directories on successful build if you need to. +This will remove the default environment variables set in the template, and tell docker-compose to use the variables specified in your file. It is not mandatory that the *.env file be placed in the service's service directory, but is strongly suggested. Keep in mind the [PostBuild Script](PostBuild-Script.md) functionality to automatically copy your *.env files into their directories on successful build if you need to. ### Adding custom services diff --git a/docs/Getting-Started.md b/docs/Getting-Started.md index 9c7223f8..0f57ff66 100644 --- a/docs/Getting-Started.md +++ b/docs/Getting-Started.md @@ -423,7 +423,7 @@ Cannot create container for service [service name here]: unknown log opt 'max-fi Logging limits were added to prevent Docker using up lots of RAM if log2ram is enabled, or SD cards being filled with log data and degraded from unnecessary IO. See [Docker Logging configurations](https://docs.docker.com/config/containers/logging/configure/) -You can also turn logging off or set it to use another option for any service by using the IOTstack `docker-compose-override.yml` file mentioned at [IOTstack/Custom](https://sensorsiot.github.io/IOTstack/Custom/). +You can also turn logging off or set it to use another option for any service by using the IOTstack `docker-compose-override.yml` file mentioned at [IOTstack/Custom](Custom.md). ### starting an individual container diff --git a/docs/New-Menu-Release-Notes.md b/docs/New-Menu-Release-Notes.md index 8176bd27..5c6c29d1 100755 --- a/docs/New-Menu-Release-Notes.md +++ b/docs/New-Menu-Release-Notes.md @@ -8,7 +8,7 @@ There are many features that are needing to be introduced into the new menu syst ## Breaking changes There are a few changes that you need to be aware of: -* Docker Environmental `*.env` files are no longer a thing by default. Everything needed is specified in the service.yml file, you can still optionally use them though either with [Custom Overrides](https://sensorsiot.github.io/IOTstack/Custom) or with the [PostBuild](https://sensorsiot.github.io/IOTstack/PostBuild-Script) script. Specific config files for certain services still work as they once did. +* Docker Environmental `*.env` files are no longer a thing by default. Everything needed is specified in the service.yml file, you can still optionally use them though either with [Custom Overrides](Custom.md) or with the [PostBuild](PostBuild-Script.md) script. Specific config files for certain services still work as they once did. * Python 3, pip3, PyYAML and Blessed are all required to be installed. * Not backwards compatible with old menu system. You will be able to switch back to the old menu system for a period of time by changing to the `old-menu` branch. It will be unmaintained except for critical updates. It will eventually be removed - but not before everyone is ready to leave it. @@ -26,4 +26,4 @@ There are a few changes that you need to be aware of: * Removed env files * Backup and restoring more streamlined * Documentation updated for all services -* No longer needs to be installed in the home directory `~`. \ No newline at end of file +* No longer needs to be installed in the home directory `~`. From 1eacd40635e7a0016dd2f8c12d146f46c230d772 Mon Sep 17 00:00:00 2001 From: ukkopahis <95980324+ukkopahis@users.noreply.github.com> Date: Sun, 16 Jan 2022 22:26:39 +0200 Subject: [PATCH 08/24] docs: Add how to write documentation --- ...ntributing-Services.md => Contributing.md} | 22 +++++++++++++++++-- docs/index.md | 2 +- 2 files changed, 21 insertions(+), 3 deletions(-) rename docs/Developers/{Contributing-Services.md => Contributing.md} (66%) diff --git a/docs/Developers/Contributing-Services.md b/docs/Developers/Contributing.md similarity index 66% rename from docs/Developers/Contributing-Services.md rename to docs/Developers/Contributing.md index 7fe80dc1..a0d2fb2a 100755 --- a/docs/Developers/Contributing-Services.md +++ b/docs/Developers/Contributing.md @@ -1,6 +1,24 @@ -# Contributing a service to IOTstack +# Contributing -On this page you can find information on how to contribute a service to IOTstack. We are generally very accepting of new services where they are useful. Keep in mind that if it is not IOTstack, selfhosted, or automation related we may not approve the PR. +## Writing documentation + +Documentation is is written as markdown, processed using mkdocs ([docs](https://www.mkdocs.org/user-guide/writing-your-docs/#writing-your-docs)) and the Material theme ([docs](https://squidfunk.github.io/mkdocs-material/reference/)). The Material theme is not just styling, but provides additional syntax extensions. + +Setup your system for Material: +``` +pip3 install mkdocs-material +pip3 install mkdocs-git-revision-date-localized-plugin +``` + +To test your local changes while writing them and before making a pull-request: +``` +cd ~/IOTstack +mkdocs serve +``` + +## Creating a new service + +In this section you can find information on how to contribute a service to IOTstack. We are generally very accepting of new services where they are useful. Keep in mind that if it is not IOTstack, selfhosted, or automation related we may not approve the PR. Services will grow over time, we may split up the buildstack menu into subsections or create filters to make organising all the services we provide easier to find. diff --git a/docs/index.md b/docs/index.md index f880d0fb..3bab6fd4 100644 --- a/docs/index.md +++ b/docs/index.md @@ -10,4 +10,4 @@ Welcome to the IOTstack Wiki: * If you are just getting started with IOTstack, see [Getting Started](./Getting-Started.md). -* To contribute see [Contributing Services](Developers/Contributing-Services.md) +* To contribute see [Contributing](Developers/Contributing.md) From 6be71a56f0ab6eee0808c9bd85e7a0c193a00e8b Mon Sep 17 00:00:00 2001 From: ukkopahis <95980324+ukkopahis@users.noreply.github.com> Date: Tue, 25 Jan 2022 22:36:29 +0200 Subject: [PATCH 09/24] docs: Add top navigation tabs * Add top tabs: Basic setup, Containers, Developers * these were chosen in order to be in a sensible alphabetical order * files moved to corresponding folders and links updated * Add redirect from old file locations * "Basic setup" has a lot of items. Update related pages organized into the "Updates" subtopic, to make it a bit more clear. --- .github/workflows/main.yml | 1 + ...Accessing-your-Device-from-the-internet.md | 2 +- docs/{ => Basic_setup}/Backup-and-Restore.md | 0 docs/{ => Basic_setup}/Custom.md | 2 +- docs/{ => Basic_setup}/Default-Configs.md | 2 +- docs/{ => Basic_setup}/Docker-commands.md | 0 .../{ => Basic_setup}/How-the-script-works.md | 0 docs/{ => Basic_setup}/Misc.md | 0 docs/{ => Basic_setup}/Native-RTL_433.md | 0 docs/{ => Basic_setup}/Networking.md | 0 docs/{ => Basic_setup}/RPIEasy_native.md | 0 .../Understanding-Containers.md | 0 .../Updates}/New-Menu-Release-Notes.md | 3 +- .../Updates}/Updating-the-Project.md | 0 .../Updates}/gcgarner-migration.md | 0 .../index.md} | 2 +- docs/Containers/WireGuard.md | 4 +-- docs/{ => Developers}/PostBuild-Script.md | 2 +- docs/Developers/{Contributing.md => index.md} | 4 +-- docs/index.md | 4 +-- mkdocs.yml | 33 ++++++++++++++++--- 21 files changed, 43 insertions(+), 16 deletions(-) rename docs/{ => Basic_setup}/Accessing-your-Device-from-the-internet.md (99%) rename docs/{ => Basic_setup}/Backup-and-Restore.md (100%) rename docs/{ => Basic_setup}/Custom.md (97%) rename docs/{ => Basic_setup}/Default-Configs.md (98%) rename docs/{ => Basic_setup}/Docker-commands.md (100%) rename docs/{ => Basic_setup}/How-the-script-works.md (100%) rename docs/{ => Basic_setup}/Misc.md (100%) rename docs/{ => Basic_setup}/Native-RTL_433.md (100%) rename docs/{ => Basic_setup}/Networking.md (100%) rename docs/{ => Basic_setup}/RPIEasy_native.md (100%) rename docs/{ => Basic_setup}/Understanding-Containers.md (100%) rename docs/{ => Basic_setup/Updates}/New-Menu-Release-Notes.md (94%) rename docs/{ => Basic_setup/Updates}/Updating-the-Project.md (100%) rename docs/{ => Basic_setup/Updates}/gcgarner-migration.md (100%) rename docs/{Getting-Started.md => Basic_setup/index.md} (99%) rename docs/{ => Developers}/PostBuild-Script.md (83%) rename docs/Developers/{Contributing.md => index.md} (93%) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 3c27211b..8eb3eb18 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -14,5 +14,6 @@ jobs: with: python-version: 3.x - run: pip install mkdocs-material + - run: pip install mkdocs-redirects - run: pip install mkdocs-git-revision-date-localized-plugin - run: mkdocs gh-deploy --force diff --git a/docs/Accessing-your-Device-from-the-internet.md b/docs/Basic_setup/Accessing-your-Device-from-the-internet.md similarity index 99% rename from docs/Accessing-your-Device-from-the-internet.md rename to docs/Basic_setup/Accessing-your-Device-from-the-internet.md index 8ffd315f..08c2c9c2 100644 --- a/docs/Accessing-your-Device-from-the-internet.md +++ b/docs/Basic_setup/Accessing-your-Device-from-the-internet.md @@ -152,7 +152,7 @@ $ cat /dev/null >~/Logs/duck.log ### WireGuard -WireGuard is supplied as part of IOTstack. See [WireGuard documentation](Containers/WireGuard.md). +WireGuard is supplied as part of IOTstack. See [WireGuard documentation](../Containers/WireGuard.md). ### PiVPN diff --git a/docs/Backup-and-Restore.md b/docs/Basic_setup/Backup-and-Restore.md similarity index 100% rename from docs/Backup-and-Restore.md rename to docs/Basic_setup/Backup-and-Restore.md diff --git a/docs/Custom.md b/docs/Basic_setup/Custom.md similarity index 97% rename from docs/Custom.md rename to docs/Basic_setup/Custom.md index f6277d78..03a3508c 100644 --- a/docs/Custom.md +++ b/docs/Basic_setup/Custom.md @@ -125,7 +125,7 @@ services: environment: ``` -This will remove the default environment variables set in the template, and tell docker-compose to use the variables specified in your file. It is not mandatory that the *.env file be placed in the service's service directory, but is strongly suggested. Keep in mind the [PostBuild Script](PostBuild-Script.md) functionality to automatically copy your *.env files into their directories on successful build if you need to. +This will remove the default environment variables set in the template, and tell docker-compose to use the variables specified in your file. It is not mandatory that the *.env file be placed in the service's service directory, but is strongly suggested. Keep in mind the [PostBuild Script](../Developers/PostBuild-Script.md) functionality to automatically copy your *.env files into their directories on successful build if you need to. ### Adding custom services diff --git a/docs/Default-Configs.md b/docs/Basic_setup/Default-Configs.md similarity index 98% rename from docs/Default-Configs.md rename to docs/Basic_setup/Default-Configs.md index 6ef88715..51b05314 100755 --- a/docs/Default-Configs.md +++ b/docs/Basic_setup/Default-Configs.md @@ -1,4 +1,4 @@ -# Build Stack Default Passwords for Services +# Default Passwords and ports Here you can find a list of the default configurations for IOTstack for quick referece. diff --git a/docs/Docker-commands.md b/docs/Basic_setup/Docker-commands.md similarity index 100% rename from docs/Docker-commands.md rename to docs/Basic_setup/Docker-commands.md diff --git a/docs/How-the-script-works.md b/docs/Basic_setup/How-the-script-works.md similarity index 100% rename from docs/How-the-script-works.md rename to docs/Basic_setup/How-the-script-works.md diff --git a/docs/Misc.md b/docs/Basic_setup/Misc.md similarity index 100% rename from docs/Misc.md rename to docs/Basic_setup/Misc.md diff --git a/docs/Native-RTL_433.md b/docs/Basic_setup/Native-RTL_433.md similarity index 100% rename from docs/Native-RTL_433.md rename to docs/Basic_setup/Native-RTL_433.md diff --git a/docs/Networking.md b/docs/Basic_setup/Networking.md similarity index 100% rename from docs/Networking.md rename to docs/Basic_setup/Networking.md diff --git a/docs/RPIEasy_native.md b/docs/Basic_setup/RPIEasy_native.md similarity index 100% rename from docs/RPIEasy_native.md rename to docs/Basic_setup/RPIEasy_native.md diff --git a/docs/Understanding-Containers.md b/docs/Basic_setup/Understanding-Containers.md similarity index 100% rename from docs/Understanding-Containers.md rename to docs/Basic_setup/Understanding-Containers.md diff --git a/docs/New-Menu-Release-Notes.md b/docs/Basic_setup/Updates/New-Menu-Release-Notes.md similarity index 94% rename from docs/New-Menu-Release-Notes.md rename to docs/Basic_setup/Updates/New-Menu-Release-Notes.md index 5c6c29d1..c5f894d9 100755 --- a/docs/New-Menu-Release-Notes.md +++ b/docs/Basic_setup/Updates/New-Menu-Release-Notes.md @@ -8,7 +8,8 @@ There are many features that are needing to be introduced into the new menu syst ## Breaking changes There are a few changes that you need to be aware of: -* Docker Environmental `*.env` files are no longer a thing by default. Everything needed is specified in the service.yml file, you can still optionally use them though either with [Custom Overrides](Custom.md) or with the [PostBuild](PostBuild-Script.md) script. Specific config files for certain services still work as they once did. + +* Docker Environmental `*.env` files are no longer a thing by default. Everything needed is specified in the service.yml file, you can still optionally use them though either with [Custom Overrides](../Custom.md) or with the [PostBuild](../../Developers/PostBuild-Script.md) script. Specific config files for certain services still work as they once did. * Python 3, pip3, PyYAML and Blessed are all required to be installed. * Not backwards compatible with old menu system. You will be able to switch back to the old menu system for a period of time by changing to the `old-menu` branch. It will be unmaintained except for critical updates. It will eventually be removed - but not before everyone is ready to leave it. diff --git a/docs/Updating-the-Project.md b/docs/Basic_setup/Updates/Updating-the-Project.md similarity index 100% rename from docs/Updating-the-Project.md rename to docs/Basic_setup/Updates/Updating-the-Project.md diff --git a/docs/gcgarner-migration.md b/docs/Basic_setup/Updates/gcgarner-migration.md similarity index 100% rename from docs/gcgarner-migration.md rename to docs/Basic_setup/Updates/gcgarner-migration.md diff --git a/docs/Getting-Started.md b/docs/Basic_setup/index.md similarity index 99% rename from docs/Getting-Started.md rename to docs/Basic_setup/index.md index 0f57ff66..26972d2f 100644 --- a/docs/Getting-Started.md +++ b/docs/Basic_setup/index.md @@ -132,7 +132,7 @@ If you prefer to automate your installations using scripts, see: If you are still running on gcgarner/IOTstack and need to migrate to SensorsIot/IOTstack, see: -* [Migrating IOTstack from gcgarner to SensorsIot](./gcgarner-migration.md). +* [Migrating IOTstack from gcgarner to SensorsIot](Updates/gcgarner-migration.md). ## recommended system patches diff --git a/docs/Containers/WireGuard.md b/docs/Containers/WireGuard.md index 86a1aa93..20b84d2f 100644 --- a/docs/Containers/WireGuard.md +++ b/docs/Containers/WireGuard.md @@ -29,7 +29,7 @@ $ sudo apt upgrade -y Before you can use WireGuard (or any VPN solution), you need a mechanism for your remote clients to reach your home router. You have two choices: 1. Obtain a permanent IP address for your home router from your Internet Service Provider (ISP). Approach your ISP if you wish to pursue this option. It generally involves additional charges. -2. Use a Dynamic DNS service. See IOTstack documentation [Accessing your device from the internet](../Accessing-your-Device-from-the-internet.md). The rest of this documentation assumes you have chosen this option. +2. Use a Dynamic DNS service. See IOTstack documentation [Accessing your device from the internet](../Basic_setup/Accessing-your-Device-from-the-internet.md). The rest of this documentation assumes you have chosen this option. ### Step 3: Understand the Service Definition @@ -226,7 +226,7 @@ Of the two, the first is generally the simpler and means you don't have to re-ru #### Method 2: Configure WireGuard using `compose-override.yml` -The [Custom services and overriding default settings for IOTstack](../Custom.md) page describes how to use an override file to allow the menu to incorporate your custom configurations into the final `docker-compose.yml` file. +The [Custom services and overriding default settings for IOTstack](../Basic_setup/Custom.md) page describes how to use an override file to allow the menu to incorporate your custom configurations into the final `docker-compose.yml` file. You will need to create the `compose-override.yml` **before** running the menu to build your stack. If you have already built your stack, you'll have to rebuild it after creating `compose-override.yml`. diff --git a/docs/PostBuild-Script.md b/docs/Developers/PostBuild-Script.md similarity index 83% rename from docs/PostBuild-Script.md rename to docs/Developers/PostBuild-Script.md index 2cda0f3c..cd3142b1 100755 --- a/docs/PostBuild-Script.md +++ b/docs/Developers/PostBuild-Script.md @@ -2,7 +2,7 @@ The postbuild bash script allows for executing arbitrary execution of bash commands after the stack has been build. ## How to use -Place a file in the main directory called `postbuild.sh`. When the buildstack [build logic](Developers/Menu-System.md) finishes, it'll execute the `postbuild.sh` script, passing in each service selected from the buildstack menu as a parameter. This script is run each time the buildstack logic runs. +Place a file in the main directory called `postbuild.sh`. When the buildstack [build logic](../Developers/Menu-System.md) finishes, it'll execute the `postbuild.sh` script, passing in each service selected from the buildstack menu as a parameter. This script is run each time the buildstack logic runs. ## Updates The `postbuild.sh` file has been added to gitignore, so it won't be updated by IOTstack when IOTstack is updated. It has also been added to the backup script so that it will be backed up with your personal IOTstack backups. diff --git a/docs/Developers/Contributing.md b/docs/Developers/index.md similarity index 93% rename from docs/Developers/Contributing.md rename to docs/Developers/index.md index a0d2fb2a..25b20202 100755 --- a/docs/Developers/Contributing.md +++ b/docs/Developers/index.md @@ -27,7 +27,7 @@ Services will grow over time, we may split up the buildstack menu into subsectio * `build.py` file is correct * Service allows for changing external WUI port from Build Stack's options menu if service uses a HTTP/S port * Use a default password, or allow the user to generate a random password for the service for initial installation. If the service asks to setup an account this can be ignored. -* Ensure [Default Configs](../Default-Configs.md) is updated with WUI port and username/password. +* Ensure [Default Configs](../Basic_setup/Default-Configs.md) is updated with WUI port and username/password. * Must detect port confilicts with other services on [BuildStack](Menu-System.md) Menu. * `Pre` and `Post` hooks work with no errors. * Does not require user to edit config files in order to get the service running. @@ -40,7 +40,7 @@ If your new service is approved and merged then congratulations! Please watch th Links: -* [Default configs](../Default-Configs.md) +* [Default configs](../Basic_setup/Default-Configs.md) * [Password configuration for Services](BuildStack-RandomPassword.md) * [Build Stack Menu System](Menu-System.md) * [Coding a new service](BuildStack-Services.md) diff --git a/docs/index.md b/docs/index.md index 3bab6fd4..f320b064 100644 --- a/docs/index.md +++ b/docs/index.md @@ -8,6 +8,6 @@ Welcome to the IOTstack Wiki: * If you are looking for information on a specific container, click on the "Containers" folder at the bottom of the list. -* If you are just getting started with IOTstack, see [Getting Started](./Getting-Started.md). +* If you are just getting started with IOTstack, see [Getting Started](Basic_setup/). -* To contribute see [Contributing](Developers/Contributing.md) +* To contribute see [Contributing](Developers/) diff --git a/mkdocs.yml b/mkdocs.yml index 57195c7b..24017ed5 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -20,10 +20,35 @@ theme: name: Switch to light mode features: - tabs - - navigation.expand - plugins: - - search - - git-revision-date-localized + - navigation.tabs + - navigation.sections + +plugins: + - search + - redirects: + # Forward renamed pages to avoid breaking old links. + redirect_maps: + Getting-Started.md: Basic_setup/index.md + Accessing-your-Device-from-the-internet.md: Basic_setup/Accessing-your-Device-from-the-internet.md + Backup-and-Restore.md: Basic_setup/Backup-and-Restore.md + Custom.md: Basic_setup/Custom.md + Default-Configs.md: Basic_setup/Default-Configs.md + Docker-commands.md: Basic_setup/Docker-commands.md + How-the-script-works.md: Basic_setup/How-the-script-works.md + Misc.md: Basic_setup/Misc.md + Native-RTL_433.md: Basic_setup/Native-RTL_433.md + Networking.md: Basic_setup/Networking.md + RPIEasy_native.md: Basic_setup/RPIEasy_native.md + Understanding-Containers.md: Basic_setup/Understanding-Containers.md + Updates/New-Menu-Release-Notes.md: Basic_setup/Updates/New-Menu-Release-Notes.md + Updates/Updating-the-Project.md: Basic_setup/Updates/Updating-the-Project.md + Updates/gcgarner-migration.md: Basic_setup/Updates/gcgarner-migration.md + PostBuild-Script.md: Developers/PostBuild-Script.md + BuildStack-RandomPassword.md: Developers/BuildStack-RandomPassword.md + BuildStack-Services.md: Developers/BuildStack-Services.md + Menu-System.md: Developers/Menu-System.md + Contributing-Services.md: Developers/index.md + markdown_extensions: - admonition - pymdownx.superfences From 1fc5105aee1fa7dc435c51f6f9436147e0d08441 Mon Sep 17 00:00:00 2001 From: ukkopahis <95980324+ukkopahis@users.noreply.github.com> Date: Tue, 25 Jan 2022 23:39:01 +0200 Subject: [PATCH 10/24] docs: autogenerate heading link anchors Remove custom anchor links and generate them automatically using the toc markdown extension. Links updated to match new anchors. This fixes the custom links coloring the heading blue, which isn't the best if user selects the dark theme. Heading changes done by: cd docs && sed -i -r 's/(#*).*> (.*?) <\/a>/\1 \2/g' *md */*md */*/*md --- ...Accessing-your-Device-from-the-internet.md | 10 +- .../Basic_setup/Updates/gcgarner-migration.md | 42 ++++----- docs/Basic_setup/index.md | 92 +++++++++---------- docs/Containers/AdGuardHome.md | 8 +- docs/Containers/Blynk_server.md | 34 +++---- docs/Containers/Chronograf.md | 8 +- docs/Containers/Home-Assistant.md | 40 ++++---- docs/Containers/Kapacitor.md | 6 +- docs/Containers/MariaDB.md | 10 +- docs/Containers/Mosquitto.md | 68 +++++++------- docs/Containers/NextCloud.md | 14 +-- docs/Containers/Node-RED.md | 80 ++++++++-------- docs/Containers/Portainer-ce.md | 18 ++-- docs/Containers/Prometheus.md | 54 +++++------ docs/Containers/Python.md | 42 ++++----- docs/Containers/Telegraf.md | 52 +++++------ docs/Containers/WireGuard.md | 64 ++++++------- mkdocs.yml | 2 + 18 files changed, 323 insertions(+), 321 deletions(-) diff --git a/docs/Basic_setup/Accessing-your-Device-from-the-internet.md b/docs/Basic_setup/Accessing-your-Device-from-the-internet.md index 08c2c9c2..fca16019 100644 --- a/docs/Basic_setup/Accessing-your-Device-from-the-internet.md +++ b/docs/Basic_setup/Accessing-your-Device-from-the-internet.md @@ -6,14 +6,14 @@ From time to time the IP address that your ISP assigns changes and it's difficul Secondly, how do you get into your home network? Your router has a firewall that is designed to keep the rest of the internet out of your network to protect you. The solution to that is a Virtual Private Network (VPN) or "tunnel". -## Dynamic DNS +## Dynamic DNS There are two parts to a Dynamic DNS service: 1. You have to register with a Dynamic DNS service provider and obtain a domain name that is not already taken by someone else. 2. Something on your side of the network needs to propagate updates so that your chosen domain name remains in sync with your router's dynamically-allocated public IP address. -### Register with a Dynamic DNS service provider +### Register with a Dynamic DNS service provider The first part is fairly simple and there are quite a few Dynamic DNS service providers including: @@ -24,7 +24,7 @@ The first part is fairly simple and there are quite a few Dynamic DNS service pr Some router vendors also provide their own built-in Dynamic DNS capabilities for registered customers so it's a good idea to check your router's capabilities before you plough ahead. -### Dynamic DNS propagation +### Dynamic DNS propagation The "something" on your side of the network propagating WAN IP address changes can be either: @@ -39,7 +39,7 @@ A behind-the-router technique usually relies on sending updates according to a s > This seems to be a problem for DuckDNS which takes a beating because almost every person using it is sending an update bang-on five minutes. -### DuckDNS client +### DuckDNS client IOTstack provides a solution for DuckDNS. The best approach to running it is: @@ -99,7 +99,7 @@ A null result indicates failure so check your work. Remember, the Domain Name System is a *distributed* database. It takes *time* for changes to propagate. The response you get from directing a query to ns1.duckdns.org may not be the same as the response you get from any other DNS server. You often have to wait until cached records expire and a recursive query reaches the authoritative DuckDNS name-servers. -#### Running the DuckDNS client automatically +#### Running the DuckDNS client automatically The recommended arrangement for keeping your Dynamic DNS service up-to-date is to invoke `duck.sh` from `cron` at five minute intervals. diff --git a/docs/Basic_setup/Updates/gcgarner-migration.md b/docs/Basic_setup/Updates/gcgarner-migration.md index 2db53811..43178476 100644 --- a/docs/Basic_setup/Updates/gcgarner-migration.md +++ b/docs/Basic_setup/Updates/gcgarner-migration.md @@ -6,9 +6,9 @@ Migrating to SensorsIot/IOTstack was fairly easy when this repository was first The probability of conflicts developing increases as a function of time since the fork. Conflicts were and are pretty much inevitable so a more involved procedure is needed. -## Migration Steps +## Migration Steps -### Step 1 – Check your assumptions +### Step 1 – Check your assumptions Make sure that you are, *actually*, on gcgarner. Don't assume! @@ -20,7 +20,7 @@ origin https://github.com/gcgarner/IOTstack.git (push) Do not proceed if you don't see those URLs! -### Step 2 – Take IOTstack down +### Step 2 – Take IOTstack down Take your stack down. This is not *strictly* necessary but we'll be moving the goalposts a bit so it's better to be on the safe side. @@ -29,22 +29,22 @@ $ cd ~/IOTstack $ docker-compose down ``` -### Step 3 – Choose your migration method +### Step 3 – Choose your migration method There are two basic approaches to switching from gcgarner/IOTstack to SensorsIot/IOTstack: -- [Migration by changing upstream repository](#migrateChangeUpstream) -- [Migration by clone and merge](#migrateCloneMerge) +- [Migration by changing upstream repository](#migration-option-1-change-upstream-repository) +- [Migration by clone and merge](#migration-option-2-clone-and-merge) You can think of the first as "working *with* git" while the second is "using brute force". The first approach will work if you haven't tried any other migration steps and/or have not made too many changes to items in your gcgarner/IOTstack that are under git control. -If you are already stuck or you try the first approach and get a mess, or it all looks far too hard to sort out, then try the [Migration by clone and merge](#migrateCloneMerge) approach. +If you are already stuck or you try the first approach and get a mess, or it all looks far too hard to sort out, then try the [Migration by clone and merge](#migration-option-2-clone-and-merge) approach. -#### Migration Option 1 – change upstream repository +#### Migration Option 1 – change upstream repository -##### Check for local changes +##### Check for local changes Make sure you are on the master branch (you probably are so this is just a precaution), and then see if Git thinks you have made any local changes: @@ -93,7 +93,7 @@ The simplest way to deal with modified files is to rename them to move them out menu.sh.jqh ``` -##### Synchronise with gcgarner on GitHub +##### Synchronise with gcgarner on GitHub Make sure your local copy of gcgarner is in sync with GitHub. @@ -101,7 +101,7 @@ Make sure your local copy of gcgarner is in sync with GitHub. $ git pull ``` -##### Get rid of any upstream reference +##### Get rid of any upstream reference There may or may not be any "upstream" set. The most likely reason for this to happen is if you used your local copy as the basis of a Pull Request. @@ -111,7 +111,7 @@ The next command will probably return an error, which you should ignore. It's ju $ git remote remove upstream ``` -##### Point to SensorsIot +##### Point to SensorsIot Change your local repository to point to SensorsIot. @@ -119,7 +119,7 @@ Change your local repository to point to SensorsIot. $ git remote set-url origin https://github.com/SensorsIot/IOTstack.git ``` -##### Synchronise with SensorsIot on GitHub +##### Synchronise with SensorsIot on GitHub This is where things can get a bit tricky so please read these instructions carefully **before** you proceed. @@ -174,7 +174,7 @@ Auto-merging .templates/someRandomService/service.yml If you don't use `someRandomService` then you could safely ignore this on the basis that it was "probably right". However, if you did use that service and it started to misbehave after migration, you would know that the `service.yml` file was a good place to start looking for explanations. -##### Finish with a pull +##### Finish with a pull At this point, only the migrated master branch is present on your local copy of the repository. The next command brings you fully in-sync with GitHub: @@ -182,11 +182,11 @@ At this point, only the migrated master branch is present on your local copy of $ git pull ``` -#### Migration Option 2 – clone and merge +#### Migration Option 2 – clone and merge If you have been following the process correctly, your IOTstack will already be down. -##### Rename your existing IOTstack folder +##### Rename your existing IOTstack folder Move your old IOTstack folder out of the way, like this: @@ -199,7 +199,7 @@ Note: * You should not need `sudo` for the `mv` command but it is OK to use it if necessary. -##### Fetch a clean clone of SensorsIot/IOTstack +##### Fetch a clean clone of SensorsIot/IOTstack ``` $ git clone https://github.com/SensorsIot/IOTstack.git ~/IOTstack @@ -240,7 +240,7 @@ Observe what is **not** there: From this, it should be self-evident that a clean checkout from GitHub is the factory for *all* IOTstack installations, while the contents of `backups`, `services`, `volumes` and `docker-compose.yml` represent each user's individual choices, configuration options and data. -##### Merge old into new +##### Merge old into new Execute the following commands: @@ -272,7 +272,7 @@ There is no need to migrate the `backups` directory. You are better off creating $ mkdir ~/IOTstack/backups ``` -### Step 4 – Choose your menu +### Step 4 – Choose your menu If you have reached this point, you have migrated to SensorsIot/IOTstack where you are on the "master" branch. This implies "new menu". @@ -353,7 +353,7 @@ Although you can freely change branches, it's probably not a good idea to try to Even so, nothing will change **until** you run your chosen menu to completion and allow it to generate a new `docker-compose.yml`. -### Step 5 – Bring up your stack +### Step 5 – Bring up your stack Unless you have gotten ahead of yourself and have already run the menu (old or new) then nothing will have changed in the parts of your `~/IOTstack` folder that define your IOTstack implementation. You can safely: @@ -361,7 +361,7 @@ Unless you have gotten ahead of yourself and have already run the menu (old or n $ docker-compose up -d ``` -## See also +## See also There is another gist [Installing Docker for IOTstack](https://gist.github.com/Paraphraser/d119ae81f9e60a94e1209986d8c9e42f) which explains how to overcome problems with outdated Docker and Docker-Compose installations. diff --git a/docs/Basic_setup/index.md b/docs/Basic_setup/index.md index 26972d2f..dd7f7b00 100644 --- a/docs/Basic_setup/index.md +++ b/docs/Basic_setup/index.md @@ -1,6 +1,6 @@ # Getting Started -## introduction to IOTstack - videos +## introduction to IOTstack - videos Andreas Spiess Video #295: Raspberry Pi Server based on Docker, with VPN, Dropbox backup, Influx, Grafana, etc: IOTstack @@ -10,7 +10,7 @@ Andreas Spiess Video #352: Raspberry Pi4 Home Automation Server (incl. Docker, O [![#352 Raspberry Pi4 Home Automation Server (incl. Docker, OpenHAB, HASSIO, NextCloud)](http://img.youtube.com/vi/KJRMjUzlHI8/0.jpg)](https://www.youtube.com/watch?v=KJRMjUzlHI8) -## assumptions +## assumptions IOTstack makes the following assumptions: @@ -46,15 +46,15 @@ If the first three assumptions hold, assumptions four through six are Raspberry Please don't read these assumptions as saying that IOTstack will not run on other hardware, other operating systems, or as a different user. It is just that IOTstack gets most of its testing under these conditions. The further you get from these implicit assumptions, the more your mileage may vary. -### other platforms +### other platforms Users have reported success on other platforms, including: * [Orange Pi WinPlus](https://github.com/SensorsIot/IOTstack/issues/375) -## new installation +## new installation -### automatic (recommended) +### automatic (recommended) 1. Install `curl`: @@ -82,7 +82,7 @@ Users have reported success on other platforms, including: $ docker-compose up -d ``` -### manual +### manual 1. Install `git`: @@ -122,21 +122,21 @@ Users have reported success on other platforms, including: $ docker-compose up -d ``` -### scripted +### scripted If you prefer to automate your installations using scripts, see: * [Installing Docker for IOTstack](https://gist.github.com/Paraphraser/d119ae81f9e60a94e1209986d8c9e42f#scripting-iotstack-installations). -## migrating from the old repo (gcgarner)? +## migrating from the old repo (gcgarner)? If you are still running on gcgarner/IOTstack and need to migrate to SensorsIot/IOTstack, see: * [Migrating IOTstack from gcgarner to SensorsIot](Updates/gcgarner-migration.md). -## recommended system patches +## recommended system patches -### patch 1 – restrict DHCP +### patch 1 – restrict DHCP Run the following commands: @@ -147,7 +147,7 @@ $ sudo reboot See [Issue 219](https://github.com/SensorsIot/IOTstack/issues/219) and [Issue 253](https://github.com/SensorsIot/IOTstack/issues/253) for more information. -### patch 2 – update libseccomp2 +### patch 2 – update libseccomp2 This patch is **ONLY** for Raspbian Buster. Do **NOT** install this patch if you are running Raspbian Bullseye. @@ -189,7 +189,7 @@ Enable by running (takes effect after reboot): echo $(cat /boot/cmdline.txt) cgroup_memory=1 cgroup_enable=memory | sudo tee /boot/cmdline.txt ``` -## a word about the `sudo` command +## a word about the `sudo` command Many first-time users of IOTstack get into difficulty by misusing the `sudo` command. The problem is best understood by example. In the following, you would expect `~` (tilde) to expand to `/home/pi`. It does: @@ -237,13 +237,13 @@ Please try to minimise your use of `sudo` when you are working with IOTstack. He It takes time, patience and practice to learn when `sudo` is **actually** needed. Over-using `sudo` out of habit, or because you were following a bad example you found on the web, is a very good way to find that you have created so many problems for yourself that will need to reinstall your IOTstack. *Please* err on the side of caution! -## the IOTstack menu +## the IOTstack menu The menu is used to install Docker and then build the `docker-compose.yml` file which is necessary for starting the stack. > The menu is only an aid. It is a good idea to learn the `docker` and `docker-compose` commands if you plan on using Docker in the long run. -### menu item: Install Docker (old menu only) +### menu item: Install Docker (old menu only) Please do **not** try to install `docker` and `docker-compose` via `sudo apt install`. There's more to it than that. Docker needs to be installed by `menu.sh`. The menu will prompt you to install docker if it detects that docker is not already installed. You can manually install it from within the `Native Installs` menu: @@ -260,7 +260,7 @@ Note: * New menu (master branch) automates this step. -### menu item: Build Stack +### menu item: Build Stack `docker-compose` uses a `docker-compose.yml` file to configure all your services. The `docker-compose.yml` file is created by the menu: @@ -292,15 +292,15 @@ Some containers also need to be built locally. Node-RED is an example. Depending Be patient (and ignore the huge number of warnings). -### menu item: Docker commands +### menu item: Docker commands The commands in this menu execute shell scripts in the root of the project. -### other menu items +### other menu items The old and new menus differ in the options they offer. You should come back and explore them once your stack is built and running. -## switching menus +## switching menus At the time of writing, IOTstack supports three menus: @@ -334,7 +334,7 @@ $ git checkout -- .templates/mosquitto/Dockerfile When `git status` reports no more "modified" files, it is safe to switch your branch. -### current menu (master branch) +### current menu (master branch) ```bash $ cd ~/IOTstack/ @@ -352,7 +352,7 @@ $ git checkout old-menu $ ./menu.sh ``` -### experimental branch +### experimental branch Switch to the experimental branch to try the latest and greatest features. @@ -377,14 +377,14 @@ Notes: * The way back is to take down your stack, restore a backup, and bring up your stack again. -## useful commands: docker & docker-compose +## useful commands: docker & docker-compose Handy rules: * `docker` commands can be executed from anywhere, but * `docker-compose` commands need to be executed from within `~/IOTstack` -### starting your IOTstack +### starting your IOTstack To start the stack: @@ -395,7 +395,7 @@ $ docker-compose up -d Once the stack has been brought up, it will stay up until you take it down. This includes shutdowns and reboots of your Raspberry Pi. If you do not want the stack to start automatically after a reboot, you need to stop the stack before you issue the reboot command. -#### logging journald errors +#### logging journald errors If you get docker logging error like: @@ -425,7 +425,7 @@ Logging limits were added to prevent Docker using up lots of RAM if log2ram is e You can also turn logging off or set it to use another option for any service by using the IOTstack `docker-compose-override.yml` file mentioned at [IOTstack/Custom](Custom.md). -### starting an individual container +### starting an individual container To start a particular container: @@ -434,7 +434,7 @@ $ cd ~/IOTstack $ docker-compose up -d «container» ``` -### stopping your IOTstack +### stopping your IOTstack Stopping aka "downing" the stack stops and deletes all containers, and removes the internal network: @@ -450,7 +450,7 @@ $ cd ~/IOTstack $ docker-compose stop ``` -### stopping an individual container +### stopping an individual container `stop` can also be used to stop individual containers, like this: @@ -480,7 +480,7 @@ $ cd ~/IOTstack $ docker-compose up -d «container» ``` -### checking container status +### checking container status You can check the status of containers with: @@ -495,7 +495,7 @@ $ cd ~/IOTstack $ docker-compose ps ``` -### viewing container logs +### viewing container logs You can inspect the logs of most containers like this: @@ -517,7 +517,7 @@ $ docker logs -f nodered Terminate with a Control+C. Note that restarting a container will also terminate a followed log. -### restarting a container +### restarting a container You can restart a container in several ways: @@ -544,9 +544,9 @@ $ cd ~/IOTstack $ docker-compose up -d --force-recreate «container» ``` -See also [updating images built from Dockerfiles](#updateDockerfile) if you need to force `docker-compose` to notice a change to a Dockerfile. +See also [updating images built from Dockerfiles](#updating-images-not-built-from-dockerfiles) if you need to force `docker-compose` to notice a change to a Dockerfile. -## persistent data +## persistent data Docker allows a container's designer to map folders inside a container to a folder on your disk (SD, SSD, HD). This is done with the "volumes" key in `docker-compose.yml`. Consider the following snippet for Node-RED: @@ -588,7 +588,7 @@ is mirrored at the same relative path **inside** the container at: /data ``` -### deleting persistent data +### deleting persistent data If you need a "clean slate" for a container, you can delete its volumes. Using InfluxDB as an example: @@ -616,9 +616,9 @@ When InfluxDB starts, it sees that the folder on right-hand-side of the volumes This is how **most** containers behave. There are exceptions so it's always a good idea to keep a backup. -## stack maintenance +## stack maintenance -### update Raspberry Pi OS +### update Raspberry Pi OS You should keep your Raspberry Pi up-to-date. Despite the word "container" suggesting that *containers* are fully self-contained, they sometimes depend on operating system components ("WireGuard" is an example). @@ -627,7 +627,7 @@ $ sudo apt update $ sudo apt upgrade -y ``` -### git pull +### git pull Although the menu will generally do this for you, it does not hurt to keep your local copy of the IOTstack repository in sync with the master version on GitHub. @@ -636,7 +636,7 @@ $ cd ~/IOTstack $ git pull ``` -### container image updates +### container image updates There are two kinds of images used in IOTstack: @@ -650,7 +650,7 @@ The easiest way to work out which type of image you are looking at is to inspect * `image:` keyword then the image is **not** built using a Dockerfile. * `build:` keyword then the image **is** built using a Dockerfile. -#### updating images not built from Dockerfiles +#### updating images not built from Dockerfiles If new versions of this type of image become available on DockerHub, your local IOTstack copies can be updated by a `pull` command: @@ -665,7 +665,7 @@ The `pull` downloads any new images. It does this without disrupting the running The `up -d` notices any newly-downloaded images, builds new containers, and swaps old-for-new. There is barely any downtime for affected containers. -#### updating images built from Dockerfiles +#### updating images built from Dockerfiles Containers built using Dockerfiles have a two-step process: @@ -685,7 +685,7 @@ Note: * You can also add nodes to Node-RED using Manage Palette. -##### when Dockerfile changes (*local* image only) +##### when Dockerfile changes (*local* image only) When your Dockerfile changes, you need to rebuild like this: @@ -697,7 +697,7 @@ $ docker system prune This only rebuilds the *local* image and, even then, only if `docker-compose` senses a *material* change to the Dockerfile. -If you are trying to force the inclusion of a later version of an add-on node, you need to treat it like a [DockerHub update](#rebuildDockerfile). +If you are trying to force the inclusion of a later version of an add-on node, you need to treat it like a [DockerHub update](#updating-images-built-from-dockerfiles). Key point: @@ -712,7 +712,7 @@ Note: $ docker-compose up --build -d nodered ``` -##### when DockerHub updates (*base* and *local* images) +##### when DockerHub updates (*base* and *local* images) When a newer version of the *base* image appears on DockerHub, you need to rebuild like this: @@ -728,7 +728,7 @@ This causes DockerHub to be checked for the later version of the *base* image, d Then, the Dockerfile is run to produce a new *local* image. The Dockerfile run happens even if a new *base* image was not downloaded in the previous step. -### deleting unused images +### deleting unused images As your system evolves and new images come down from DockerHub, you may find that more disk space is being occupied than you expected. Try running: @@ -762,9 +762,9 @@ $ docker rmi dbf28ba50432 In general, you can use the repository name to remove an image but the Image ID is sometimes needed. The most common situation where you are likely to need the Image ID is after an image has been updated on DockerHub and pulled down to your Raspberry Pi. You will find two containers with the same name. One will be tagged "latest" (the running version) while the other will be tagged "\" (the prior version). You use the Image ID to resolve the ambiguity. -### pinning to specific versions +### pinning to specific versions -See [container image updates](#imageUpdates) to understand how to tell the difference between images that are used "as is" from DockerHub versus those that are built from local Dockerfiles. +See [container image updates](#container-image-updates) to understand how to tell the difference between images that are used "as is" from DockerHub versus those that are built from local Dockerfiles. Note: @@ -820,7 +820,7 @@ To pin an image to a specific version: $ docker-compose up -d --build mosquitto ``` -## the nuclear option - use with caution +## the nuclear option - use with caution If you create a mess and can't see how to recover, try proceeding like this: @@ -840,7 +840,7 @@ In words: 4. Move your existing IOTstack directory out of the way. If you get a permissions problem: * Re-try the command with `sudo`; and - * Read [a word about the `sudo` command](#aboutSudo). Needing `sudo` in this situation is an example of over-using `sudo`. + * Read [a word about the `sudo` command](#a-word-about-the-sudo-command). Needing `sudo` in this situation is an example of over-using `sudo`. 5. Check out a clean copy of IOTstack. diff --git a/docs/Containers/AdGuardHome.md b/docs/Containers/AdGuardHome.md index 3404cac8..1fd59f19 100644 --- a/docs/Containers/AdGuardHome.md +++ b/docs/Containers/AdGuardHome.md @@ -9,7 +9,7 @@ AdGuard Home and PiHole perform similar functions. They use the same ports so you can **not** run both at the same time. You must choose one or the other. -## Quick Start +## Quick Start When you first install AdGuard Home: @@ -34,7 +34,7 @@ When you first install AdGuard Home: Port 8089 is the default administrative user interface for AdGuard Home running under IOTstack. -Port 8089 is not active until you have completed the [Quick Start](#quickStart) procedure. You must start by connecting to port 3001. +Port 8089 is not active until you have completed the [Quick Start](#quick-start) procedure. You must start by connecting to port 3001. Because of AdGuard Home limitations, you must take special precautions if you decide to change to a different port number: @@ -50,11 +50,11 @@ Because of AdGuard Home limitations, you must take special precautions if you de $ docker-compose up -d adguardhome ``` -3. Repeat the [Quick Start](#quickStart) procedure, this time substituting the new Admin Web Interface port where you see "8089". +3. Repeat the [Quick Start](#quick-start) procedure, this time substituting the new Admin Web Interface port where you see "8089". ## About port 3001:3000 -Port 3001 (external, 3000 internal) is only used during [Quick Start](#quickStart) procedure. Once port 8089 becomes active, port 3001 ceases to be active. +Port 3001 (external, 3000 internal) is only used during [Quick Start](#quick-start) procedure. Once port 8089 becomes active, port 3001 ceases to be active. In other words, you need to keep port 3001 reserved even though it is only ever used to set up port 8089. diff --git a/docs/Containers/Blynk_server.md b/docs/Containers/Blynk_server.md index 10a8eca2..95a5b90d 100644 --- a/docs/Containers/Blynk_server.md +++ b/docs/Containers/Blynk_server.md @@ -2,7 +2,7 @@ This document discusses an IOTstack-specific version of Blynk-Server. It is built on top of an [Ubuntu](https://hub.docker.com/_/ubuntu) base image using a *Dockerfile*. -## References +## References - [Ubuntu base image](https://hub.docker.com/_/ubuntu) at DockerHub - [Peter Knight Blynk-Server fork](https://github.com/Peterkn2001/blynk-server) at GitHub (includes documentation) @@ -18,7 +18,7 @@ Acknowledgement: - Original writeup from @877dev -## Significant directories and files +## Significant directories and files ``` ~/IOTstack @@ -56,19 +56,19 @@ Everything in ❽: * will be replaced if it is not present when the container starts; but * will never be overwritten if altered by you. -## How Blynk Server gets built for IOTstack +## How Blynk Server gets built for IOTstack -### GitHub Updates +### GitHub Updates Periodically, the source code is updated and a new version is released. You can check for the latest version at the [releases page](https://github.com/Peterkn2001/blynk-server/releases/). -### IOTstack menu +### IOTstack menu When you select Blynk Server in the IOTstack menu, the *template service definition* is copied into the *Compose* file. > Under old menu, it is also copied to the *working service definition* and then not really used. -### IOTstack first run +### IOTstack first run On a first install of IOTstack, you run the menu, choose your containers, and are told to do this: @@ -129,7 +129,7 @@ ubuntu latest 897590a6c564 7 days ago 49.8MB You will see the same pattern in *Portainer*, which reports the ***base image*** as "unused". You should not remove the ***base*** image, even though it appears to be unused. -## Logging +## Logging You can inspect Blynk Server's log by: @@ -137,7 +137,7 @@ You can inspect Blynk Server's log by: $ docker logs blynk_server ``` -## Changing Blynk Server's configuration +## Changing Blynk Server's configuration The first time you launch the `blynk_server` container, the following structure will be created in the persistent storage area: @@ -156,7 +156,7 @@ $ cd ~/IOTstack $ docker-compose restart blynk_server ``` -## Getting a clean slate +## Getting a clean slate Erasing Blynk Server's persistent storage area triggers self-healing and restores known defaults: @@ -176,7 +176,7 @@ Note: $ docker-compose restart blynk_server ``` -## Upgrading Blynk Server +## Upgrading Blynk Server To find out when a new version has been released, you need to visit the [Blynk-Server releases](https://github.com/Peterkn2001/blynk-server/releases/) page at GitHub. @@ -216,11 +216,11 @@ At the time of writing, version 0.41.16 was the most up-to-date. Suppose that ve $ docker system prune -f ``` -## Using Blynk Server +## Using Blynk Server See the [References](#references) for documentation links. -### Connecting to the administrative UI +### Connecting to the administrative UI To connect to the administrative interface, navigate to: @@ -233,7 +233,7 @@ You may encounter browser security warnings which you will have to acknowledge i - username = `admin@blynk.cc` - password = `admin` -### Change username and password +### Change username and password 1. Click on Users > "email address" and edit email, name and password. 2. Save changes. @@ -244,19 +244,19 @@ You may encounter browser security warnings which you will have to acknowledge i $ docker-compose restart blynk_server ``` -### Setup gmail +### Setup gmail Optional step, useful for getting the auth token emailed to you. (To be added once confirmed working....) -### iOS/Android app setup +### iOS/Android app setup 1. When setting up the application on your mobile be sure to select "custom" setup [see](https://github.com/Peterkn2001/blynk-server#app-and-sketch-changes). 2. Press "New Project" 3. Give it a name, choose device "Raspberry Pi 3 B" so you have plenty of [virtual pins](http://help.blynk.cc/en/articles/512061-what-is-virtual-pins) available, and lastly select WiFi. 4. Create project and the [auth token](https://docs.blynk.cc/#getting-started-getting-started-with-the-blynk-app-4-auth-token) will be emailed to you (if emails configured). You can also find the token in app under the phone app settings, or in the admin web interface by clicking Users>"email address" and scroll down to token. -### Quick usage guide for app +### Quick usage guide for app 1. Press on the empty page, the widgets will appear from the right. 2. Select your widget, let's say a button. @@ -269,7 +269,7 @@ Optional step, useful for getting the auth token emailed to you. Enter Node-Red..... -### Node-RED +### Node-RED 1. Install `node-red-contrib-blynk-ws` from Manage Palette. 2. Drag a "write event" node into your flow, and connect to a debug node diff --git a/docs/Containers/Chronograf.md b/docs/Containers/Chronograf.md index c2fa8179..02d276fe 100644 --- a/docs/Containers/Chronograf.md +++ b/docs/Containers/Chronograf.md @@ -1,12 +1,12 @@ # Chronograf -## References +## References - [*influxdata Chronograf* documentation](https://docs.influxdata.com/chronograf/) - [*GitHub*: influxdata/influxdata-docker/chronograf](https://github.com/influxdata/influxdata-docker/tree/master/chronograf) - [*DockerHub*: influxdata Chronograf](https://hub.docker.com/_/chronograf) -## Kapacitor integration +## Kapacitor integration If you selected Kapacitor in the menu and want Chronograf to be able to interact with it, you need to edit `docker-compose.yml` to un-comment the lines which are commented-out in the following: @@ -28,7 +28,7 @@ $ cd ~IOTstack $ docker-compose up -d chronograf ``` -## Upgrading Chronograf +## Upgrading Chronograf You can update the container via: @@ -45,7 +45,7 @@ In words: * `docker-compose up -d` causes any newly-downloaded images to be instantiated as containers (replacing the old containers); and * the `prune` gets rid of the outdated images. -### Chronograf version pinning +### Chronograf version pinning If you need to pin to a particular version: diff --git a/docs/Containers/Home-Assistant.md b/docs/Containers/Home-Assistant.md index 8e886674..585ce34e 100644 --- a/docs/Containers/Home-Assistant.md +++ b/docs/Containers/Home-Assistant.md @@ -2,7 +2,7 @@ Home Assistant is a home automation platform running on Python 3. It is able to track and control all devices at your home and offer a platform for automating control. -## References +## References - [Home Assistant home page](https://www.home-assistant.io/) @@ -31,7 +31,7 @@ Note: * Technically, both versions can **run** at the same time but it is not **supported**. Each version runs in "host mode" and binds to port 8123 so, in practice, the first version to start will claim the port and the second version will then be blocked. -### Hass.io +### Hass.io Hass.io uses its own orchestration: @@ -45,13 +45,13 @@ Hass.io uses its own orchestration: IOTstack can only offer limited configuration of Hass.io since it is its own platform. -### Home Assistant Container +### Home Assistant Container Home Assistant Container runs as a single Docker container, and doesn't support all the features that Hass.io does (such as add-ons). -## Menu installation +## Menu installation -### Installing Hass.io +### Installing Hass.io Hass.io creates a conundrum: @@ -59,7 +59,7 @@ Hass.io creates a conundrum: * One of Hass.io's dependencies is [Network Manager](https://wiki.archlinux.org/index.php/NetworkManager). Network Manager makes **serious** changes to your operating system, with side-effects you may not expect such as giving your Raspberry Pi's WiFi interface a random MAC address both during the installation and, then, each time you reboot. You are in for a world of pain if you install Network Manager without first understanding what is going to happen and planning accordingly. * If you don't install Hass.io's dependencies before you install Docker, you will either have to uninstall Docker or rebuild your system. This is because both Docker and Network Manager adjust your Raspberry Pi's networking. Docker is happy to install after Network Manager, but the reverse is not true. -#### Step 1: If Docker is already installed, uninstall it +#### Step 1: If Docker is already installed, uninstall it ```bash $ sudo apt -y purge docker-ce docker-ce-cli containerd.io @@ -71,21 +71,21 @@ Note: * Removing Docker does **not** interfere with your existing `~/IOTstack` folder. -#### Step 2: Ensure your system is fully up-to-date +#### Step 2: Ensure your system is fully up-to-date ```bash $ sudo apt update $ sudo apt upgrade -y ``` -#### Step 3: Install Hass.io dependencies (stage 1) +#### Step 3: Install Hass.io dependencies (stage 1) ```bash $ sudo apt install -y apparmor apparmor-profiles apparmor-utils $ sudo apt install -y software-properties-common apt-transport-https ca-certificates dbus ``` -#### Step 4: Connect to your Raspberry Pi via Ethernet +#### Step 4: Connect to your Raspberry Pi via Ethernet You can skip this step if you interact with your Raspberry Pi via a screen connected to its HDMI port, along with a keyboard and mouse. @@ -127,7 +127,7 @@ You *may* be able to re-connect after the WiFi interface acquires a new IP addre The advice about using Ethernet is well-intentioned. You should heed this advice even if means you need to temporarily relocate your Raspberry Pi just so you can attach it via Ethernet for the next few steps. You can go back to WiFi later, once everything is set up. You have been warned! -#### Step 5: Install Hass.io dependencies (stage 2) +#### Step 5: Install Hass.io dependencies (stage 2) Install Network Manager: @@ -135,9 +135,9 @@ Install Network Manager: $ sudo apt install -y network-manager ``` -#### Step 6: Consider disabling random MAC address allocation +#### Step 6: Consider disabling random MAC address allocation -To understand why you should consider disabling random MAC address allocation, see [why random MACs are such a hassle ](#aboutRandomMACs). +To understand why you should consider disabling random MAC address allocation, see [why random MACs are such a hassle ](#why-random-macs-are-such-a-hassle). You can stop Network Manager from allocating random MAC addresses to your WiFi interface by running the following commands: @@ -150,7 +150,7 @@ Acknowledgement: * This tip came from [@steveatk on Discord](https://discordapp.com/channels/638610460567928832/638610461109256194/758825690715652116). -#### Step 7: Re-install Docker +#### Step 7: Re-install Docker You can re-install Docker using the IOTstack menu or one of the scripts provided with IOTstack but the following commands guarantee an up-to-date version of `docker-compose` and also include a dependency needed if you want to run with the 64-bit kernel: @@ -169,7 +169,7 @@ Note: * Installing or re-installing Docker does **not** interfere with your existing `~/IOTstack` folder. -#### Step 8: Run the Hass.io installation +#### Step 8: Run the Hass.io installation Start at: @@ -184,7 +184,7 @@ The installation of Hass.io takes up to 20 minutes (depending on your internet c Hass.io installation is provided as a convenience. It is independent of, is not maintained by, and does not appear in the `docker-compose.yml` for IOTstack. Hass.io has its own service for maintaining its uptime. -#### Re-check random MAC address allocation +#### Re-check random MAC address allocation Installing Hass.io can re-enable random MAC address allocation. You should check this via: @@ -195,9 +195,9 @@ wifi.scan-rand-mac-address=no ``` -If you do **NOT** see `wifi.scan-rand-mac-address=no`, repeat [Step 6](#disableRandomMac1). +If you do **NOT** see `wifi.scan-rand-mac-address=no`, repeat [Step 6](#step-6-consider-disabling-random-mac-address-allocation). -### Installing Home Assistant Container +### Installing Home Assistant Container Home Assistant can be found in the `Build Stack` menu. Selecting it in this menu results in a service definition being added to: @@ -222,7 +222,7 @@ $ cd ~/IOTstack $ docker-compose up -d ``` -## Deactivating Hass.io +## Deactivating Hass.io Because Hass.io is independent of IOTstack, you can't deactivate it with any of the commands you normally use for IOTstack. @@ -249,7 +249,7 @@ You can use Portainer to view what is running and clean up the unused images. At this point, Hass.io is stopped and will not start again after a reboot. Your options are: * Leave things as they are; or -* Re-install Hass.io by starting over at [Installing Hass.io](#installHassio); or +* Re-install Hass.io by starting over at [Installing Hass.io](#installing-hassio); or * Re-activate Hass.io by: ```bash @@ -257,7 +257,7 @@ At this point, Hass.io is stopped and will not start again after a reboot. Your $ sudo systemctl start hassio-supervisor.service ``` -## Why random MACs are such a hassle +## Why random MACs are such a hassle > This material was originally posted as part of [Issue 312](https://github.com/SensorsIot/IOTstack/issues/312). It was moved here following a suggestion by [lole-elol](https://github.com/lole-elol). diff --git a/docs/Containers/Kapacitor.md b/docs/Containers/Kapacitor.md index a126675e..11296e31 100644 --- a/docs/Containers/Kapacitor.md +++ b/docs/Containers/Kapacitor.md @@ -1,12 +1,12 @@ # Kapacitor -## References +## References - [*influxdata Kapacitor* documentation](https://docs.influxdata.com/kapacitor/) - [*GitHub*: influxdata/influxdata-docker/kapacitor](https://github.com/influxdata/influxdata-docker/tree/master/kapacitor) - [*DockerHub*: influxdata Kapacitor](https://hub.docker.com/_/kapacitor) -## Upgrading Kapacitor +## Upgrading Kapacitor You can update the container via: @@ -23,7 +23,7 @@ In words: * `docker-compose up -d` causes any newly-downloaded images to be instantiated as containers (replacing the old containers); and * the `prune` gets rid of the outdated images. -### Kapacitor version pinning +### Kapacitor version pinning If you need to pin to a particular version: diff --git a/docs/Containers/MariaDB.md b/docs/Containers/MariaDB.md index 90a3739d..c32c3437 100644 --- a/docs/Containers/MariaDB.md +++ b/docs/Containers/MariaDB.md @@ -67,9 +67,9 @@ To close the terminal session, either: * type "exit" and press return; or * press control+d. -## Container health check +## Container health check -### theory of operation +### theory of operation A script , or "agent", to assess the health of the MariaDB container has been added to the *local image* via the *Dockerfile*. In other words, the script is specific to IOTstack. @@ -87,11 +87,11 @@ The agent is invoked 30 seconds after the container starts, and every 30 seconds mysqld is alive ``` -3. If the command returned the expected response, the agent tests the responsiveness of the TCP port the `mysqld` daemon should be listening on (see [customising health-check](#healthCheckCustom)). +3. If the command returned the expected response, the agent tests the responsiveness of the TCP port the `mysqld` daemon should be listening on (see [customising health-check](#customising-health-check)). 4. If all of those steps succeed, the agent concludes that MariaDB is functioning properly and returns "healthy". -### monitoring health-check +### monitoring health-check Portainer's *Containers* display contains a *Status* column which shows health-check results for all containers that support the feature. @@ -124,7 +124,7 @@ Possible reply patterns are: mariadb Up About a minute (unhealthy) ``` -### customising health-check +### customising health-check You can customise the operation of the health-check agent by editing the `mariadb` service definition in your *Compose* file: diff --git a/docs/Containers/Mosquitto.md b/docs/Containers/Mosquitto.md index defa226e..8811111d 100644 --- a/docs/Containers/Mosquitto.md +++ b/docs/Containers/Mosquitto.md @@ -6,7 +6,7 @@ This document discusses an IOTstack-specific version of Mosquitto built on top o
-## References +## References - [*Eclipse Mosquitto* home](https://mosquitto.org) - [*GitHub*: eclipse/mosquitto](https://github.com/eclipse/mosquitto) @@ -14,7 +14,7 @@ This document discusses an IOTstack-specific version of Mosquitto built on top o - [Setting up passwords](https://www.youtube.com/watch?v=1msiFQT_flo) (video) - [Tutorial: from MQTT to InfluxDB via Node-Red](https://gist.github.com/Paraphraser/c9db25d131dd4c09848ffb353b69038f) -## Significant directories and files +## Significant directories and files ``` ~/IOTstack @@ -57,23 +57,23 @@ This document discusses an IOTstack-specific version of Mosquitto built on top o * You will normally need `sudo` to make changes in this area. * Each time Mosquitto starts, it automatically replaces anything originating in ❹ that has gone missing from ❼. This "self-repair" function is intended to provide reasonable assurance that Mosquitto will at least **start** instead of going into a restart loop. -## How Mosquitto gets built for IOTstack +## How Mosquitto gets built for IOTstack -### Mosquitto source code ([*GitHub*](https://github.com)) +### Mosquitto source code ([*GitHub*](https://github.com)) The source code for Mosquitto lives at [*GitHub* eclipse/mosquitto](https://github.com/eclipse/mosquitto). -### Mosquitto images ([*DockerHub*](https://hub.docker.com)) +### Mosquitto images ([*DockerHub*](https://hub.docker.com)) Periodically, the source code is recompiled and the resulting image is pushed to [eclipse-mosquitto](https://hub.docker.com/_/eclipse-mosquitto?tab=tags&page=1&ordering=last_updated) on *DockerHub*. -### IOTstack menu +### IOTstack menu When you select Mosquitto in the IOTstack menu, the *template service definition* is copied into the *Compose* file. > Under old menu, it is also copied to the *working service definition* and then not really used. -### IOTstack first run +### IOTstack first run On a first install of IOTstack, you run the menu, choose Mosquitto as one of your containers, and are told to do this: @@ -82,7 +82,7 @@ $ cd ~/IOTstack $ docker-compose up -d ``` -> See also the [Migration considerations](#migration) (below). +> See also the [Migration considerations](#migration-considerations) (below). `docker-compose` reads the *Compose* file. When it arrives at the `mosquitto` fragment, it finds: @@ -107,7 +107,7 @@ The *Dockerfile* begins with: FROM eclipse-mosquitto:latest ``` -> If you need to pin to a particular version of Mosquitto, the *Dockerfile* is the place to do it. See [Mosquitto version pinning](#versionPinning). +> If you need to pin to a particular version of Mosquitto, the *Dockerfile* is the place to do it. See [Mosquitto version pinning](#mosquitto-version-pinning). The `FROM` statement tells the build process to pull down the ***base image*** from [*DockerHub*](https://hub.docker.com). @@ -142,7 +142,7 @@ eclipse-mosquitto latest 46ad1893f049 4 weeks ago 8.31MB You will see the same pattern in Portainer, which reports the *base image* as "unused". You should not remove the *base* image, even though it appears to be unused. -### Migration considerations +### Migration considerations Under the original IOTstack implementation of Mosquitto (just "as it comes" from *DockerHub*), the service definition expected the configuration files to be at: @@ -203,7 +203,7 @@ Using `mosquitto.conf` as the example, assume you wish to use your existing file 5. If necessary, repeat these steps with `filter.acl`. -## Logging +## Logging Mosquitto logging is controlled by `mosquitto.conf`. This is the default configuration: @@ -246,9 +246,9 @@ $ sudo tail ~/IOTstack/volumes/mosquitto/log/mosquitto.log Logs written to `mosquitto.log` do not disappear when your IOTstack is restarted. They persist until you take action to prune the file. -## Security +## Security -### Configuring security +### Configuring security Mosquitto security is controlled by `mosquitto.conf`. These are the relevant directives: @@ -267,7 +267,7 @@ enabled | true | credentials optional | | enabled | false | credentials required | | -### Password file management +### Password file management The password file for Mosquitto is part of a mapped volume: @@ -285,7 +285,7 @@ The Mosquitto container performs self-repair each time the container is brought * If `false` then **all** MQTT requests will be rejected. -#### create username and password +#### create username and password To create a username and password, use the following as a template. @@ -301,9 +301,9 @@ $ docker exec mosquitto mosquitto_passwd -b /mosquitto/pwfile/pwfile hello world Note: -* See also [customising health-check](#healthCheckCustom). If you are creating usernames and passwords, you may also want to create credentials for the health-check agent. +* See also [customising health-check](#customising-health-check). If you are creating usernames and passwords, you may also want to create credentials for the health-check agent. -#### check password file +#### check password file There are two ways to verify that the password file exists and has the expected content: @@ -327,7 +327,7 @@ Each credential starts with the username and occupies one line in the file: hello:$7$101$ZFOHHVJLp2bcgX+h$MdHsc4rfOAhmGG+65NpIEJkxY0beNeFUyfjNAGx1ILDmI498o4cVOaD9vDmXqlGUH9g6AgHki8RPDEgjWZMkDA== ``` -#### remove entry from password file +#### remove entry from password file To remove an entry from the password file: @@ -335,7 +335,7 @@ To remove an entry from the password file: $ docker exec mosquitto mosquitto_passwd -D /mosquitto/pwfile/pwfile «username» ``` -#### reset the password file +#### reset the password file There are several ways to reset the password file. Your options are: @@ -366,7 +366,7 @@ There are several ways to reset the password file. Your options are: The result is an empty password file. -### Activate Mosquitto security +### Activate Mosquitto security 1. Use `sudo` and your favourite text editor to open the following file: @@ -409,15 +409,15 @@ There are several ways to reset the password file. Your options are: $ docker-compose restart mosquitto ``` -### Testing Mosquitto security +### Testing Mosquitto security -#### assumptions +#### assumptions 1. You have created at least one username ("hello") and password ("world"). 2. `password_file` is enabled. 3. `allow_anonymous` is `false`. -#### install testing tools +#### install testing tools If you do not have the Mosquitto clients installed on your Raspberry Pi (ie `$ which mosquitto_pub` does not return a path), install them using: @@ -425,7 +425,7 @@ If you do not have the Mosquitto clients installed on your Raspberry Pi (ie `$ w $ sudo apt install -y mosquitto-clients ``` -#### test: *anonymous access is prohibited* +#### test: *anonymous access is prohibited* Test **without** providing credentials: @@ -439,7 +439,7 @@ Note: * The error is the expected result and shows that Mosquitto will not allow anonymous access. -#### test: *access with credentials is permitted* +#### test: *access with credentials is permitted* Test with credentials @@ -452,7 +452,7 @@ Note: * The absence of any error message means the message was sent. Silence = success! -#### test: *round-trip with credentials is permitted* +#### test: *round-trip with credentials is permitted* Prove round-trip connectivity will succeed when credentials are provided. First, set up a subscriber as a background process. This mimics the role of a process like Node-Red: @@ -480,9 +480,9 @@ $ [1]+ Terminated mosquitto_sub -v -h 127.0.0.1 -p 1883 -t "/password/test" -F "%I %t %p" -u hello -P world ``` -## Container health check +## Container health check -### theory of operation +### theory of operation A script , or "agent", to assess the health of the Mosquitto container has been added to the *local image* via the *Dockerfile*. In other words, the script is specific to IOTstack. @@ -497,7 +497,7 @@ The agent is invoked 30 seconds after the container starts, and every 30 seconds * Subscribes to the same broker for the same topic for a single message event. * Compares the payload sent with the payload received. If the payloads (ie time-stamps) match, the agent concludes that the Mosquitto broker (the process running inside the same container) is functioning properly for round-trip messaging. -### monitoring health-check +### monitoring health-check Portainer's *Containers* display contains a *Status* column which shows health-check results for all containers that support the feature. @@ -543,7 +543,7 @@ Notes: * If you enable authentication for your Mosquitto broker, you will need to add `-u «user»` and `-P «password»` parameters to this command. * You should expect to see a new message appear approximately every 30 seconds. That indicates the health-check agent is functioning normally. Use control+c to terminate the command. -### customising health-check +### customising health-check You can customise the operation of the health-check agent by editing the `mosquitto` service definition in your *Compose* file: @@ -563,7 +563,7 @@ You can customise the operation of the health-check agent by editing the `mosqui Note: - * You will also need to use the same topic string in the `mosquitto_sub` command shown at [monitoring health-check](#healthCheckMonitor). + * You will also need to use the same topic string in the `mosquitto_sub` command shown at [monitoring health-check](#monitoring-health-check). 3. If you have enabled authentication for your Mosquitto broker service, you will need to provide appropriate credentials for your health-check agent: @@ -592,7 +592,7 @@ You can customise the operation of the health-check agent by editing the `mosqui You must remove the entire `healthcheck:` clause. -## Upgrading Mosquitto +## Upgrading Mosquitto You can update most containers like this: @@ -634,7 +634,7 @@ Your existing Mosquitto container continues to run while the rebuild proceeds. O The `prune` is the simplest way of cleaning up. The first call removes the old *local image*. The second call cleans up the old *base image*. -### Mosquitto version pinning +### Mosquitto version pinning If you need to pin Mosquitto to a particular version: @@ -670,7 +670,7 @@ Note: * As well as preventing Docker from updating the *base image*, pinning will also block incoming updates to the *Dockerfile* from a `git pull`. Nothing will change until you decide to remove the pin. -## About Port 9001 +## About Port 9001 Earlier versions of the IOTstack service definition for Mosquitto included two port mappings: diff --git a/docs/Containers/NextCloud.md b/docs/Containers/NextCloud.md index d26ff469..649aba0a 100644 --- a/docs/Containers/NextCloud.md +++ b/docs/Containers/NextCloud.md @@ -1,6 +1,6 @@ # Nextcloud -## Service definition +## Service definition This is the **core** of the IOTstack Nextcloud service definition: @@ -54,7 +54,7 @@ Under new-menu, the menu can generate random passwords for you. You can either u The passwords need to be set before you bring up the Nextcloud service for the first time but the following initialisation steps assume you might not have done that and always start over from a clean slate. -## Initialising Nextcloud +## Initialising Nextcloud 1. Be in the correct directory: @@ -108,7 +108,7 @@ The passwords need to be set before you bring up the Nextcloud service for the f * You **can't** use a multicast domain name (eg `myrpi.local`). An mDNS name will not work until Nextcloud has been initialised! * Once you have picked a connection method, **STICK TO IT**. - * You are only stuck with this restriction until Nextcloud has been initialised. You **can** (and should) fix it later by completing the steps in ["Access through untrusted domain"](#untrustedDomain). + * You are only stuck with this restriction until Nextcloud has been initialised. You **can** (and should) fix it later by completing the steps in ["Access through untrusted domain"](#access-through-untrusted-domain). 7. On a computer that is **not** the Raspberry Pi running Nextcloud, launch a browser and point to the Raspberry Pi running Nextcloud using your chosen connection method. Examples: @@ -243,7 +243,7 @@ See also: * [Nextcloud documentation - trusted domains](https://docs.nextcloud.com/server/21/admin_manual/installation/installation_wizard.html#trusted-domains). -### Using a DNS alias for your Nextcloud service +### Using a DNS alias for your Nextcloud service The examples above include using a DNS alias (a CNAME record) for your Nextcloud service. If you decide to do that, you may see this warning in the log: @@ -261,13 +261,13 @@ You can silence the warning by editing the Nextcloud service definition in `dock Nextcloud traffic is not encrypted. Do **not** expose it to the web by opening a port on your home router. Instead, use a VPN like Wireguard to provide secure access to your home network, and let your remote clients access Nextcloud over the VPN tunnel. -## Container health check +## Container health check A script , or "agent", to assess the health of the MariaDB container has been added to the *local image* via the *Dockerfile*. In other words, the script is specific to IOTstack. Because it is an instance of MariaDB, Nextcloud_DB inherits the health-check agent. See the [IOTstack MariaDB](MariaDB.md) documentation for more information. -## Keeping Nextcloud up-to-date +## Keeping Nextcloud up-to-date To update the `nextcloud` container: @@ -290,7 +290,7 @@ $ docker system prune The first "prune" removes the old *local* image, the second removes the old *base* image. -## Backups +## Backups Nextcloud is currently excluded from the IOTstack-supplied backup scripts due to its potential size. diff --git a/docs/Containers/Node-RED.md b/docs/Containers/Node-RED.md index ad1d629c..32d12276 100644 --- a/docs/Containers/Node-RED.md +++ b/docs/Containers/Node-RED.md @@ -1,13 +1,13 @@ # Node-RED -## References +## References - [nodered.org home](https://nodered.org/) - [GitHub: node-red/node-red-docker](https://github.com/node-red/node-red-docker) - [DockerHub: nodered/node-red](https://hub.docker.com/r/nodered/node-red) - [Tutorial: from MQTT to InfluxDB via Node-Red](https://gist.github.com/Paraphraser/c9db25d131dd4c09848ffb353b69038f) -## Significant files +## Significant files ``` ~/IOTstack @@ -33,7 +33,7 @@ 6. Data directory (mapped volume). 7. SSH directory (mapped volume). -## How Node-RED gets built for IOTstack +## How Node-RED gets built for IOTstack ### Node-RED source code ([GitHub](https://github.com)) @@ -64,7 +64,7 @@ You choose add-on nodes from a supplementary menu. We recommend accepting the de Key points: * Under new menu, you must press the right arrow to access the supplementary menu. Under old menu, the list of add-on nodes is displayed automatically. -* Do not be concerned if you can't find an add-on node you need in the list. You can also add nodes via Manage Palette once Node-RED is running. See [adding extra nodes](#addingExtraNodes). +* Do not be concerned if you can't find an add-on node you need in the list. You can also add nodes via Manage Palette once Node-RED is running. See [adding extra nodes](#adding-extra-nodes). Choosing add-on nodes in the menu causes the *Dockerfile* to be created. @@ -133,9 +133,9 @@ You will see the same pattern in Portainer, which reports the *base image* as "u You should not remove the *base* image, even though it appears to be unused. -## Securing Node-RED +## Securing Node-RED -### Setting an encryption key for your credentials +### Setting an encryption key for your credentials After you install Node-RED, you should set an encryption key. Completing this step will silence the warning you will see when you run: @@ -190,7 +190,7 @@ $ cd ~/IOTstack $ docker-compose restart nodered ``` -### Setting a username and password for Node-RED +### Setting a username and password for Node-RED To secure Node-RED you need a password hash. Run the following command, replacing `PASSWORD` with your own password: @@ -206,7 +206,7 @@ $2a$08$gTdx7SkckJVCw1U98o4r0O7b8P.gd5/LAPlZI6geg5LRg4AUKuDhS Copy that text to your clipboard, then follow the instructions at [Node-RED User Guide - Securing Node-RED - Username & Password-based authentication](https://nodered.org/docs/user-guide/runtime/securing-node-red#usernamepassword-based-authentication). -## Referring to other containers +## Referring to other containers Node-RED can run in two modes. By default, it runs in "non-host mode" but you can also move the container to "host mode" by editing the Node-RED service definition in your *Compose* file to: @@ -218,7 +218,7 @@ Node-RED can run in two modes. By default, it runs in "non-host mode" but you ca 2. Remove the `ports` directive and the mapping of port 1880. -### When Node-RED is not in host mode +### When Node-RED is not in host mode Most examples on the web assume Node-RED and other services in the MING (Mosquitto, InfluxDB, Node-RED, Grafana) stack have been installed natively, rather than in Docker containers. Those examples typically include the loopback address + port syntax, like this: @@ -242,7 +242,7 @@ influxdb:8086 Behind the scenes, Docker maintains a table similar to an `/etc/hosts` file mapping container names to the IP addresses on the internal bridged network that are assigned, dynamically, by Docker when it spins up each container. -### When Node-RED is in host mode +### When Node-RED is in host mode This is where you use loopback+port syntax, such as the following to communicate with Mosquitto: @@ -252,7 +252,7 @@ This is where you use loopback+port syntax, such as the following to communicate What actually occurs is that Docker is listening to external port 1883 on behalf of Mosquitto. It receives the packet and routes it (layer three) to the internal bridged network, performing network address translation (NAT) along the way to map the external port to the internal port. Then the packet is delivered to Mosquitto. The reverse happens when Mosquitto replies. It works but is less efficient than when all containers are in non-host mode. -## GPIO Access +## GPIO Access To communicate with your Raspberry Pi's GPIO you need to do the following: @@ -263,7 +263,7 @@ To communicate with your Raspberry Pi's GPIO you need to do the following: $ sudo apt install pigpio python-pigpio python3-pigpio ``` -2. Install the `node-red-node-pi-gpiod` node. See [Adding extra nodes](#addingExtraNodes). It allows you to connect to multiple Pis from the same Node-RED service. +2. Install the `node-red-node-pi-gpiod` node. See [Adding extra nodes](#adding-extra-nodes). It allows you to connect to multiple Pis from the same Node-RED service. 3. Make sure that the `pigpdiod` daemon is running. The recommended method is listed [here](https://github.com/node-red/node-red-nodes/tree/master/hardware/pigpiod). In essence, you need to: - Use `sudo` to edit `/etc/rc.local`; @@ -279,7 +279,7 @@ To communicate with your Raspberry Pi's GPIO you need to do the following: 4. Drag a gpio node onto the canvas and configure it using the IP address of your Raspberry Pi (eg 192.168.1.123). Don't try to use 127.0.0.1 because that is the loopback address of the Node-RED container. -## Sharing files between Node-RED and the Raspberry Pi +## Sharing files between Node-RED and the Raspberry Pi Containers run in a sandboxed environment. A process running inside a container can't see the Raspberry Pi's file system. Neither can a process running outside a container access files inside the container. @@ -370,7 +370,7 @@ Deploying the flow and clicking on the Inject node results in the debug message You can reverse this process. Any file you place within the path `~/IOTstack/volumes/nodered/data` can be read by a "File in" node. -## Executing commands outside the Node-RED container +## Executing commands outside the Node-RED container A reasonably common requirement in a Node-RED flow is the ability to execute a command on the host system. The standard tool for this is an "exec" node. @@ -398,7 +398,7 @@ The same thing will happen if a Node-RED "exec" node executes that `grep` comman Docker doesn't provide any mechanism for a container to execute an arbitrary command **outside** of its container. A workaround is to utilise SSH. This remainder of this section explains how to set up the SSH scaffolding so that "exec" nodes running in a Node-RED container can invoke arbitrary commands **outside** container-space. -### Task Goal +### Task Goal Be able to use a Node-RED `exec` node to perform the equivalent of: @@ -411,14 +411,14 @@ where: * `«HOSTNAME»` is any host under your control (not just the Raspberry Pi running IOTstack); and * `«COMMAND»` is any command known to the target host. -### Assumptions +### Assumptions * [SensorsIot/IOTstack](https://github.com/SensorsIot/IOTstack) is installed on your Raspberry Pi. * The Node-RED container is running. These instructions are specific to IOTstack but the underlying concepts should apply to any installation of Node-RED in a Docker container. -### Executing commands "inside" a container +### Executing commands "inside" a container These instructions make frequent use of the ability to run commands "inside" the Node-RED container. For example, suppose you want to execute: @@ -464,11 +464,11 @@ You have several options: 3. Run the command from Portainer by selecting the container, then clicking the ">_ console" link. This is identical to opening a shell. -### Variable definitions +### Variable definitions You will need to have a few concepts clear in your mind before you can set up SSH successfully. I use double-angle quote marks (guillemets) to mean "substitute the appropriate value here". -#### «HOSTNAME» (required) +#### «HOSTNAME» (required) The name of your Raspberry Pi. When you first booted your RPi, it had the name "raspberrypi" but you probably changed it using `raspi-config`. Example: @@ -513,7 +513,7 @@ The user ID of the account on «HOSTNAME» where you want Node-RED flows to be a pi ``` -### Step 1: *Generate SSH key-pair for Node-RED* (one time) +### Step 1: *Generate SSH key-pair for Node-RED* (one time) Create a key-pair for Node-RED. This is done by executing the `ssh-keygen` command **inside** the container: @@ -579,7 +579,7 @@ and check to make sure that only the key(s) you wanted were added. If you do not see an indication that a key has been added, you may need to retrace your steps. -### Step 3: *Perform the recommended test* +### Step 3: *Perform the recommended test* The output above recommends a test. The test needs to be run **inside** the Node-RED container so the syntax is: @@ -600,9 +600,9 @@ If everything works as expected, you should see a list of the files in your IOTs Assuming success, think about what just happened? You told SSH **inside** the Node-RED container to run the `ls` command **outside** the container on your Raspberry Pi. You broke through the containerisation. -### Understanding what's where and what each file does +### Understanding what's where and what each file does -#### What files are where +#### What files are where Six files are relevant to Node-RED's ability to execute commands outside of container-space: @@ -615,7 +615,7 @@ Six files are relevant to Node-RED's ability to execute commands outside of cont Unless you take precautions, those keys will change whenever your Raspberry Pi is rebuilt from scratch and that **will** stop SSH from working. - You can recover by re-running [`ssh-copy-id`](#sshStep2). + You can recover by re-running [`ssh-copy-id`](#step-2-exchange-keys-with-target-hosts-once-per-target-host). * in `~/IOTstack/volumes/nodered/ssh`: @@ -630,7 +630,7 @@ Six files are relevant to Node-RED's ability to execute commands outside of cont If you lose or destroy these keys, SSH **will** stop working. - You can recover by [generating new keys](#sshStep1) and then re-running [`ssh-copy-id`](#sshStep2). + You can recover by [generating new keys](#step-1-generate-ssh-key-pair-for-node-red-one-time) and then re-running [`ssh-copy-id`](#sshStep2). - `known_hosts` @@ -652,7 +652,7 @@ Six files are relevant to Node-RED's ability to execute commands outside of cont Note that providing the correct password at the command line will auto-repair the `authorized_keys` file. -#### What each file does +#### What each file does SSH running **inside** the Node-RED container uses the Node-RED container's private key to provide assurance to SSH running **outside** the container that it (the Node-RED container) is who it claims to be. @@ -662,7 +662,7 @@ SSH running **outside** container-space uses the Raspberry Pi's private host key SSH running **inside** the Node-RED container verifies that assurance by using its copy of the Raspberry Pi's public host key stored in `known_hosts`. -### Config file (optional) +### Config file (optional) You don't **have** to do this step but it will simplify your exec node commands and reduce your maintenance problems if you do. @@ -739,7 +739,7 @@ $ sudo chown root:root config $ sudo mv config ./volumes/nodered/ssh ``` -#### Re-test with config file in place +#### Re-test with config file in place The previous test used this syntax: @@ -761,7 +761,7 @@ $ docker exec nodered ssh «HOSTNAME» ls -1 /home/pi/IOTstack The result should be the same as the earlier test. -### A test flow +### A test flow ![node-red-exec-node-ssh-test](./images/nodered-exec-node-ssh-test.jpeg) @@ -803,7 +803,7 @@ In the Node-RED GUI: The first line is the result of running the command inside the Node-RED container. The second line is the result of running the same command outside the Node-RED container on the Raspberry Pi. -### Suppose you want to add another «HOSTNAME» +### Suppose you want to add another «HOSTNAME» 1. Exchange keys with the new target host using: @@ -819,7 +819,7 @@ In the Node-RED GUI: to define the new host. Remember to use `sudo` to edit the file. There is no need to restart Node-RED or recreate the container. -## Upgrading Node-RED +## Upgrading Node-RED You can update most containers like this: @@ -860,7 +860,7 @@ Your existing Node-RED container continues to run while the rebuild proceeds. On The `prune` is the simplest way of cleaning up old images. Sometimes you need to run this twice, the first time to clean up the old local image, the second time for the old base image. -## Customising Node-RED +## Customising Node-RED You customise your *local* image of Node-RED by making changes to: @@ -878,7 +878,7 @@ $ docker system prune The `--build` option on the `up` command (as distinct from a `docker-compose build` command) works in this situation because you've made a substantive change to your *Dockerfile*. -### Node.js version +### Node.js version Out of the box, IOTstack starts the Node-RED *Dockerfile* with: @@ -900,9 +900,9 @@ In the unlikely event that you need to run an add-on node that needs version 10 FROM nodered/node-red:latest-10 ``` -Once you have made that change, follow the steps at [apply *Dockerfile* changes](#applyDockerfileChange). +Once you have made that change, follow the steps at [apply *Dockerfile* changes](#upgrading-node-red). -### Adding extra packages +### Adding extra packages As well as providing the Node-RED service, the nodered container is an excellent testbed. Installing the DNS tools, Mosquitto clients and tcpdump will help you to figure out what is going on **inside** container-space. @@ -924,13 +924,13 @@ RUN apk update && apk add --no-cache eudev-dev mosquitto-clients bind-tools tcpd You can add as many extra packages as you like. They will persist until you change the *Dockerfile* again. -Once you have made this change, follow the steps at [apply *Dockerfile* changes](#applyDockerfileChange). +Once you have made this change, follow the steps at [apply *Dockerfile* changes](#upgrading-node-red). -### Adding extra nodes +### Adding extra nodes You can install nodes by: -1. Adding nodes to the *Dockerfile* and then following the steps at [apply *Dockerfile* changes](#applyDockerfileChange). +1. Adding nodes to the *Dockerfile* and then following the steps at [apply *Dockerfile* changes](#upgrading-node-red). This is also what will happen if you re-run the menu and change the selected nodes, except that the menu will also blow away any other customisations you may have made to your *Dockerfile*. @@ -994,7 +994,7 @@ If you're wondering about "backup", nodes installed via: Basically, if you're running IOTstack backups then your add-on nodes will be backed-up. -### Node precedence +### Node precedence Add-on nodes that are installed via *Dockerfile* wind up at the **internal** path: @@ -1034,7 +1034,7 @@ take precedence over those installed at: Or, to put it more simply: in any contest, Manage Palette prevails over *Dockerfile*. -### Resolving node duplication +### Resolving node duplication Sometimes, even when you are 100% certain that **you** didn't do it, an add-on node will turn up in both places. There is probably some logical reason for this but I don't know what it is. diff --git a/docs/Containers/Portainer-ce.md b/docs/Containers/Portainer-ce.md index 3ee4484b..dbcc386d 100644 --- a/docs/Containers/Portainer-ce.md +++ b/docs/Containers/Portainer-ce.md @@ -1,11 +1,11 @@ # Portainer CE -## References +## References - [Docker](https://hub.docker.com/r/portainer/portainer-ce/) - [Website](https://www.portainer.io/portainer-ce/) -## Definition +## Definition - "#yourip" means any of the following: @@ -13,7 +13,7 @@ - the multicast domain name of your Raspberry Pi (eg `iot-hub.local`) - the domain name of your Raspberry Pi (eg `iot-hub.mydomain.com`) -## About *Portainer CE* +## About *Portainer CE* *Portainer CE* (Community Edition) is an application for managing Docker. It is a successor to *Portainer*. According to [the *Portainer CE* documentation](https://www.portainer.io/2020/08/portainer-ce-2-0-what-to-expect/) @@ -21,7 +21,7 @@ From that it should be clear that *Portainer* is deprecated and that *Portainer CE* is the way forward. -## Installing *Portainer CE* +## Installing *Portainer CE* Run the menu: @@ -40,7 +40,7 @@ Ignore any message like this: > WARNING: Found orphan containers (portainer) for this project … -## First run of *Portainer CE* +## First run of *Portainer CE* In your web browser navigate to `#yourip:9000/`: @@ -51,7 +51,7 @@ From there, you can click on the "Local" group and take a look around. One of th There are 'Quick actions' to view logs and other stats. This can all be done from terminal commands but *Portainer CE* makes it easier. -## Setting the Public IP address for your end-point +## Setting the Public IP address for your end-point If you click on a "Published Port" in the "Containers" list, your browser may return an error saying something like "can't connect to server" associated with an IP address of "0.0.0.0". @@ -79,7 +79,7 @@ Keep in mind that clicking on a "Published Port" does not guarantee that your br > All things considered, you will get more consistent behaviour if you simply bookmark the URLs you want to use for your IOTstack services. -## If you forget your password +## If you forget your password If you forget the password you created for *Portainer CE*, you can recover by doing the following: @@ -92,5 +92,5 @@ $ docker-compose start portainer-ce Then, follow the steps in: -1. [First run of *Portainer CE*](#firstRun); and -2. [Setting the Public IP address for your end-point](#setPublicIP). +1. [First run of *Portainer CE*](#first-run-of-portainer-ce); and +2. [Setting the Public IP address for your end-point](#setting-the-public-ip-address-for-your-end-point). diff --git a/docs/Containers/Prometheus.md b/docs/Containers/Prometheus.md index c8a325e5..f9152209 100644 --- a/docs/Containers/Prometheus.md +++ b/docs/Containers/Prometheus.md @@ -1,6 +1,6 @@ # Prometheus -## References +## References * [*Prometheus* home](https://prometheus.io) * *GitHub*: @@ -15,7 +15,7 @@ - [*CAdvisor*](https://hub.docker.com/r/zcube/cadvisor) - [*Node Exporter*](https://hub.docker.com/r/prom/node-exporter) -## Overview +## Overview Prometheus is a collection of three containers: @@ -23,11 +23,11 @@ Prometheus is a collection of three containers: * *CAdvisor* * *Node Exporter* -The [default configuration](#activeConfig) for *Prometheus* supplied with IOTstack scrapes information from all three containers. +The [default configuration](#active-configuration-file) for *Prometheus* supplied with IOTstack scrapes information from all three containers. -## Installing Prometheus +## Installing Prometheus -### *if you are running New Menu …* +### *if you are running New Menu …* When you select *Prometheus* in the IOTstack menu, you must also select: @@ -36,7 +36,7 @@ When you select *Prometheus* in the IOTstack menu, you must also select: If you do not select all three containers, Prometheus will not start. -### *if you are running Old Menu …* +### *if you are running Old Menu …* When you select *Prometheus* in the IOTstack menu, the service definition includes the three containers: @@ -44,7 +44,7 @@ When you select *Prometheus* in the IOTstack menu, the service definition includ * *CAdvisor* * *Node Exporter* -## Significant directories and files +## Significant directories and files ``` ~/IOTstack @@ -75,25 +75,25 @@ When you select *Prometheus* in the IOTstack menu, the service definition includ 5. The *working service definition* (only relevant to old-menu, copied from ❶). 6. The *Compose* file (includes ❶). 7. The *persistent storage area*. -8. The [configuration directory](#configDir). +8. The [configuration directory](#configuration-directory). -## How *Prometheus* gets built for IOTstack +## How *Prometheus* gets built for IOTstack -### *Prometheus* source code ([*GitHub*](https://github.com)) +### *Prometheus* source code ([*GitHub*](https://github.com)) The source code for *Prometheus* lives at [*GitHub* prometheus/prometheus](https://github.com/prometheus/prometheus). -### *Prometheus* images ([*DockerHub*](https://hub.docker.com)) +### *Prometheus* images ([*DockerHub*](https://hub.docker.com)) Periodically, the source code is recompiled and the resulting image is pushed to [prom/prometheus](https://hub.docker.com/r/prom/prometheus) on *DockerHub*. -### IOTstack menu +### IOTstack menu When you select *Prometheus* in the IOTstack menu, the *template service definition* is copied into the *Compose* file. > Under old menu, it is also copied to the *working service definition* and then not really used. -### IOTstack first run +### IOTstack first run On a first install of IOTstack, you run the menu, choose *Prometheus* as one of your containers, and are told to do this: @@ -124,7 +124,7 @@ The *Dockerfile* begins with: FROM prom/prometheus:latest ``` -> If you need to pin to a particular version of *Prometheus*, the *Dockerfile* is the place to do it. See [*Prometheus* version pinning](#versionPinning). +> If you need to pin to a particular version of *Prometheus*, the *Dockerfile* is the place to do it. See [*Prometheus* version pinning](#prometheus-version-pinning). The `FROM` statement tells the build process to pull down the ***base image*** from [*DockerHub*](https://hub.docker.com). @@ -156,15 +156,15 @@ prom/prometheus latest 3f9575991a6c 3 days ago 169MB You will see the same pattern in Portainer, which reports the *base image* as "unused". You should not remove the *base* image, even though it appears to be unused. -### Dependencies: *CAdvisor* and *Node Exporter* +### Dependencies: *CAdvisor* and *Node Exporter* The *CAdvisor* and *Node Exporter* are included in the *Prometheus* service definition as dependent containers. What that means is that each time you start *Prometheus*, `docker-compose` ensures that *CAdvisor* and *Node Exporter* are already running, and keeps them running. -The [default configuration](#activeConfig) for *Prometheus* assumes *CAdvisor* and *Node Exporter* are running and starts scraping information from those targets as soon as it launches. +The [default configuration](#active-configuration-file) for *Prometheus* assumes *CAdvisor* and *Node Exporter* are running and starts scraping information from those targets as soon as it launches. -## Configuring **Prometheus** +## Configuring **Prometheus** -### Configuration directory +### Configuration directory The configuration directory for the IOTstack implementation of *Prometheus* is at the path: @@ -179,9 +179,9 @@ That directory contains two files: If you delete either file, *Prometheus* will replace it with a default the next time the container starts. This "self-repair" function is intended to provide reasonable assurance that *Prometheus* will at least **start** instead of going into a restart loop. -Unless you [decide to change it](#environmentVars), the `config` folder and its contents are owned by "pi:pi". This means you can edit the files in the configuration directory without needing the `sudo` command. Ownership is enforced each time the container restarts. +Unless you [decide to change it](#environment-variables), the `config` folder and its contents are owned by "pi:pi". This means you can edit the files in the configuration directory without needing the `sudo` command. Ownership is enforced each time the container restarts. -#### Active configuration file +#### Active configuration file The file named `config.yml` is the active configuration. This is the file you should edit if you want to make changes. The default structure of the file is: @@ -211,7 +211,7 @@ Note: * The YAML parser used by *Prometheus* seems to be ***exceptionally*** sensitive to syntax errors (far less tolerant than `docker-compose`). For this reason, you should **always** check the *Prometheus* log after any configuration change. -#### Reference configuration file +#### Reference configuration file The file named `prometheus.yml` is a reference configuration. It is a **copy** of the original configuration file that ships inside the *Prometheus* container at the path: @@ -229,7 +229,7 @@ $ docker-compose restart prometheus $ docker logs prometheus ``` -### Environment variables +### Environment variables The IOTstack implementation of *Prometheus* supports two environment variables: @@ -239,11 +239,11 @@ environment: - IOTSTACK_GID=1000 ``` -Those variables control ownership of the [Configuration directory](#configDir) and its contents. Those environment variables are present in the standard IOTstack service definition for *Prometheus* and have the effect of assigning ownership to "pi:pi". +Those variables control ownership of the [Configuration directory](#configuration-directory) and its contents. Those environment variables are present in the standard IOTstack service definition for *Prometheus* and have the effect of assigning ownership to "pi:pi". -If you delete those environment variables from your *Compose* file, the [Configuration directory](#configDir) will be owned by "nobody:nobody"; otherwise the directory and its contents will be owned by whatever values you pass for those variables. +If you delete those environment variables from your *Compose* file, the [Configuration directory](#configuration-directory) will be owned by "nobody:nobody"; otherwise the directory and its contents will be owned by whatever values you pass for those variables. -### Migration considerations +### Migration considerations Under the original IOTstack implementation of *Prometheus* (just "as it comes" from *DockerHub*), the service definition expected the configuration file to be at: @@ -274,7 +274,7 @@ Note: * The YAML parser used by *Prometheus* is very sensitive to syntax errors. Always check the *Prometheus* log after any configuration change. -## Upgrading *Prometheus* +## Upgrading *Prometheus* You can update `cadvisor` and `nodeexporter` like this: @@ -316,7 +316,7 @@ Your existing *Prometheus* container continues to run while the rebuild proceeds The `prune` is the simplest way of cleaning up. The first call removes the old *local image*. The second call cleans up the old *base image*. -### *Prometheus* version pinning +### *Prometheus* version pinning If you need to pin *Prometheus* to a particular version: diff --git a/docs/Containers/Python.md b/docs/Containers/Python.md index 24e613bb..caa54606 100644 --- a/docs/Containers/Python.md +++ b/docs/Containers/Python.md @@ -1,12 +1,12 @@ # Python -## references +## references * [Python.org](https://www.python.org) * [Dockerhub image library](https://hub.docker.com/_/python) * [GitHub docker-library/python](https://github.com/docker-library/python) -## selecting Python in the IOTstack menu +## selecting Python in the IOTstack menu When you select Python in the menu: @@ -48,7 +48,7 @@ When you select Python in the menu: * This service definition is for "new menu" (master branch). The only difference with "old menu" (old-menu branch) is the omission of the last two lines. -### customising your Python service definition +### customising your Python service definition The service definition contains a number of customisation points: @@ -76,7 +76,7 @@ $ cd ~/IOTstack $ docker-compose up -d python ``` -## Python - first launch +## Python - first launch After running the menu, you are told to run the commands: @@ -145,7 +145,7 @@ This is what happens: Pressing control+c terminates the log display but does not terminate the running container. -## stopping the Python service +## stopping the Python service To stop the container from running, either: @@ -163,7 +163,7 @@ To stop the container from running, either: $ docker-compose rm --force --stop -v python ``` -## starting the Python service +## starting the Python service To bring up the container again after you have stopped it, either: @@ -181,7 +181,7 @@ To bring up the container again after you have stopped it, either: $ docker-compose up -d python ``` -## Python - second-and-subsequent launch +## Python - second-and-subsequent launch Each time you launch the Python container *after* the first launch: @@ -189,7 +189,7 @@ Each time you launch the Python container *after* the first launch: 2. The `docker-entrypoint.sh` script runs and performs "self-repair" by replacing any files that have gone missing from the persistent storage area. Self-repair does **not** overwrite existing files! 3. The `app.py` Python script is run. -## when things go wrong - check the log +## when things go wrong - check the log If the container misbehaves, the log is your friend: @@ -197,7 +197,7 @@ If the container misbehaves, the log is your friend: $ docker logs python ``` -## project development life-cycle +## project development life-cycle It is **critical** that you understand that **all** of your project development should occur within the folder: @@ -207,7 +207,7 @@ It is **critical** that you understand that **all** of your project development So long as you are performing some sort of routine backup (either with a supplied script or a third party solution like [Paraphraser/IOTstackBackup](https://github.com/Paraphraser/IOTstackBackup)), your work will be protected. -### getting started +### getting started Start by editing the file: @@ -228,7 +228,7 @@ $ cd ~/IOTstack $ docker-compose restart python ``` -### reading and writing to disk +### reading and writing to disk Consider this line in the service definition: @@ -255,7 +255,7 @@ What it means is that: If your script writes into any other directory inside the container, the data will be lost when the container re-launches. -### getting a clean slate +### getting a clean slate If you make a mess of things and need to start from a clean slate, erase the persistent storage area: @@ -268,7 +268,7 @@ $ docker-compose up -d python The container will re-initialise the persistent storage area from its defaults. -### adding packages +### adding packages As you develop your project, you may find that you need to add supporting packages. For this example, we will assume you want to add "[Flask](https://pypi.org/project/Flask/)" and "[beautifulsoup4](https://pypi.org/project/beautifulsoup4/)". @@ -322,7 +322,7 @@ To make *Flask* and *beautifulsoup4* a permanent part of your container: Flask==2.0.1 ``` -5. Continue your development work by returning to [getting started](#gettingStarted). +5. Continue your development work by returning to [getting started](#getting-started). Note: @@ -346,11 +346,11 @@ Note: The `requirements.txt` file will be recreated and it will be a copy of the version in the *services* directory as of the last image rebuild. -### making your own Python script the default +### making your own Python script the default -Suppose the Python script you have been developing reaches a major milestone and you decide to "freeze dry" your work up to that point so that it becomes the default when you ask for a [clean slate](#cleanSlate). Proceed like this: +Suppose the Python script you have been developing reaches a major milestone and you decide to "freeze dry" your work up to that point so that it becomes the default when you ask for a [clean slate](#getting-a-clean-slate). Proceed like this: -1. If you have added any packages by following the steps in [adding packages](#addingPackages), run the following command: +1. If you have added any packages by following the steps in [adding packages](#adding-packages), run the following command: ```bash $ docker exec python bash -c 'pip3 freeze >requirements.txt' @@ -412,11 +412,11 @@ Suppose the Python script you have been developing reaches a major milestone and $ docker system prune -f ``` -### canning your project +### canning your project Suppose your project has reached the stage where you wish to put it into production as a service under its own name. Make two further assumptions: -1. You have gone through the steps in [making your own Python script the default](#scriptBaking) and you are **certain** that the content of `./services/python/app` correctly captures your project. +1. You have gone through the steps in [making your own Python script the default](#making-your-own-python-script-the-default) and you are **certain** that the content of `./services/python/app` correctly captures your project. 2. You want to give your project the name "wishbone". Proceed like this: @@ -479,7 +479,7 @@ Remember: ~/IOTstack/volumes/wishbone/app ``` -## routine maintenance +## routine maintenance To make sure you are running from the most-recent **base** image of Python from Dockerhub: @@ -503,4 +503,4 @@ The old base image can't be removed until the old local image has been removed, Note: -* If you have followed the steps in [canning your project](#scriptCanning) and your service has a name other than `python`, just substitute the new name where you see `python` in the two `dockerc-compose` commands. +* If you have followed the steps in [canning your project](#canning-your-project) and your service has a name other than `python`, just substitute the new name where you see `python` in the two `dockerc-compose` commands. diff --git a/docs/Containers/Telegraf.md b/docs/Containers/Telegraf.md index f317f16a..0bc37dbd 100644 --- a/docs/Containers/Telegraf.md +++ b/docs/Containers/Telegraf.md @@ -7,13 +7,13 @@ The purpose of the Dockerfile is to: * tailor the default configuration to be IOTstack-ready; and * enable the container to perform self-repair if essential elements of the persistent storage area disappear. -## References +## References - [*influxdata Telegraf* home](https://www.influxdata.com/time-series-platform/telegraf/) - [*GitHub*: influxdata/influxdata-docker/telegraf](https://github.com/influxdata/influxdata-docker/tree/master/telegraf) - [*DockerHub*: influxdata Telegraf](https://hub.docker.com/_/telegraf) -## Significant directories and files +## Significant directories and files ``` ~/IOTstack @@ -38,14 +38,14 @@ The purpose of the Dockerfile is to: 1. The *Dockerfile* used to customise Telegraf for IOTstack. 2. A replacement for the `telegraf` container script of the same name, extended to handle container self-repair. -3. The *additions folder*. See [Applying optional additions](#optionalAdditions). +3. The *additions folder*. See [Applying optional additions](#applying-optional-additions). 4. The *auto_include folder*. Additions automatically applied to - `telegraf.conf`. See [Automatic includes to telegraf.conf](#autoInclude). + `telegraf.conf`. See [Automatic includes to telegraf.conf](#automatic-includes-to-telegrafconf). 5. The *template service definition*. 6. The *working service definition* (only relevant to old-menu, copied from ❹). 7. The *persistent storage area* for the `telegraf` container. -8. A working copy of the *additions folder* (copied from ❸). See [Applying optional additions](#optionalAdditions). -9. The *reference configuration file*. See [Changing Telegraf's configuration](#editConfiguration). +8. A working copy of the *additions folder* (copied from ❸). See [Applying optional additions](#applying-optional-additions). +9. The *reference configuration file*. See [Changing Telegraf's configuration](#changing-telegrafs-configuration). 10. The *active configuration file*. A subset of ➒ altered to support communication with InfluxDB running in a container in the same IOTstack instance. Everything in the persistent storage area ❼: @@ -53,19 +53,19 @@ Everything in the persistent storage area ❼: * will be replaced if it is not present when the container starts; but * will never be overwritten if altered by you. -## How Telegraf gets built for IOTstack +## How Telegraf gets built for IOTstack -### Telegraf images ([*DockerHub*](https://hub.docker.com)) +### Telegraf images ([*DockerHub*](https://hub.docker.com)) Periodically, the source code is recompiled and the resulting image is pushed to [influxdata Telegraf](https://hub.docker.com/_/telegraf?tab=tags&page=1&ordering=last_updated) on *DockerHub*. -### IOTstack menu +### IOTstack menu When you select Telegraf in the IOTstack menu, the *template service definition* is copied into the *Compose* file. > Under old menu, it is also copied to the *working service definition* and then not really used. -### IOTstack first run +### IOTstack first run On a first install of IOTstack, you run the menu, choose your containers, and are told to do this: @@ -74,7 +74,7 @@ $ cd ~/IOTstack $ docker-compose up -d ``` -> See also the [Migration considerations](#migration) (below). +> See also the [Migration considerations](#migration-considerations) (below). `docker-compose` reads the *Compose* file. When it arrives at the `telegraf` fragment, it finds: @@ -99,7 +99,7 @@ The *Dockerfile* begins with: FROM telegraf:latest ``` -> If you need to pin to a particular version of Telegraf, the *Dockerfile* is the place to do it. See [Telegraf version pinning](#versionPinning). +> If you need to pin to a particular version of Telegraf, the *Dockerfile* is the place to do it. See [Telegraf version pinning](#telegraf-version-pinning). The `FROM` statement tells the build process to pull down the ***base image*** from [*DockerHub*](https://hub.docker.com). @@ -132,7 +132,7 @@ telegraf latest a721ac170fad 3 days ago 273MB You will see the same pattern in *Portainer*, which reports the ***base image*** as "unused". You should not remove the ***base*** image, even though it appears to be unused. -### Migration considerations +### Migration considerations Under the original IOTstack implementation of Telegraf (just "as it comes" from *DockerHub*), the service definition expected `telegraf.conf` to be at: @@ -152,9 +152,9 @@ With one exception, all prior and current versions of the default configuration > In other words, once you strip away comments and blank lines, and remove any "active" configuration options that simply repeat their default setting, you get the same subset of "active" configuration options. The default configuration file supplied with gcgarner/IOTstack is available [here](https://github.com/gcgarner/IOTstack/blob/master/.templates/telegraf/telegraf.conf) if you wish to refer to it. -The exception is `[[inputs.mqtt_consumer]]` which is now provided as an optional addition. If your existing Telegraf configuration depends on that input, you will need to apply it. See [applying optional additions](#optionalAdditions). +The exception is `[[inputs.mqtt_consumer]]` which is now provided as an optional addition. If your existing Telegraf configuration depends on that input, you will need to apply it. See [applying optional additions](#applying-optional-additions). -## Logging +## Logging You can inspect Telegraf's log by: @@ -164,7 +164,7 @@ $ docker logs telegraf These logs are ephemeral and will disappear when your Telegraf container is rebuilt. -### log message: *database "telegraf" creation failed* +### log message: *database "telegraf" creation failed* The following log message can be misleading: @@ -176,7 +176,7 @@ If InfluxDB is not running when Telegraf starts, the `depends_on:` clause in Tel What this error message *usually* means is that Telegraf has tried to communicate with InfluxDB before the latter is ready to accept connections. Telegraf typically retries after a short delay and is then able to communicate with InfluxDB. -## Changing Telegraf's configuration +## Changing Telegraf's configuration The first time you launch the Telegraf container, the following structure will be created in the persistent storage area: @@ -202,7 +202,7 @@ The file: - is created by removing all comment lines and blank lines from `telegraf-reference.conf`, leaving only the "active" configuration options, and then adding options necessary for IOTstack. - is less than 30 lines and is significantly easier to understand than `telegraf-reference.conf`. -* `inputs.docker.conf` – see [Applying optional additions](#optionalAdditions) below. +* `inputs.docker.conf` – see [Applying optional additions](#applying-optional-additions) below. The intention of this structure is that you: @@ -217,7 +217,7 @@ $ cd ~/IOTstack $ docker-compose restart telegraf ``` -### Automatic includes to telegraf.conf +### Automatic includes to telegraf.conf * `inputs.docker.conf` instructs Telegraf to collect metrics from Docker. Requires kernel control groups to be enabled to collect memory usage data. If not done during initial installation, @@ -227,9 +227,9 @@ $ docker-compose restart telegraf ``` * `inputs.cpu_temp.conf' collects cpu temperature. -### Applying optional additions +### Applying optional additions -The *additions folder* (see [Significant directories and files](#significantFiles)) is a mechanism for additional *IOTstack-ready* configuration options to be provided for Telegraf. +The *additions folder* (see [Significant directories and files](#significant-directories-and-files)) is a mechanism for additional *IOTstack-ready* configuration options to be provided for Telegraf. Currently there is one addition: @@ -247,9 +247,9 @@ $ docker-compose restart telegraf The `grep` strips comment lines and the `sudo tee` is a safe way of appending the result to `telegraf.conf`. The `restart` causes Telegraf to notice the change. -## Getting a clean slate +## Getting a clean slate -### Erasing the persistent storage area +### Erasing the persistent storage area Erasing Telegraf's persistent storage area triggers self-healing and restores known defaults: @@ -270,7 +270,7 @@ Note: $ docker-compose restart telegraf ``` -### Resetting the InfluxDB database +### Resetting the InfluxDB database To reset the InfluxDB database that Telegraf writes into, proceed like this: @@ -291,7 +291,7 @@ In words: * Delete the `telegraf` database, and then exit the CLI. * Start the Telegraf container. This re-creates the database automatically. -## Upgrading Telegraf +## Upgrading Telegraf You can update most containers like this: @@ -333,7 +333,7 @@ Your existing Telegraf container continues to run while the rebuild proceeds. On The `prune` is the simplest way of cleaning up. The first call removes the old ***local image***. The second call cleans up the old ***base image***. -### Telegraf version pinning +### Telegraf version pinning If you need to pin Telegraf to a particular version: diff --git a/docs/Containers/WireGuard.md b/docs/Containers/WireGuard.md index 20b84d2f..49ebdc8d 100644 --- a/docs/Containers/WireGuard.md +++ b/docs/Containers/WireGuard.md @@ -11,27 +11,27 @@ Assumptions: * These instructions assume that you have privileges to configure your network's gateway (router). If you are not able to make changes to your network's firewall settings, then you will not be able to finish this setup. * In common with most VPN technologies, WireGuard assumes that the WAN side of your network's gateway has a public IP address which is reachable directly. WireGuard may not work if that assumption does not hold. If you strike this problem, you have to take it up with your ISP. -## Installing WireGuard under IOTstack +## Installing WireGuard under IOTstack You increase your chances of a trouble-free installation by performing the installation steps in the following order. -### Step 1: Update your Raspberry Pi OS +### Step 1: Update your Raspberry Pi OS -To be able to run WireGuard successfully, your Raspberry Pi needs to be **fully** up-to-date. If you want to understand why, see [the read only flag](#readOnlyFlag). +To be able to run WireGuard successfully, your Raspberry Pi needs to be **fully** up-to-date. If you want to understand why, see [the read only flag](#the-read-only-flag). ```bash $ sudo apt update $ sudo apt upgrade -y ``` -### Step 2: Set up a Dynamic DNS name +### Step 2: Set up a Dynamic DNS name Before you can use WireGuard (or any VPN solution), you need a mechanism for your remote clients to reach your home router. You have two choices: 1. Obtain a permanent IP address for your home router from your Internet Service Provider (ISP). Approach your ISP if you wish to pursue this option. It generally involves additional charges. 2. Use a Dynamic DNS service. See IOTstack documentation [Accessing your device from the internet](../Basic_setup/Accessing-your-Device-from-the-internet.md). The rest of this documentation assumes you have chosen this option. -### Step 3: Understand the Service Definition +### Step 3: Understand the Service Definition This is the service definition *template* that IOTstack uses for WireGuard: @@ -69,9 +69,9 @@ Key points: * Everything in the `environment:` section from `SERVERURL=` down to `PEERDNS=` (inclusive) affects WireGuard's generated configurations (the QR codes). In other words, any time you change any of those values, any existing QR codes will stop working. * WireGuard does not need to communicate directly with other Docker containers so there is no need for it to join `iotstack_nw`. -### Step 4: Decide what to configure +### Step 4: Decide what to configure -With most containers, you can continue to tweak environment variables and settings without upsetting the container's basic behaviour. WireGuard is a little different. You really need to think, carefully, about how you want to configure the service before you start. If you change your mind later, you generally have to [start from a clean slate](#cleanSlate). +With most containers, you can continue to tweak environment variables and settings without upsetting the container's basic behaviour. WireGuard is a little different. You really need to think, carefully, about how you want to configure the service before you start. If you change your mind later, you generally have to [start from a clean slate](#getting-a-clean-slate). #### Fields that you should always configure @@ -101,7 +101,7 @@ With most containers, you can continue to tweak environment variables and settin - Many examples on the web use "PEERS=n" where "n" is a number. In practice, that approach seems to be a little fragile and is not recommended for IOTstack. -#### Optional configuration - DNS resolution for peers +#### Optional configuration - DNS resolution for peers You have several options for how your remote peers resolve DNS requests: @@ -109,7 +109,7 @@ You have several options for how your remote peers resolve DNS requests: The default value of `auto` instructs the WireGuard *service* running within the WireGuard *container* to use the same DNS as the WireGuard *container* when resolving requests from connected peers. In practice, that means the *service* directs queries to 127.0.0.11, which Docker intercepts and forwards to whichever resolvers are specified in the Raspberry Pi's `/etc/resolv.conf`. -* `PEERDNS=auto` with `custom-cont-init` +* `PEERDNS=auto` with `custom-cont-init` This configuration instructs WireGuard to forward DNS queries from remote peers to any **container** which is listening on port 53. This is the option you will want to choose if you are running an ad-blocking DNS server (eg *PiHole* or *AdGuardHome*) in a container on the same host as WireGuard, and you want your remote clients to obtain DNS resolution via the ad-blocker. @@ -162,7 +162,7 @@ You have several options for how your remote peers resolve DNS requests: - PEERDNS=192.168.203.65 ``` -#### Optional configuration - WireGuard ports +#### Optional configuration - WireGuard ports The WireGuard service definition template follows the convention of using UDP port "51820" in three places. You can leave it like that and it will just work. There is no reason to change the defaults unless you want to. @@ -189,15 +189,15 @@ Rule #1: Rule #2: -* The *«public»* port forms part of the QR codes. If you decide to change the *«public»* port after you generate the QR codes, you will have to [start over from a clean slate](#cleanSlate). +* The *«public»* port forms part of the QR codes. If you decide to change the *«public»* port after you generate the QR codes, you will have to [start over from a clean slate](#getting-a-clean-slate). Rule #3: * Your router needs to know about both the *«public»* and *«external»* ports so, if you decide to change either of those, you must also reconfigure your router. -See [Understanding WireGuard's port numbers](#understandingPorts) if you want more information on how the various port numbers are used. +See [Understanding WireGuard's port numbers](#understanding-wireguards-port-numbers) if you want more information on how the various port numbers are used. -### Step 5: Configure WireGuard +### Step 5: Configure WireGuard There are two approaches: @@ -206,7 +206,7 @@ There are two approaches: Of the two, the first is generally the simpler and means you don't have to re-run the menu whenever you want to change WireGuard's configuration. -#### Method 1: Configure WireGuard by editing `docker-compose.yml` +#### Method 1: Configure WireGuard by editing `docker-compose.yml` 1. Run the menu: @@ -221,10 +221,10 @@ Of the two, the first is generally the simpler and means you don't have to re-ru 5. Choose Exit. 6. Open `docker-compose.yml` in your favourite text editor. 7. Navigate to the WireGuard service definition. -8. Implement the decisions you took in [decide what to configure](#configureWhat). +8. Implement the decisions you took in [decide what to configure](#step-4-decide-what-to-configure). 9. Save your work. -#### Method 2: Configure WireGuard using `compose-override.yml` +#### Method 2: Configure WireGuard using `compose-override.yml` The [Custom services and overriding default settings for IOTstack](../Basic_setup/Custom.md) page describes how to use an override file to allow the menu to incorporate your custom configurations into the final `docker-compose.yml` file. @@ -236,7 +236,7 @@ You will need to create the `compose-override.yml` **before** running the menu t ~/IOTstack/compose-override.yml ``` -2. Define overrides to implement the decisions you took in [Decide what to configure](#configureWhat). For example: +2. Define overrides to implement the decisions you took in [Decide what to configure](#step-4-decide-what-to-configure). For example: ```yml services: @@ -277,7 +277,7 @@ You will need to create the `compose-override.yml` **before** running the menu t and verify that the `wireguard` service definition is as you expect. -### Step 6: Start WireGuard +### Step 6: Start WireGuard 1. To start WireGuard, bring up your stack: @@ -298,7 +298,7 @@ You will need to create the `compose-override.yml` **before** running the menu t $ docker logs wireguard ``` - See also discussion of [the read-only flag](#readOnlyFlag). + See also discussion of [the read-only flag](#the-read-only-flag). 3. Confirm that WireGuard has generated the expected configurations. For example, given the following setting in `docker-compose.yml`: @@ -341,7 +341,7 @@ You will need to create the `compose-override.yml` **before** running the menu t Notice how each element in the `PEERS=` list is represented by a sub-directory prefixed with `peer_`. You should expect the same pattern for your peers. -### Step 7: Save your WireGuard client configuration files (QR codes) +### Step 7: Save your WireGuard client configuration files (QR codes) The first time you launch WireGuard, it generates cryptographically protected configurations for your remote clients and encapsulates those configurations in QR codes. You can see the QR codes by running: @@ -382,7 +382,7 @@ In this case: Keep in mind that each QR code contains everything needed for **any** device to access your home network via WireGuard. Treat your `.png` files as "sensitive documents". -### Step 8: Configure your router with a NAT rule +### Step 8: Configure your router with a NAT rule A typical home network will have a firewall that effectively blocks all incoming attempts from the Internet to open a new connection with a device on your network. @@ -427,7 +427,7 @@ A typical configuration process goes something like this: * *Public Port* or *External Port* needs to be the value you chose for «public» in the WireGuard service definition (51820 if you didn't change it). * *Service Name* (or *Service Type*) is typically a text field, an editable menu (where you can either make a choice or type your own value), or a button approximating an editable menu. If you are given the option of choosing "WireGuard", do that, otherwise just type that name into the field. It has no significance other than reminding you what the rule is for. -### Step 9: Configure your remote WireGuard clients +### Step 9: Configure your remote WireGuard clients This is a massive topic and one which is well beyond the scope of this guide. You really will have to work it out for yourself. Start by Googling: @@ -443,7 +443,7 @@ For portable devices (eg iOS and Android) it usually boils down to: 4. Point the device's camera at the QR code. 5. Follow your nose. -## Understanding WireGuard's port numbers +## Understanding WireGuard's port numbers Here's a concrete example configuration using three different port numbers: @@ -466,7 +466,7 @@ You also need to make a few assumptions: 1. The host running the remote WireGuard client (eg a mobile phone with the WireGuard app installed) has been allocated the IP address 55.66.77.88 when it connected to the Internet over 3G/4G/5G. 2. When the remote WireGuard client initiated the session, it chose UDP port 44524 as its source port. The actual number chosen is (essentially) random and only significant to the client. 3. Your Internet Service Provider allocated the IP address 12.13.14.15 to the WAN side of your router. -4. You have done all the steps in [Set up a Dynamic DNS name](#obtainDDNS) and your WAN IP address (12.13.14.15) is being propagated to your Dynamic DNS service provider. +4. You have done all the steps in [Set up a Dynamic DNS name](#step-2-set-up-a-dynamic-dns-name) and your WAN IP address (12.13.14.15) is being propagated to your Dynamic DNS service provider. Here's a reference model to help explain what occurs: @@ -494,9 +494,9 @@ Even if you use port 51820 everywhere (the default), all this Network Address Tr This model is a slight simplification because the remote client may also be also operating behind a router performing Network Address Translation. It is just easier to understand the basic concepts if you assume the remote client has a publicly-routable IP address. -## Debugging techniques +## Debugging techniques -### Monitor WireGuard traffic between your router and your Raspberry Pi +### Monitor WireGuard traffic between your router and your Raspberry Pi If `tcpdump` is not installed on your Raspberry Pi, you can install it by: @@ -512,7 +512,7 @@ $ sudo tcpdump -i eth0 -n udp port «external» Press ctrlc to terminate the capture. -### Monitor WireGuard traffic between your Raspberry Pi and the WireGuard container +### Monitor WireGuard traffic between your Raspberry Pi and the WireGuard container First, you need to add `tcpdump` to the container. You only need to do this once per debugging session. The package will remain in place until the next time you re-create the container. @@ -528,7 +528,7 @@ $ docker exec wireguard tcpdump -i eth0 -n udp port «internal» Press ctrlc to terminate the capture. -### Is Docker listening on the Raspberry Pi's «external» port? +### Is Docker listening on the Raspberry Pi's «external» port? ```bash $ PORT=«external»; sudo nmap -sU -p $PORT 127.0.0.1 | grep "$PORT/udp" @@ -541,7 +541,7 @@ There will be a short delay. The expected answer is either: Success implies that the container is also listening. -### Is your router listening on the «public» port? +### Is your router listening on the «public» port? ```bash $ PORT=«public»; sudo nmap -sU -p $PORT downunda.duckdns.org | grep "$PORT/udp" @@ -552,7 +552,7 @@ There will be a short delay. The expected answer is either: * `«public»/udp open|filtered unknown` = router is listening * `«public»/udp closed unknown` = router is not listening -## The read-only flag +## The read-only flag The `:ro` at the end of the following line in WireGuard's service definition means "read only": @@ -568,7 +568,7 @@ Writing into `/lib/modules` is not needed on a Raspberry Pi, providing that Rasp If WireGuard refuses to install and you have good reason to suspect that WireGuard may be trying to write to `/lib/modules` then you can *consider* removing the `:ro` flag and re-trying. Just be aware that WireGuard will likely be modifying your operating system. -## Updating WireGuard +## Updating WireGuard To update the WireGuard container: @@ -584,7 +584,7 @@ $ docker-compose up -d wireguard $ docker system prune ``` -## Getting a clean slate +## Getting a clean slate If WireGuard misbehaves, you can start over from a clean slate. You *may* also need to do this if you change any of the following environment variables: diff --git a/mkdocs.yml b/mkdocs.yml index 24017ed5..ac813ca0 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -52,3 +52,5 @@ plugins: markdown_extensions: - admonition - pymdownx.superfences + - toc: + permalink: true From d38a122cb4fc4008c7f23e62c9d01033905c6f8f Mon Sep 17 00:00:00 2001 From: ukkopahis <95980324+ukkopahis@users.noreply.github.com> Date: Wed, 26 Jan 2022 00:31:34 +0200 Subject: [PATCH 11/24] docs: keep top tabs always visible and hide footer * per default, navigation tabs disappear when you scroll down the page, which may confuse some users * hide "Made with Material" footer which is visible when scrolling to the bottom, which takes up space and may make the left menu scrollbar to appear, and thus reduce usability. --- docs/style.css | 4 ++++ mkdocs.yml | 4 ++++ 2 files changed, 8 insertions(+) create mode 100644 docs/style.css diff --git a/docs/style.css b/docs/style.css new file mode 100644 index 00000000..4ba1f2c7 --- /dev/null +++ b/docs/style.css @@ -0,0 +1,4 @@ +/* hide "Made with Material" footer */ +.md-footer-meta { + display: none; +} diff --git a/mkdocs.yml b/mkdocs.yml index ac813ca0..63513a1b 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -21,6 +21,7 @@ theme: features: - tabs - navigation.tabs + - navigation.tabs.sticky - navigation.sections plugins: @@ -49,6 +50,9 @@ plugins: Menu-System.md: Developers/Menu-System.md Contributing-Services.md: Developers/index.md +extra_css: + - style.css + markdown_extensions: - admonition - pymdownx.superfences From b05029c929725f829f4af7ab39614754cf359ba8 Mon Sep 17 00:00:00 2001 From: ukkopahis <95980324+ukkopahis@users.noreply.github.com> Date: Thu, 20 Jan 2022 03:23:30 +0200 Subject: [PATCH 12/24] homeassistant: add docs for https reverse proxy setup --- docs/Containers/Home-Assistant.md | 113 ++++++++++++++++++++++++++++++ 1 file changed, 113 insertions(+) diff --git a/docs/Containers/Home-Assistant.md b/docs/Containers/Home-Assistant.md index 585ce34e..aa03d63a 100644 --- a/docs/Containers/Home-Assistant.md +++ b/docs/Containers/Home-Assistant.md @@ -222,6 +222,119 @@ $ cd ~/IOTstack $ docker-compose up -d ``` +## HTTPS with a valid certificate + +Some HA integrations (e.g google assistant) require your HA API to be +accessible via https with a valid certificate. You can configure HA to do this: +[docs](https://www.home-assistant.io/docs/configuration/remote/) / +[guide](https://www.home-assistant.io/docs/ecosystem/certificates/lets_encrypt/) +or use a reverse proxy container, as described below. + +The linuxserver Secure Web Access Gateway container +([swag](https://docs.linuxserver.io/general/swag)) ([Docker hub +docs](https://hub.docker.com/r/linuxserver/swag)) will automatically generate a +SSL-certificate, update the SSL certificate before it expires and act as a +reverse proxy. + +1. First test your HA is working correctly: `http://raspberrypi.local:8123/` (assuming +your RPi hostname is raspberrypi) +2. Make sure you have duckdns working. +3. On your internet router, forward public port 443 to the RPi port 443 +4. Add swag to ~/IOTstack/docker-compose.yml beneath the `services:`-line: +``` + swag: + image: ghcr.io/linuxserver/swag + cap_add: + - NET_ADMIN + environment: + - PUID=1000 + - PGID=1000 + - TZ=Etc/UTC + - URL=.duckdns.org + - SUBDOMAINS=wildcard + - VALIDATION=duckdns + - DUCKDNSTOKEN= + - CERTPROVIDER=zerossl + - EMAIL= # required when using zerossl + volumes: + - ./volumes/swag/config:/config + ports: + - 443:443 + restart: unless-stopped +``` + Replace the bracketed values. Do NOT use any "-characters to enclose the values. + +5. Start the swag container, this creates the file to be edited in the next step: + ``` + cd ~/IOTstack && docker-compose up -d + ``` + + Check it starts up OK: `docker-compose logs -f swag`. It will take a minute or two before it finally logs "Server ready". + +6. Enable reverse proxy for `raspberrypi.local`. `homassistant.*` is already by default. and fix homeassistant container name ("upstream_app"): + ``` + sed -e 's/server_name/server_name *.local/' \ + volumes/swag/config/nginx/proxy-confs/homeassistant.subdomain.conf.sample \ + > volumes/swag/config/nginx/proxy-confs/homeassistant.subdomain.conf + ``` + +7. Forward to correct IP when target is a container running in "network_mode: + host" (like Home Assistant does): + ``` + cat << 'EOF' | sudo tee volumes/swag/config/custom-cont-init.d/add-host.docker.internal.sh + #!/bin/sh + DOCKER_GW=$(ip route | awk 'NR==1 {print $3}') + + sed -i -e "s/upstream_app .*/upstream_app ${DOCKER_GW};/" \ + /config/nginx/proxy-confs/homeassistant.subdomain.conf + EOF + sudo chmod u+x volumes/swag/config/custom-cont-init.d/add-host.docker.internal.sh + ``` + (This needs to be copy-pasted/entered as-is, ignore any "> "-prefixes printed + by bash) + +8. (optional) Add reverse proxy password protection if you don't want to rely + on the HA login for security, doesn't affect API-access: + ``` + sed -i -e 's/#auth_basic/auth_basic/' \ + volumes/swag/config/nginx/proxy-confs/homeassistant.subdomain.conf + docker-compose exec swag htpasswd -c /config/nginx/.htpasswd anyusername + ``` +9. Add `use_x_forwarded_for` and `trusted_proxies` to your homeassistant [http + config](https://www.home-assistant.io/integrations/http). The configuration + file is at `volumes/home_assistant/configuration.yaml` For a default install + the resulting http-section should be: + ``` + http: + use_x_forwarded_for: true + trusted_proxies: + - 192.168.0.0/16 + - 172.16.0.0/12 + - 10.77.0.0/16 + ``` +10. Refresh the stack: `cd ~/IOTstack && docker-compose stop && docker-compose + up -d` (again may take 1-3 minutes for swag to start if it recreates + certificates) +11. Test homeassistant is still working correctly: + `http://raspberrypi.local:8123/` (assuming your RPi hostname is + raspberrypi) +12. Test the reverse proxy https is working correctly: + `https://raspberrypi.local/` (browser will issue a warning about wrong + certificate domain, as the certificate is issued for you duckdns-domain, we + are just testing) + + Or from the command line in the RPi: + ``` + curl --resolve homeassistant..duckdns.org:443:127.0.0.1 \ + https://homeassistant..duckdns.org/ + ``` + (output should end in `if (!window.latestJS) { }`) + +13. And finally test your router forwards correctly by accessing it from + outside your LAN(e.g. using a mobile phone): + `https://homeassistant..duckdns.org/` Now the certificate + should work without any warnings. + ## Deactivating Hass.io Because Hass.io is independent of IOTstack, you can't deactivate it with any of the commands you normally use for IOTstack. From 118648d8fc949ee6979ddb695974d0acf58667f3 Mon Sep 17 00:00:00 2001 From: ukkopahis <95980324+ukkopahis@users.noreply.github.com> Date: Sat, 29 Jan 2022 10:39:54 +0200 Subject: [PATCH 13/24] docs: fix to reflect network change PR #477 uses default network, updating docs to match --- docs/Containers/Python.md | 8 -------- docs/Containers/WireGuard.md | 1 - 2 files changed, 9 deletions(-) diff --git a/docs/Containers/Python.md b/docs/Containers/Python.md index caa54606..7d5dc49c 100644 --- a/docs/Containers/Python.md +++ b/docs/Containers/Python.md @@ -40,14 +40,8 @@ When you select Python in the menu: # - "external:internal" volumes: - ./volumes/python/app:/usr/src/app - networks: - - iotstack_nw ``` - Note: - - * This service definition is for "new menu" (master branch). The only difference with "old menu" (old-menu branch) is the omission of the last two lines. - ### customising your Python service definition The service definition contains a number of customisation points: @@ -456,8 +450,6 @@ Proceed like this: # - "external:internal" # - "external:internal" volumes: volumes: - ./volumes/python/app:/usr/src/app | - ./volumes/wishbone/app:/usr/src/app - networks: networks: - - iotstack_nw - iotstack_nw ``` Note: diff --git a/docs/Containers/WireGuard.md b/docs/Containers/WireGuard.md index 49ebdc8d..98fcc610 100644 --- a/docs/Containers/WireGuard.md +++ b/docs/Containers/WireGuard.md @@ -67,7 +67,6 @@ Unfortunately, that service definition will not work "as is". It needs to be con Key points: * Everything in the `environment:` section from `SERVERURL=` down to `PEERDNS=` (inclusive) affects WireGuard's generated configurations (the QR codes). In other words, any time you change any of those values, any existing QR codes will stop working. -* WireGuard does not need to communicate directly with other Docker containers so there is no need for it to join `iotstack_nw`. ### Step 4: Decide what to configure From 0d9b9820d3576df07ca7181351db1d14559ca570 Mon Sep 17 00:00:00 2001 From: ukkopahis <95980324+ukkopahis@users.noreply.github.com> Date: Sat, 29 Jan 2022 11:46:38 +0200 Subject: [PATCH 14/24] Wireguard: better document how PEERDNS works with host resolv.conf Fix script comment to properly describe how th bridge gateway works. --- .templates/wireguard/use-container-dns.sh | 7 ++++--- docs/Containers/WireGuard.md | 22 ++++++++++++++++++++-- 2 files changed, 24 insertions(+), 5 deletions(-) diff --git a/.templates/wireguard/use-container-dns.sh b/.templates/wireguard/use-container-dns.sh index 18737322..7f13f219 100644 --- a/.templates/wireguard/use-container-dns.sh +++ b/.templates/wireguard/use-container-dns.sh @@ -1,8 +1,9 @@ # Forward DNS requests from remote WireGuard clients to the default # gateway on the internal bridged network that the WireGuard container -# is attached to. This results in queries being sent to any other -# container on the same internal bridged network that is listening -# on port 53 (eg PiHole, AdGuardHome or bind9). +# is attached to. The gateway routes queries out from the bridged network to +# the host's network. This results in queries being sent to any daemon or +# container that is listening on host port 53 (eg PiHole, AdGuardHome, dnsmasq +# or bind9). # # Acknowledgement: @ukkopahis diff --git a/docs/Containers/WireGuard.md b/docs/Containers/WireGuard.md index 98fcc610..ba73d6de 100644 --- a/docs/Containers/WireGuard.md +++ b/docs/Containers/WireGuard.md @@ -106,11 +106,26 @@ You have several options for how your remote peers resolve DNS requests: * `PEERDNS=auto` - The default value of `auto` instructs the WireGuard *service* running within the WireGuard *container* to use the same DNS as the WireGuard *container* when resolving requests from connected peers. In practice, that means the *service* directs queries to 127.0.0.11, which Docker intercepts and forwards to whichever resolvers are specified in the Raspberry Pi's `/etc/resolv.conf`. + DNS queries made on connected WireGuard clients should work as if they were + made on the host. If you [configure](Pi-hole.md#pi-hole-as-dns-server) your + ad-blocker into the host's `resolveconf.conf`, Wireguard clients will also + automatically use it. + + Details: The default value of `auto` instructs the WireGuard *service* + running within the WireGuard *container* to use a DNS-service, coredns, + also running in the Wireguard container. Coredns by default directs queries + to 127.0.0.11, which Docker intercepts and forwards to whichever resolvers + are specified in the Raspberry Pi's `/etc/resolv.conf`. * `PEERDNS=auto` with `custom-cont-init` - This configuration instructs WireGuard to forward DNS queries from remote peers to any **container** which is listening on port 53. This is the option you will want to choose if you are running an ad-blocking DNS server (eg *PiHole* or *AdGuardHome*) in a container on the same host as WireGuard, and you want your remote clients to obtain DNS resolution via the ad-blocker. + This configuration instructs WireGuard to forward DNS queries from remote + peers to any host daemon or **container** which is listening on port 53. + This is the option you will want to choose if you are running an + ad-blocking DNS server (eg *PiHole* or *AdGuardHome*) in a container on the + same host as WireGuard, and you want your remote clients to obtain DNS + resolution via the ad-blocker, but don't want your Raspberry Pi host to use + it. > Acknowledgement: thanks to @ukkopahis for developing this option. @@ -160,6 +175,9 @@ You have several options for how your remote peers resolve DNS requests: ```yml - PEERDNS=192.168.203.65 ``` + Do note that changes to `PEERDNS` will not be updated to existing clients, + and as such you may want to use `PEERDNS=auto` unless you have a very + specific requirement. #### Optional configuration - WireGuard ports From 4f52cf0ca6abf7cefb09f85e160b157873d5c588 Mon Sep 17 00:00:00 2001 From: ukkopahis <95980324+ukkopahis@users.noreply.github.com> Date: Sun, 30 Jan 2022 02:18:02 +0200 Subject: [PATCH 15/24] docs: fix container menu order Alphabetical ordering is case sensitive, rename all files to start with a capital letter. --- docs/Containers/{deconz.md => Deconz.md} | 0 docs/Containers/{diyHue.md => DiyHue.md} | 0 docs/Containers/{openHAB.md => OpenHab.md} | 0 docs/Containers/{x2go.md => X2go.md} | 0 4 files changed, 0 insertions(+), 0 deletions(-) rename docs/Containers/{deconz.md => Deconz.md} (100%) rename docs/Containers/{diyHue.md => DiyHue.md} (100%) rename docs/Containers/{openHAB.md => OpenHab.md} (100%) rename docs/Containers/{x2go.md => X2go.md} (100%) diff --git a/docs/Containers/deconz.md b/docs/Containers/Deconz.md similarity index 100% rename from docs/Containers/deconz.md rename to docs/Containers/Deconz.md diff --git a/docs/Containers/diyHue.md b/docs/Containers/DiyHue.md similarity index 100% rename from docs/Containers/diyHue.md rename to docs/Containers/DiyHue.md diff --git a/docs/Containers/openHAB.md b/docs/Containers/OpenHab.md similarity index 100% rename from docs/Containers/openHAB.md rename to docs/Containers/OpenHab.md diff --git a/docs/Containers/x2go.md b/docs/Containers/X2go.md similarity index 100% rename from docs/Containers/x2go.md rename to docs/Containers/X2go.md From c614c20fbc4d8df4ee5a53b4d02dec5c838abb23 Mon Sep 17 00:00:00 2001 From: ukkopahis <95980324+ukkopahis@users.noreply.github.com> Date: Thu, 3 Feb 2022 01:15:28 +0200 Subject: [PATCH 16/24] influxdb: document basic usage --- docs/Containers/InfluxDB.md | 97 ++++++++++++++++++++++++++++++++++++- 1 file changed, 96 insertions(+), 1 deletion(-) diff --git a/docs/Containers/InfluxDB.md b/docs/Containers/InfluxDB.md index 4f714771..81d1d5bc 100644 --- a/docs/Containers/InfluxDB.md +++ b/docs/Containers/InfluxDB.md @@ -1,9 +1,104 @@ # InfluxDB +A time series database. + +InfluxDB has configurable aggregation and retention policies allowing +measurement resolution reduction, storing all added data points for recent data +and only aggregated values for older data. + +To connect use: + +| Field | Default | +| --------- | ---------- | +| User | nodered | +| Password | nodered | +| URL (from other services) | http://influxdb:8086 | +| URL (on the host machine) | http://localhost:8086 | + ## References - [Docker](https://hub.docker.com/_/influxdb) - [Website](https://www.influxdata.com/) -## Security +## Setup + +To access the influx console, show current databases and database measurements: +``` +pi@raspberrypi:~/IOTstack $ docker-compose exec influxdb bash +root@6bca535a945f:/# influx +Connected to http://localhost:8086 version 1.8.10 +InfluxDB shell version: 1.8.10 +> show databases +name: databases +name +---- +_internal +telegraf +> use telegraf +Using database telegraf +> show measurements +name: measurements +name +---- +cpu +cpu_temperature +disk +diskio +etc... +``` + +To create a new database and set a limited retention policy, here for instance +any data older than 52 weeks is deleted: + +``` +> create database mydb +> show retention policies on mydb +name duration shardGroupDuration replicaN default +---- -------- ------------------ -------- ------- +autogen 0s 168h0m0s 1 true +> alter retention policy "autogen" on "mydb" duration 52w shard duration 1w replication 1 default +> show retention policies on mydb +name duration shardGroupDuration replicaN default +---- -------- ------------------ -------- ------- +autogen 8736h0m0s 168h0m0s 1 true + +``` + +Aggregation, on the other hand, is where you keep your relevant statistics, but +decrease their time-resolution and lose individual data-points. This is a much +more complicated topic and harder to configure. As such it is well outside the +scope of this guide. + + +## Reducing flash wear-out + +SSD-drives have pretty good controllers spreading out writes, so this isn't a +this isn't really a concern for them. But if you store data on a SD-card, +flash wear may cause the card to fail prematurely. Flash memory has a limited +number of erase-write cycles per physical block. These blocks may be multiple +megabytes. You can use `sudo lsblk -D` to see how big the erase granularity is +on your card. The goal is to avoid writing lots of small changes targeting the +same physical blocks. Here are some tips to mitigate SD-card wear: + +* Don't use short retention policies. This may mask heavy disk IO without + increasing disk space usage. Depending on the file system used, new data may + be written to the same flash blocks that were freed by expiration, wearing + them out. +* Take care not to add measurements too often. If possible no more often than + once a minute. Add all measurements in one operation. +* Adding measurements directly to Influxdb will cause a write on every + operation. If your client code can't aggregate multiple measurements into one + write, consider routing them via Telegraf. It has the + `flush_interval`-option, which will combine the measurements into one write. +* All InfluxDB queries are logged by default and logs are written to the + SD-card. To disable this, add to docker-compose.yml, next to the other + INFLUXDB_\* entries: + ``` + - INFLUXDB_DATA_QUERY_LOG_ENABLED=false + - INFLUXDB_HTTP_LOG_ENABLED=false + ``` + This is especially important if you plan on having Grafana or Chronograf + displaying up-to-date data on a dashboard. + +## Old-menu branch The credentials and default database name for influxdb are stored in the file called influxdb/influx.env . The default username and password is set to "nodered" for both. It is HIGHLY recommended that you change them. The environment file contains several commented out options allowing you to set several access options such as default admin user credentials as well as the default database name. Any change to the environment file will require a restart of the service. To access the terminal for influxdb execute `./services/influxdb/terminal.sh`. Here you can set additional parameters or create other databases. From a15ae1fa9bf273dc3226c4e0ba78bc6e50b23d4b Mon Sep 17 00:00:00 2001 From: Phill Kelley <34226495+Paraphraser@users.noreply.github.com> Date: Fri, 18 Feb 2022 20:26:24 +1100 Subject: [PATCH 17/24] Pi-hole: improve docs includes some changes from #501 Fixes #500 Co-authored-by: ukkopahis <95980324+ukkopahis@users.noreply.github.com> --- docs/Containers/Pi-hole.md | 107 ++++++++++++++++++++++++++++--------- 1 file changed, 82 insertions(+), 25 deletions(-) diff --git a/docs/Containers/Pi-hole.md b/docs/Containers/Pi-hole.md index 038c7b84..1035aa93 100644 --- a/docs/Containers/Pi-hole.md +++ b/docs/Containers/Pi-hole.md @@ -1,39 +1,82 @@ # Pi-hole Pi-hole is a fantastic utility to reduce ads. -The interface can be found on `http://"your_ip":8089/admin` +The web interface can be found at `http://«your_ip»:8089/admin` +where «your_ip» can be: -Default password is `IOtSt4ckP1Hol3`. This can be changed with: +* The IP address of the Raspberry Pi running Pi-hole. +* The domain name of the Raspberry Pi running Pi-hole. +* The multicast DNS name (eg "raspberrypi.local") of the Raspberry Pi running + Pi-hole. + +Default password is random, it can be changed by running: ``` -docker exec pihole pihole -a -p myNewPassword +docker-compose exec pihole pihole -a -p myNewPassword ``` -## Pi-hole as DNS server +References: + +* [Pi-hole on GitHub](https://github.com/pi-hole/docker-pi-hole) +* [Pi-hole on Dockerhub](https://hub.docker.com/r/pihole/pihole) + +## Environment variables + +Environment variables govern much of Pi-hole's behaviour. If you are running new +menu (master branch), the variables are inline in `docker-compose.yml`. If you +are running old menu, the variables will be in: +`~/IOTstack/services/pihole/pihole.env` + +The first time Pi-hole is launched, it checks for the `WEBPASSWORD` environment +variable. If found, sets the initial password. + +Pi-hole supports a [long list of environment +variables](https://github.com/pi-hole/docker-pi-hole#environment-variables). + +## Using Pi-hole as your DNS resolver + +In order for the Pi-hole to ad-block or resolve anything, it needs to be +defined as the DNS server. This can either be done manually to each device or +you can define it as a DNS-nameserver for the whole LAN. -In order for the Pi-hole to work, it needs to be defined as the DNS server. -This can either be done manually to each device or you can define it as a -DNS-nameserver for the whole LAN. +Note that using Pi-hole for clients on your network pretty much **requires** the +Raspberry Pi running Pi-hole to have a fixed IP address. -Assuming your pihole hostname is `raspberrypi` and has the IP `192.168.1.10`: +Assuming your RPi hostname is `raspberrypi` and has the static IP +`192.168.1.10`: -1. Go to the Pi-hole web interface: `http://raspberrypi.local:8089/admin` and Login - * From Left menu: Select Local DNS -> DNS Records - * Enter Domain: `raspberrypi.home.arpa` and IP Address: `192.168.1.10`. Press Add. -2. Go to your DHCP server, usually this is your Wireless Access Point / WLAN Router web interface. +1. Go to your network's DHCP server, usually this is your Wireless Access Point + / WLAN Router. + * Login into its web-interface * Find where DNS servers are defined - * Change all DNS fields to `192.168.1.10`. -3. All local machines have to be rebooted or have their DHCP leases released. Without this they will continue to use the old DNS setting from an old DHCP lease for quite some time. + * Change all DNS fields to `192.168.1.10` +2. All local machines have to be rebooted. Without this they will continue to + use the old DNS setting from an old DHCP lease for quite some time. -Now you can use `raspberrypi.home.arpa` as the domain name for the Raspberry Pi in your whole local network. +## Adding domain names -For the Raspberry Pi itself to also use the Pi-hole DNS server, run: +Login to the Pi-hole web interface: `http://raspberrypi.local:8089/admin`: + +1. Select from Left menu: Local DNS -> DNS Records +2. Enter Domain: `raspberrypi.home.arpa` and IP Address: `192.168.1.10`. Press + Add. + +Now you can use `raspberrypi.home.arpa` as the domain name for the Raspberry Pi +in your whole local network. You can also add domain names for your other +devices, provided they too have static IPs. + +The Raspberry Pi itself must also use be configured to use the Pi-hole DNS +server. This is especially important when you add your own domains names, +otherwise DNS may work differently on the Pi than on other devices. Configure +this by running: ```bash echo "name_servers=127.0.0.1" | sudo tee -a /etc/resolvconf.conf echo "name_servers_append=8.8.8.8" | sudo tee -a /etc/resolvconf.conf echo "resolv_conf_local_only=NO" | sudo tee -a /etc/resolvconf.conf sudo resolvconf -u # Ignore "Too few arguments."-complaint ``` -Quick explanation: resolv_conf_local_only is disabled and a public nameserver is added, so that in case the Pi-hole container is stopped, the Raspberry won't lose DNS functionality. It will just fallback to 8.8.8.8. +Quick explanation: resolv_conf_local_only is disabled and a public nameserver +is added, so that in case the Pi-hole container is stopped, the Raspberry won't +lose DNS functionality. It will just fallback to 8.8.8.8. ### Testing & Troubleshooting @@ -42,19 +85,28 @@ Install dig: apt install dnsutils ``` -Test that pi-hole is correctly configured (should respond 192.168.1.10): +Test that Pi-hole is correctly configured (should respond 192.168.1.10): ``` dig raspberrypi.home.arpa @192.168.1.10 ``` -To test on your desktop if your network configuration is correct, and an ESP will resolve its DNS queries correctly, restart your desktop machine to ensure DNS changes are updated and then use: +To test on your desktop if your network configuration is correct, and an ESP +will resolve its DNS queries correctly, restart your desktop machine to ensure +DNS changes are updated and then use: ``` dig raspberrypi.home.arpa ``` This should produce the same result as the previous command. -If this fails to resolve the IP, check that the server in the response is `192.168.1.10`. -If it's `127.0.0.xx` check `/etc/resolv.conf` begins with `nameserver 192.168.1.10`. +If this fails to resolve the IP, check that the server in the response is +`192.168.1.10`. If it's `127.0.0.xx` check `/etc/resolv.conf` begins with +`nameserver 192.168.1.10`. + +## Why .home.arpa? + +Instead of `.home.arpa` - which is the real standard, but a mouthful - you may +use `.internal`. Using `.local` would technically also work, but it should be +reserved only for mDNS use. ## Microcontrollers @@ -62,9 +114,14 @@ If you want to avoid hardcoding your Raspberry Pi IP to your ESPhome devices, you need a DNS server that will do the resolving. This can be done using the Pi-hole container as described above. -## Why .home.arpa? +!!! info "`*.local` won't work for ESPhome" -Instead of `.home.arpa` - which is the real standard, but a mouthful - you may use `.internal`. -Using `.local` would technically also work, but it should be reserved only for mDNS use. + There is a special case for resolving `*.local` addresses. If you do a + `ping raspberrypi.local` on your desktop linux or the RPI, it will first + try using mDNS/bonjour to resolve the IP address raspberrypi.local. If this + fails it will then ask the DNS server. Esphome devices can't use mDNS to + resolve an IP address. You need a proper DNS server to respond to queries + made by an ESP. As such, `dig raspberrypi.local` will fail, simulating + ESPhome device behavior. This is as intended, and you should use + raspberrypi.home.arpa as the address on your ESP-device. -Note: There is a special case for resolving `*.local` addresses. If you do a `ping raspberrypi.local` on your desktop linux or the RPI, it will first try using mDNS/bonjour to resolve the IP address raspberrypi.local. If this fails it will then ask the DNS server. Esphome devices can't use mDNS to resolve an IP address. You need a proper DNS server to respond to queries made by an ESP. As such, `dig raspberrypi.local` will fail, simulating ESPhome device behavior. This is as intended, and you should use raspberrypi.home.arpa as the address on your ESP-device. From 6e499db0c61ef416785bd0d477df5d509817c94d Mon Sep 17 00:00:00 2001 From: ukkopahis <95980324+ukkopahis@users.noreply.github.com> Date: Thu, 24 Feb 2022 19:42:39 +0200 Subject: [PATCH 18/24] Octoprint: change doc to use shorter menu title --- docs/Containers/Octoprint.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/Containers/Octoprint.md b/docs/Containers/Octoprint.md index ec932c78..ef076707 100644 --- a/docs/Containers/Octoprint.md +++ b/docs/Containers/Octoprint.md @@ -1,3 +1,6 @@ +--- +title: Octoprint +--- # OctoPrint – the snappy web interface for your 3D printer ## References From 40d17ececd6c65b7153b298a1a23abb00c3147f1 Mon Sep 17 00:00:00 2001 From: ukkopahis <95980324+ukkopahis@users.noreply.github.com> Date: Thu, 24 Feb 2022 02:50:41 +0200 Subject: [PATCH 19/24] docs: fix edit_uri --- mkdocs.yml | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/mkdocs.yml b/mkdocs.yml index 63513a1b..dc9d1306 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -1,11 +1,10 @@ site_name: IOTstack site_description: 'Docker stack for getting started on IOT on the Raspberry PI' - # Repository repo_url: https://github.com/SensorsIot/IOTstack repo_name: SensorsIot/IOTstack -edit_uri: "https://github.com/SensorsIot/IOTstack/edit/master/" +edit_uri: "https://github.com/SensorsIot/IOTstack/edit/master/docs" theme: name: material From 519aaee30c5dd476aa7b832c2f54a433d7441c16 Mon Sep 17 00:00:00 2001 From: ukkopahis <95980324+ukkopahis@users.noreply.github.com> Date: Thu, 24 Feb 2022 03:01:40 +0200 Subject: [PATCH 20/24] docs: define mkdocs dependencies in requirements-mkdocs.txt --- .github/workflows/main.yml | 4 +--- docs/Developers/index.md | 5 ++--- requirements-mkdocs.txt | 3 +++ 3 files changed, 6 insertions(+), 6 deletions(-) create mode 100644 requirements-mkdocs.txt diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 8eb3eb18..6a03007a 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -13,7 +13,5 @@ jobs: - uses: actions/setup-python@v2 with: python-version: 3.x - - run: pip install mkdocs-material - - run: pip install mkdocs-redirects - - run: pip install mkdocs-git-revision-date-localized-plugin + - run: pip3 install -r requirements-mkdocs.txt - run: mkdocs gh-deploy --force diff --git a/docs/Developers/index.md b/docs/Developers/index.md index 25b20202..25e5e2cb 100644 --- a/docs/Developers/index.md +++ b/docs/Developers/index.md @@ -4,10 +4,9 @@ Documentation is is written as markdown, processed using mkdocs ([docs](https://www.mkdocs.org/user-guide/writing-your-docs/#writing-your-docs)) and the Material theme ([docs](https://squidfunk.github.io/mkdocs-material/reference/)). The Material theme is not just styling, but provides additional syntax extensions. -Setup your system for Material: +Setup your system for mkdocs and Material: ``` -pip3 install mkdocs-material -pip3 install mkdocs-git-revision-date-localized-plugin +pip3 install -r requirements-mkdocs.txt ``` To test your local changes while writing them and before making a pull-request: diff --git a/requirements-mkdocs.txt b/requirements-mkdocs.txt new file mode 100644 index 00000000..cddd4398 --- /dev/null +++ b/requirements-mkdocs.txt @@ -0,0 +1,3 @@ +mkdocs-material +mkdocs-material-extensions +mkdocs-redirects From 179c6336f104430e93213c006b28a703c36bfffb Mon Sep 17 00:00:00 2001 From: ukkopahis <95980324+ukkopahis@users.noreply.github.com> Date: Thu, 24 Feb 2022 11:14:00 +0200 Subject: [PATCH 21/24] docs: add "stack" logo and favicon source: https://primer.style/octicons/stack-24 Licence: MIT License The MIT-licence requires for you to include the copyright notice and the permission notice when using substantial portions of the Software. This one svg-file is not a substantial portion of the project: https://github.com/primer/octicons As such the licence is only noted in full here: MIT License Copyright (c) 2022 GitHub Inc. Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. --- docs/stack-24.svg | 1 + mkdocs.yml | 3 +++ 2 files changed, 4 insertions(+) create mode 100644 docs/stack-24.svg diff --git a/docs/stack-24.svg b/docs/stack-24.svg new file mode 100644 index 00000000..05794aee --- /dev/null +++ b/docs/stack-24.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index dc9d1306..86ea5282 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -8,6 +8,9 @@ edit_uri: "https://github.com/SensorsIot/IOTstack/edit/master/docs" theme: name: material + icon: + logo: octicons/stack-24 + favicon: stack-24.svg palette: - scheme: default toggle: From fd0340c7d7c3d8eba5935698881b201429a14026 Mon Sep 17 00:00:00 2001 From: ukkopahis <95980324+ukkopahis@users.noreply.github.com> Date: Wed, 23 Feb 2022 22:36:57 +0200 Subject: [PATCH 22/24] docs: improve Wiki home page friendliness Add Discord link to Wiki home page. Only show relevant (wide/narrow) layout navigation advice. Hide Navigation and add project mission statement quote. --- docs/index.md | 24 ++++++++++++++++++------ docs/style.css | 13 +++++++++++++ 2 files changed, 31 insertions(+), 6 deletions(-) diff --git a/docs/index.md b/docs/index.md index f320b064..5eceef24 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,13 +1,25 @@ +--- +title: Home +hide: + - navigation +--- # IOTStack Wiki -Welcome to the IOTstack Wiki: - -* Use the list of contents at the left of this page to explore this Wiki. +!!! abstract inline end "What is IOTstack" + IOTstack is a builder for docker-compose to easily make and maintain IoT + stacks on the Raspberry Pi - - If you are viewing this on a device that does not show the list by default, click the "≡" icon. +Welcome to the IOTstack Wiki: -* If you are looking for information on a specific container, click on the "Containers" folder at the bottom of the list. +* + Use the header tabs and content list at the left to explore this Wiki. + + * If you are just getting started with IOTstack, see [Getting Started](Basic_setup/). -* To contribute see [Contributing](Developers/) +* You're always welcome to ask questions on the [IOTStack Discord](https://discord.gg/ZpKHnks). + +* Fixes and improvements welcome, see [Contributing](Developers/) diff --git a/docs/style.css b/docs/style.css index 4ba1f2c7..91073fc2 100644 --- a/docs/style.css +++ b/docs/style.css @@ -1,4 +1,17 @@ +/* vim: set sw=2: */ + /* hide "Made with Material" footer */ .md-footer-meta { display: none; } + +@media screen and (max-width:76.25em) { + .show-when-wide-layout { + display:none + } +} +@media screen and (min-width:76.25em) { + .show-when-narrow-layout { + display:none + } +} From 3f9bceac51106e95a1840a166a4668f847b3a7a0 Mon Sep 17 00:00:00 2001 From: ukkopahis <95980324+ukkopahis@users.noreply.github.com> Date: Thu, 24 Feb 2022 19:39:51 +0200 Subject: [PATCH 23/24] docs: move Updates/ from subfolder to top-level tab * better usability, especially on the narrow layout --- docs/Basic_setup/index.md | 2 +- docs/{Basic_setup => }/Updates/New-Menu-Release-Notes.md | 2 +- docs/{Basic_setup => }/Updates/gcgarner-migration.md | 0 .../Updates/Updating-the-Project.md => Updates/index.md} | 2 +- mkdocs.yml | 4 +--- 5 files changed, 4 insertions(+), 6 deletions(-) rename docs/{Basic_setup => }/Updates/New-Menu-Release-Notes.md (94%) rename docs/{Basic_setup => }/Updates/gcgarner-migration.md (100%) rename docs/{Basic_setup/Updates/Updating-the-Project.md => Updates/index.md} (98%) diff --git a/docs/Basic_setup/index.md b/docs/Basic_setup/index.md index dd7f7b00..12f6efb7 100644 --- a/docs/Basic_setup/index.md +++ b/docs/Basic_setup/index.md @@ -132,7 +132,7 @@ If you prefer to automate your installations using scripts, see: If you are still running on gcgarner/IOTstack and need to migrate to SensorsIot/IOTstack, see: -* [Migrating IOTstack from gcgarner to SensorsIot](Updates/gcgarner-migration.md). +* [Migrating IOTstack from gcgarner to SensorsIot](../Updates/gcgarner-migration.md). ## recommended system patches diff --git a/docs/Basic_setup/Updates/New-Menu-Release-Notes.md b/docs/Updates/New-Menu-Release-Notes.md similarity index 94% rename from docs/Basic_setup/Updates/New-Menu-Release-Notes.md rename to docs/Updates/New-Menu-Release-Notes.md index c5f894d9..ab324a63 100644 --- a/docs/Basic_setup/Updates/New-Menu-Release-Notes.md +++ b/docs/Updates/New-Menu-Release-Notes.md @@ -9,7 +9,7 @@ There are many features that are needing to be introduced into the new menu syst ## Breaking changes There are a few changes that you need to be aware of: -* Docker Environmental `*.env` files are no longer a thing by default. Everything needed is specified in the service.yml file, you can still optionally use them though either with [Custom Overrides](../Custom.md) or with the [PostBuild](../../Developers/PostBuild-Script.md) script. Specific config files for certain services still work as they once did. +* Docker Environmental `*.env` files are no longer a thing by default. Everything needed is specified in the service.yml file, you can still optionally use them though either with [Custom Overrides](../Basic_setup/Custom.md) or with the [PostBuild](../Developers/PostBuild-Script.md) script. Specific config files for certain services still work as they once did. * Python 3, pip3, PyYAML and Blessed are all required to be installed. * Not backwards compatible with old menu system. You will be able to switch back to the old menu system for a period of time by changing to the `old-menu` branch. It will be unmaintained except for critical updates. It will eventually be removed - but not before everyone is ready to leave it. diff --git a/docs/Basic_setup/Updates/gcgarner-migration.md b/docs/Updates/gcgarner-migration.md similarity index 100% rename from docs/Basic_setup/Updates/gcgarner-migration.md rename to docs/Updates/gcgarner-migration.md diff --git a/docs/Basic_setup/Updates/Updating-the-Project.md b/docs/Updates/index.md similarity index 98% rename from docs/Basic_setup/Updates/Updating-the-Project.md rename to docs/Updates/index.md index 93654c18..e659c046 100644 --- a/docs/Basic_setup/Updates/Updating-the-Project.md +++ b/docs/Updates/index.md @@ -11,7 +11,7 @@ Periodically updates are made to project which include new or modified container 5. rebuild localy created images from new Dockerfiles: `docker-compose build --pull --no-cache` 6. update running containers to latest: `docker-compose up --build -d` -*Troubleshooting:* if a container fails to restart after update +### Troubleshooting: if a container fails to start after update * try restarting the whole stack: `docker-compose restart` * backup your stack settings: `cp docker-compose.yml docker-compose.yml.bak` diff --git a/mkdocs.yml b/mkdocs.yml index 86ea5282..c908ecd6 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -43,9 +43,7 @@ plugins: Networking.md: Basic_setup/Networking.md RPIEasy_native.md: Basic_setup/RPIEasy_native.md Understanding-Containers.md: Basic_setup/Understanding-Containers.md - Updates/New-Menu-Release-Notes.md: Basic_setup/Updates/New-Menu-Release-Notes.md - Updates/Updating-the-Project.md: Basic_setup/Updates/Updating-the-Project.md - Updates/gcgarner-migration.md: Basic_setup/Updates/gcgarner-migration.md + Updates/Updating-the-Project.md: Updates/index.md PostBuild-Script.md: Developers/PostBuild-Script.md BuildStack-RandomPassword.md: Developers/BuildStack-RandomPassword.md BuildStack-Services.md: Developers/BuildStack-Services.md From 4d691837a35c475d549e15d9a0c2477eb7a23573 Mon Sep 17 00:00:00 2001 From: ukkopahis <95980324+ukkopahis@users.noreply.github.com> Date: Thu, 24 Feb 2022 23:36:17 +0200 Subject: [PATCH 24/24] docs: improve "Getting Started" * also works on the 64-bit Raspberry OS * state importance of the pathces in no uncertain terms * rephrase assumptions in a more concise way * extract small essay about sudo into its own "What is sudo?"-page * move "updating from gcgarner"-link to the Home page --- docs/Basic_setup/What-is-sudo.md | 47 +++++++++++++++++ docs/Basic_setup/index.md | 87 +++++--------------------------- docs/index.md | 1 + 3 files changed, 62 insertions(+), 73 deletions(-) create mode 100644 docs/Basic_setup/What-is-sudo.md diff --git a/docs/Basic_setup/What-is-sudo.md b/docs/Basic_setup/What-is-sudo.md new file mode 100644 index 00000000..9f1a50bf --- /dev/null +++ b/docs/Basic_setup/What-is-sudo.md @@ -0,0 +1,47 @@ +# What is sudo? + +Many first-time users of IOTstack get into difficulty by misusing the `sudo` command. The problem is best understood by example. In the following, you would expect `~` (tilde) to expand to `/home/pi`. It does: + +```bash +$ echo ~/IOTstack +/home/pi/IOTstack +``` + +The command below sends the same `echo` command to `bash` for execution. This is what happens when you type the name of a shell script. You get a new instance of `bash` to run the script: + +```bash +$ bash -c 'echo ~/IOTstack' +/home/pi/IOTstack +``` + +Same answer. Again, this is what you expect. But now try it with `sudo` on the front: + +```bash +$ sudo bash -c 'echo ~/IOTstack' +/root/IOTstack +``` + +Different answer. It is different because `sudo` means "become root, and then run the command". The process of becoming root changes the home directory, and that changes the definition of `~`. + +Any script designed for working with IOTstack assumes `~` (or the equivalent `$HOME` variable) expands to `/home/pi`. That assumption is invalidated if the script is run by `sudo`. + +Of necessity, any script designed for working with IOTstack will have to invoke `sudo` **inside** the script **when it is required**. You do not need to second-guess the script's designer. + +Please try to minimise your use of `sudo` when you are working with IOTstack. Here are some rules of thumb: + +1. Is what you are about to run a script? If yes, check whether the script already contains `sudo` commands. Using `menu.sh` as the example: + + ```bash + $ grep -c 'sudo' ~/IOTstack/menu.sh + 28 + ``` + + There are numerous uses of `sudo` within `menu.sh`. That means the designer thought about when `sudo` was needed. + +2. Did the command you **just executed** work without `sudo`? Note the emphasis on the past tense. If yes, then your work is done. If no, and the error suggests elevated privileges are necessary, then re-execute the last command like this: + + ```bash + $ sudo !! + ``` + +It takes time, patience and practice to learn when `sudo` is **actually** needed. Over-using `sudo` out of habit, or because you were following a bad example you found on the web, is a very good way to find that you have created so many problems for yourself that will need to reinstall your IOTstack. *Please* err on the side of caution! diff --git a/docs/Basic_setup/index.md b/docs/Basic_setup/index.md index 12f6efb7..75b589c0 100644 --- a/docs/Basic_setup/index.md +++ b/docs/Basic_setup/index.md @@ -10,27 +10,25 @@ Andreas Spiess Video #352: Raspberry Pi4 Home Automation Server (incl. Docker, O [![#352 Raspberry Pi4 Home Automation Server (incl. Docker, OpenHAB, HASSIO, NextCloud)](http://img.youtube.com/vi/KJRMjUzlHI8/0.jpg)](https://www.youtube.com/watch?v=KJRMjUzlHI8) -## assumptions +## Assumptions IOTstack makes the following assumptions: 1. Your hardware is a Raspberry Pi (typically a 3B+ or 4B). - Note: - * The Raspberry Pi Zero W2 has been tested with IOTstack. It works but the 512MB RAM means you should not try to run too many containers concurrently. + * Users have also [reported success + ](https://github.com/SensorsIot/IOTstack/issues/375) on Orange Pi + Win/Plus. -2. Your Raspberry Pi has a reasonably-recent version of 32-bit Raspberry Pi OS (aka "Raspbian") installed. You can download operating-system images: +2. Your Raspberry Pi has a reasonably-recent version of 32-bit or 64-bit Raspberry Pi OS (aka "Raspbian") installed. You can download operating-system images: * [Current release](https://www.raspberrypi.com/software/operating-systems/) + : "Raspberry Pi OS with desktop" is recommended. * [Prior releases](http://downloads.raspberrypi.org/raspios_armhf/images/) + : This offers only "Raspberry Pi OS with desktop" images. - Note: - - * If you use the first link, "Raspberry Pi OS with desktop" is recommended. - * The second link only offers "Raspberry Pi OS with desktop" images. - -3. Your operating system has been kept up-to-date with: +3. Your operating system has been updated: ```bash $ sudo apt update @@ -46,13 +44,7 @@ If the first three assumptions hold, assumptions four through six are Raspberry Please don't read these assumptions as saying that IOTstack will not run on other hardware, other operating systems, or as a different user. It is just that IOTstack gets most of its testing under these conditions. The further you get from these implicit assumptions, the more your mileage may vary. -### other platforms - -Users have reported success on other platforms, including: - -* [Orange Pi WinPlus](https://github.com/SensorsIot/IOTstack/issues/375) - -## new installation +## New installation ### automatic (recommended) @@ -128,13 +120,9 @@ If you prefer to automate your installations using scripts, see: * [Installing Docker for IOTstack](https://gist.github.com/Paraphraser/d119ae81f9e60a94e1209986d8c9e42f#scripting-iotstack-installations). -## migrating from the old repo (gcgarner)? - -If you are still running on gcgarner/IOTstack and need to migrate to SensorsIot/IOTstack, see: +## Required system patches -* [Migrating IOTstack from gcgarner to SensorsIot](../Updates/gcgarner-migration.md). - -## recommended system patches +Unless you know what you are doing, assume these are needed. ### patch 1 – restrict DHCP @@ -162,9 +150,9 @@ PRETTY_NAME="Raspbian GNU/Linux 10 (buster)" If you see the word "buster", proceed to step 2. Otherwise, skip this patch. -#### step 2: if you are running "buster" … +#### step 2: if you are indeed running "buster" -You need this patch if you are running Raspbian Buster. Without this patch, Docker images will fail if: +Without this patch on Buster, Docker images will fail if: * the image is based on Alpine and the image's maintainer updates to [Alpine 3.13](https://wiki.alpinelinux.org/wiki/Release_Notes_for_Alpine_3.13.0#time64_requirement); and/or * an image's maintainer updates to a library that depends on 64-bit values for *Unix epoch time* (the so-called Y2038 problem). @@ -189,53 +177,6 @@ Enable by running (takes effect after reboot): echo $(cat /boot/cmdline.txt) cgroup_memory=1 cgroup_enable=memory | sudo tee /boot/cmdline.txt ``` -## a word about the `sudo` command - -Many first-time users of IOTstack get into difficulty by misusing the `sudo` command. The problem is best understood by example. In the following, you would expect `~` (tilde) to expand to `/home/pi`. It does: - -```bash -$ echo ~/IOTstack -/home/pi/IOTstack -``` - -The command below sends the same `echo` command to `bash` for execution. This is what happens when you type the name of a shell script. You get a new instance of `bash` to run the script: - -```bash -$ bash -c 'echo ~/IOTstack' -/home/pi/IOTstack -``` - -Same answer. Again, this is what you expect. But now try it with `sudo` on the front: - -```bash -$ sudo bash -c 'echo ~/IOTstack' -/root/IOTstack -``` - -Different answer. It is different because `sudo` means "become root, and then run the command". The process of becoming root changes the home directory, and that changes the definition of `~`. - -Any script designed for working with IOTstack assumes `~` (or the equivalent `$HOME` variable) expands to `/home/pi`. That assumption is invalidated if the script is run by `sudo`. - -Of necessity, any script designed for working with IOTstack will have to invoke `sudo` **inside** the script **when it is required**. You do not need to second-guess the script's designer. - -Please try to minimise your use of `sudo` when you are working with IOTstack. Here are some rules of thumb: - -1. Is what you are about to run a script? If yes, check whether the script already contains `sudo` commands. Using `menu.sh` as the example: - - ```bash - $ grep -c 'sudo' ~/IOTstack/menu.sh - 28 - ``` - - There are numerous uses of `sudo` within `menu.sh`. That means the designer thought about when `sudo` was needed. - -2. Did the command you **just executed** work without `sudo`? Note the emphasis on the past tense. If yes, then your work is done. If no, and the error suggests elevated privileges are necessary, then re-execute the last command like this: - - ```bash - $ sudo !! - ``` - -It takes time, patience and practice to learn when `sudo` is **actually** needed. Over-using `sudo` out of habit, or because you were following a bad example you found on the web, is a very good way to find that you have created so many problems for yourself that will need to reinstall your IOTstack. *Please* err on the side of caution! ## the IOTstack menu @@ -343,7 +284,7 @@ $ git checkout master $ ./menu.sh ``` -### old menu (old-menu branch) +### old menu (old-menu branch) ```bash $ cd ~/IOTstack/ diff --git a/docs/index.md b/docs/index.md index 5eceef24..425df460 100644 --- a/docs/index.md +++ b/docs/index.md @@ -19,6 +19,7 @@ Welcome to the IOTstack Wiki: * If you are just getting started with IOTstack, see [Getting Started](Basic_setup/). + * If you're running gcgarner/IOTstack see [Migrating to SensorsIot](Updates/gcgarner-migration.md). * You're always welcome to ask questions on the [IOTStack Discord](https://discord.gg/ZpKHnks).