Merge branch 'armhf_keys_menu_lines' into pr/126

Author: Slyke

.github/workflows/main.yml

@@ -0,0 +1,18 @@
+name: Publish docs via GitHub Pages
+on:
+  push:
+    branches:
+      - master
+
+jobs:
+  build:
+    name: Deploy docs
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout master
+        uses: actions/checkout@v1
+
+      - name: Deploy docs
+        uses: mhausenblas/mkdocs-deploy-gh-pages@master
+        env:
+          PERSONAL_TOKEN: ${{ secrets.PERSONAL_TOKEN }}

README.md

@@ -1,198 +1,3 @@
-# IOTStack
+Documentation for the project: https://sensorsiot.github.io/IOTstack/
 
-IOTstack is a builder for docker-compose to easily make and maintain IoT stacks on the Raspberry Pi
-
-## Announcements
-
-The bulk of the README has moved to the Wiki. Please check it out [here](https://github.com/gcgarner/IOTstack/wiki)
-
-* 2019-12-19 Added python container, tweaked update script
-* 2019-12-12 modified zigbee2mqtt template file
-* 2019-12-12 Added Function to add custom containers to the stack
-* 2019-12-12 PR cmskedgell: Added Homebridge
-* 2019-12-12 PR 877dev: Added trimming of online backups
-* 2019-12-03 BUGFIX Mosquitto: Fixed issue where mosquitto failed to start as a result of 11-28 change
-* 2019-12-03 Added terminal for postgres, temporarily removed setfacl from menu
-* 2019-11-28 PR @stfnhmplr added diyHue
-* 2019-11-28 Fixed update notification on menu
-* 2019-11-28 Fixed mosquitto logs and database not mapping correctly to volumes. Pull new template to fix
-* 2019-11-28 added the option to disable swapfile by setting swappiness to 0
-* 2019-11-28 PR @stfnhmplr fixed incorrect shegang on MariaDB terminal.sh
-* 2019-11-28 Added native install for RPIEasy
-* 2019-11-27 Additions: NextCloud, MariaDB, MotionEye, Mozilla Webthings, blynk-server (fixed issue with selection.txt)
-* 2019-11-22 BUGFIX selection.txt failed on fresh install, added pushd IOTstack to menu to ensure correct path
-* 2019-11-22 Added notification into menu if project update is available
-* 2019-11-20 BUGFIX influxdb backup: Placing docker_backup in crontab caused influxdb backup not to execute correctly
-* 2019-11-20 BUGFIX disable swap: swapfile recreation on reboot fixed. Re-run from menu to fix.
-* Node-RED: serial port. New template adds privileged which allows acces to serial devices
-* EspurinoHub: is available for testing see wiki entry
-
-***
-
-## Highlighted topics
-
-* [Bluetooth and Node-RED](https://github.com/gcgarner/IOTstack/wiki/Node-RED#using-bluetooth)
-* [Saving files to disk inside containers](https://github.com/gcgarner/IOTstack/wiki/Node-RED#sharing-files-between-node-red-and-the-host)
-* [Updating the Project](https://github.com/gcgarner/IOTstack/wiki/Updating-the-Project)
-
- ***
-
-## Coming soon
-
-* reverse proxy is now next on the list, I cant keep up with the ports
-* Detection of arhcitecture for seperate stack options for amd64, armhf, i386
-* autocleanup of backups on cloud
-* Gitea (in testing branch)
-* OwnCloud
-
-***
-
-## About
-
-Docker stack for getting started on IoT on the Raspberry Pi.
-
-This Docker stack consists of:
-
-* Node-RED
-* Grafana
-* InfluxDB
-* Postgres
-* Mosquitto mqtt
-* Portainer
-* Adminer
-* openHAB
-* Home Assistant (HASSIO)
-* zigbee2mqtt
-* Pi-Hole
-* TasmoAdmin (parial wiki)
-* Plex media server
-* Telegraf (wiki coming soon)
-* RTL_433
-* EspruinoHub (testing)
-* MotionEye
-* MariaDB
-* Plex
-* Homebridge
-
-In addition, there is a write-up and some scripts to get a dynamic DNS via duckdns and VPN up and running.
-
-Firstly what is docker? The correct question is "what are containers?". Docker is just one of the utilities to run a container.
-
-A Container can be thought of as ultra-minimal virtual machines, they are a collection of binaries that run in a sandbox environment. You download a preconfigured base image and create a new container. Only the differences between the base and your "VM" are stored.
-Containers don't have [GUI](https://en.wikipedia.org/wiki/Graphical_user_interface)s so generally the way you interact with them is via web services or you can launch into a terminal.
-One of the major advantages is that the image comes mostly preconfigured.  
-
-There are pro's and cons for using native installs vs containers. For me, one of the best parts of containers is that it doesn't "clutter" your device. If you don't need Postgres anymore then just stop and delete the container. It will be like the container was never there.
-
-The container will fail if you try to run the docker and native vesions as the same time. It is best to install this on a fresh system.
-
-For those looking for a script that installs native applications check out [Peter Scargill's script](https://tech.scargill.net/the-script/)
-  
-## Tested platform
-
-Raspberry Pi 3B and 4B Raspbian (Buster)
-
-### Older Pi's
-
-Docker will not run on a PiZero or A model 1 because of the CPU. It has not been tested on a Model 2. You can still use Peter Scargill's [script](https://tech.scargill.net/the-script/)
-
-## Running under a virtual machine
-
-For those wanting to test out the script in a Virtual Machine before installing on their Pi there are some limitations. The script is designed to work with Debian based distributions. Not all the container have x86_64 images. For example Portainer does not and will give an error when you try and start the stack. Please see the pinned issue [#29](https://github.com/gcgarner/IOTstack/issues/29), there is more info there.
-
-## Feature Requests
-
-Please direct all feature requests to [Discord](https://discord.gg/W45tD83)
-  
-## Youtube reference
-
-This repo was originally inspired by Andreas Spiess's video on using some of these tools. Some containers have been added to extend its functionality.
-
-[YouTube video](https://www.youtube.com/watch?v=JdV4x925au0): This is an alternative approach to the setup. Be sure to watch the video for the instructions. Just note that the network addresses are different, see the wiki under Docker Networks.
-
-### YouTube guide
-
-@peyanski (Kiril) made a YouTube video on getting started using the project, check it out [here](https://youtu.be/5JMNHuHv134)
-
-## Download the project
-
-1.On the lite image you will need to install git first
-
-```bash
-sudo apt-get install git
-```
-
-2.Download the repository with:
-
-```bash
-git clone https://github.com/gcgarner/IOTstack.git ~/IOTstack
-```
-
-Due to some script restraints, this project needs to be stored in ~/IOTstack
-
-3.To enter the directory run:
-
-```bash
-cd ~/IOTstack
-```
-
-## The Menu
-
-I've added a menu to make things easier. It is good to familiarise yourself with the installation process.
-The menu can be used to install docker and build the docker-compose.yml file necessary for starting the stack. It also runs a few common commands. I do recommend you start to learn the docker and docker-compose commands if you plan on using docker in the long run. I've added several helper scripts, have a look inside.
-
-Navigate to the project folder and run `./menu.sh`
-
-### Installing from the menu
-
-Select the first option and follow the prompts
-
-### Build the docker-compose file
-
-docker-compose uses the `docker-compose.yml` file to configure all the services. Run through the menu to select the options you want to install.
-
-### Docker commands
-
-This menu executes shell scripts in the root of the project. It is not necessary to run them from the menu. Open up the shell script files to see what is inside and what they do.
-
-### Miscellaneous commands
-
-Some helpful commands have been added like disabling swap.
-
-## Running Docker commands
-
-From this point on make sure you are executing the commands from inside the project folder. Docker-compose commands need to be run from the folder where the docker-compose.yml is located. If you want to move the folder make sure you move the whole project folder.
-
-## Starting and Stopping containers
-
-to start the stack navigate to the project folder containing the docker-compose.yml file
-
-To start the stack run:
-`docker-compose up -d` or `./scripts/start.sh`
-
-To stop:
-`docker-compose stop`
-
-The first time you run 'start' the stack docker will download all the images for the web. Depending on how many containers you selected and your internet speed this can take a long while.
-
-The `docker-compose down` command stops the containers then deletes them.
-
-## Persistent data
-
-Docker allows you to map folders inside your containers to folders on the disk. This is done with the "volume" key. There are two types of volumes. Modification to the container are reflected in the volume.
-
-## See Wiki for further info
-
-[Wiki](https://github.com/gcgarner/IOTstack/wiki)
-
-## Add to the project
-
-Feel free to add your comments on features or images that you think should be added.
-
-## Contributions
-
-If you use some of the tools in the project please consider donating or contributing on their projects. It doesn't have to be monetary, reporting bugs and PRs help improve the projects for everyone.
-
-### Thanks
-
-@mrmx, @oscrx, @brianimmel, @Slyke, @AugustasV, @Paulf007, @affankingkhan, @877dev, @Paraphraser, @stfnhmplr, @peyanski, @cmskedgell
+Video: https://youtu.be/a6mjt8tWUws

docs/Accessing-your-Device-from-the-internet.md

@@ -0,0 +1,44 @@
+# Accessing your device from the internet
+The challenge most of us face with remotely accessing your home network is that you don't have a static IP. From time to time the IP that your ISP assigns to you changes and it's difficult to keep up. Fortunately, there is a solution, a DynamicDNS. The section below shows you how to set up an easy to remember address that follows your public IP no matter when it changes.
+
+Secondly, how do you get into your home network? Your router has a firewall that is designed to keep the rest of the internet out of your network to protect you. Here we install a VPN and configure the firewall to only allow very secure VPN traffic in. 
+
+## DuckDNS
+If you want to have a dynamic DNS point to your Public IP I added a helper script.
+Register with duckdns.org and create a subdomain name. Then edit the `nano ~/IOTstack/duck/duck.sh` file and add your
+
+```bash
+DOMAINS="YOUR_DOMAINS"
+DUCKDNS_TOKEN="YOUR_DUCKDNS_TOKEN"
+```
+
+first test the script to make sure it works `sudo ~/IOTstack/duck/duck.sh` then `cat /var/log/duck.log`. If you get KO then something has gone wrong and you should check out your settings in the script. If you get an OK then you can do the next step. 
+
+Create a cron job by running the following command `crontab -e`
+
+You will be asked to use an editor option 1 for nano should be fine
+paste the following in the editor `*/5 * * * * sudo ~/IOTstack/duck/duck.sh >/dev/null 2>&1` then ctrl+s and ctrl+x to save
+
+Your Public IP should be updated every five minutes
+
+## PiVPN
+pimylifeup.com has an excellent tutorial on how to install [PiVPN](https://pimylifeup.com/raspberry-pi-vpn-server/)
+
+In point 17 and 18 they mention using noip for their dynamic DNS. Here you can use the DuckDNS address if you created one.
+
+Don't forget you need to open the port 1194 on your firewall. Most people won't be able to VPN from inside their network so download OpenVPN client for your mobile phone and try to connect over mobile data. ([More info.](https://en.wikipedia.org/wiki/Hairpinning))
+
+Once you activate your VPN (from your phone/laptop/work computer) you will effectively be on your home network and you can access your devices as if you were on the wifi at home.
+
+I personally use the VPN any time I'm on public wifi, all your traffic is secure.
+
+## Zerotier
+https://www.zerotier.com/
+
+Zerotier is an alternative to PiVPN that doesn't require port forwarding on your router. It does however require registering for their free tier service [here](https://my.zerotier.com/login). 
+
+Kevin Zhang has written a how to guide [here](https://iamkelv.in/blog/2017/06/zerotier.html). Just note that the install link is outdated and should be:
+```
+curl -s 'https://raw.githubusercontent.com/zerotier/ZeroTierOne/master/doc/contact%40zerotier.com.gpg' | gpg --import && \
+if z=$(curl -s 'https://install.zerotier.com/' | gpg); then echo "$z" | sudo bash; fi
+```

docs/Adminer.md

@@ -0,0 +1,12 @@
+# Adminer
+## References
+- [Docker](https://hub.docker.com/_/adminer)
+- [Website](https://www.adminer.org/)
+
+## About
+
+This is a nice tool for managing databases. Web interface has moved to port 9080. There was an issue where openHAB and Adminer were using the same ports. If you have an port conflict edit the docker-compose.yml and under the adminer service change the line to read:
+```
+    ports:
+      - 9080:8080
+```

docs/Backups.md

@@ -0,0 +1,113 @@
+# Backups
+Because containers can easily be rebuilt from docker hub we only have to back up the data in the "volumes" directory.
+
+## Cloud Backups
+### Dropbox-Uploader
+This a great utility to easily upload data from your Pi to the cloud. https://magpi.raspberrypi.org/articles/dropbox-raspberry-pi. It can be installed from the Menu under Backups.
+### rclone (Google Drive)
+This is a service to upload to Google Drive. The config is described [here]( https://medium.com/@artur.klauser/mounting-google-drive-on-raspberry-pi-f5002c7095c2). Install it from the menu then follow the link for these sections:
+* Getting a Google Drive Client ID
+* Setting up the Rclone Configuration
+
+When naming the service in `rclone config` ensure to call it "gdrive"
+
+**The Auto-mounting instructions for the drive in the link don't work on Rasbian**. Auto-mounting of the drive isn't necessary for the backup script.
+
+If you want your Google Drive to mount on every boot then follow the instructions at the bottom of the wiki page
+
+
+## Influxdb
+`~/IOTstack/scripts/backup_influxdb.sh` does a database snapshot and stores it in ~/IOTstack/backups/influxdb/db . This can be restored with the help a script (that I still need to write)
+
+## Docker backups
+The script `~/IOTstack/scripts/docker_backup.sh` performs the master backup for the stack. 
+
+This script can be placed in a cron job to backup on a schedule.
+Edit the crontab with`crontab -e`
+Then add `0 23 * * * ~/IOTstack/scripts/docker_backup.sh >/dev/null 2>&1` to have a backup every night at 23:00.
+
+This script cheats by copying the volume folder live. The correct way would be to stop the stack first then copy the volumes and restart. The cheating method shouldn't be a problem unless you have fast changing data like in influxdb. This is why the script makes a database export of influxdb and ignores its volume. 
+
+### Cloud integration
+The docker_backup.sh script now no longer requires modification to enable cloud backups. It now tests for the presence of and enable file in the backups folder
+#### Drobox-Uploader
+The backup tests for a file called `~/IOTstack/backups/dropbox`, if it is present it will upload to dropbox. To disable dropbox upload delete the file. To enable run `sudo touch ~/IOTstack/backups/dropbox`
+#### rclone
+The backup tests for a file called `~/IOTstack/backups/rclone`, if it is present it will upload to google drive. To disable rclone upload delete the file. To enable run `sudo touch ~/IOTstack/backups/rclone`
+
+#### Pruning online backups
+@877dev has added functionality to prune both local and cloud backups. For dropbox make sure you dont have any files that contain spaces in your backup directory as the script cannot handle it at this time.
+
+### Restoring a backup
+The "volumes" directory contains all the persistent data necessary to recreate the container. The docker-compose.yml and the environment files are optional as they can be regenerated with the menu. Simply copy the volumes directory into the IOTstack directory, Rebuild the stack and start. 
+
+## Added your Dropbox token incorrectly or aborted the install at the token screen
+
+Make sure you are running the latest version of the project [link](https://github.com/gcgarner/IOTstack/wiki/Updating-the-Project).
+
+Run `~/Dropbox-Uploader/dropbox_uploader.sh unlink` and if you have added it key then it will prompt you to confirm its removal. If no key was found it will ask you for a new key.
+
+Confirm by running `~/Dropbox-Uploader/dropbox_uploader.sh` it should ask you for your key if you removed it or show you the following prompt if it has the key:
+
+```
+ $ ~/Dropbox-Uploader/dropbox_uploader.sh
+Dropbox Uploader v1.0
+Andrea Fabrizi - [email protected]
+
+Usage: /home/pi/Dropbox-Uploader/dropbox_uploader.sh [PARAMETERS] COMMAND...
+
+Commands:
+	 upload   <LOCAL_FILE/DIR ...>  <REMOTE_FILE/DIR>
+	 download <REMOTE_FILE/DIR> [LOCAL_FILE/DIR]
+	 delete   <REMOTE_FILE/DIR>
+	 move     <REMOTE_FILE/DIR> <REMOTE_FILE/DIR>
+	 copy     <REMOTE_FILE/DIR> <REMOTE_FILE/DIR>
+	 mkdir    <REMOTE_DIR>
+....
+
+```
+
+Ensure you **are not** running as sudo as this will store your api in the /root directory as `/root/.dropbox_uploader`
+
+If you ran the command with sudo the remove the old token file if it exists with either `sudo rm /root/.dropbox_uploader` or `sudo ~/Dropbox-Uploader/dropbox_uploader.sh unlink`
+
+## Auto-mount Gdrive with rclone
+
+To enable rclone to mount on boot you will need to make a user service. Run the following commands
+
+```bash
+mkdir -p ~/.config/systemd/user
+nano ~/.config/systemd/user/gdrive.service
+```
+Copy the following code into the editor, save and exit
+
+```
+[Unit]
+Description=rclone: Remote FUSE filesystem for cloud storage
+Documentation=man:rclone(1)
+
+[Service]
+Type=notify
+ExecStartPre=/bin/mkdir -p %h/mnt/gdrive
+ExecStart= \
+  /usr/bin/rclone mount \
+  --fast-list \
+  --vfs-cache-mode writes \
+  gdrive: %h/mnt/gdrive
+
+[Install]
+WantedBy=default.target
+```
+enable it to start on boot with: (no sudo)
+```bash
+systemctl --user enable gdrive.service
+```
+start with 
+```bash
+systemctl --user start gdrive.service
+```
+if you no longer want it to start on boot then type:
+```bash
+systemctl --user disable gdrive.service
+```
+