SolarNodeOS Managed Deploy Guide

SolarNodeOS is a Debian-based operating system tailored specifically for SolarNode devices. SolarNetwork Foundation maintains a Debian package repository with core packages needed by a SolarNode device. When deploying a fleet of nodes, you will need to consider the following:

What node plugin(s) will be using, that you could have installed in each node? For example you might know in advance that all your nodes will need to integrate with modbus devices, and thus would need the Modbus Device plugin installed.
What standardized plugin configuration would be helpful, that you could have auto-configured in each node? For example you might be integrating with a known modbus device that would require a known configuration.

This guide will describe how you can create a customized SolarNodeOS, possibly with custom OS packages, that you can then easily deploy to as many nodes as needed.

⚠️ Note that when following the recommendations in this guide, you should not use the Plugins page in the SolarNode setup app to install/upgrade plugins. The reason for that is because the Plugins manager won't know that plugins have been installed via OS packages, and you could end up with a broken SolarNode system. In practice this is not much of an inconvenience, as it is easier and more consistent in the long run to maintain nodes using packages as outlined in this guide.

General requirements

This guide assumes you have access to a Debian-based Linux workstation. This could be an actual Linux workstation, a virtual machine such as VMWare or Virtual Box, or even Windows Subsystem for Linux (WSL) on Windows can work. Any Debian-based Linux distribution should work, such as Ubuntu. This guide has been written with Debian 10 (Buster) in mind. You should be familiar and comfortable with using shell commands.

At a minimum, the following tools are required:

Git, and Git LFS
Java version 8 or higher

Typically these can be installed like:

# install general requirements
sudo apt install git git-lfs openjdk-11-jdk-headless

Build environment

You will need to set up a build environment for running the SolarNodeOS customization scripts and/or creating packages. In this guide, we'll create a ~/Documents/SolarNodeOS directory to hold everything, and all example command will be based on this directory:

# create build environment base directory
mkdir -p ~/Documents/SolarNodeOS

Customize "vanilla" SolarNodeOS image

SolarNetwork Foundation provides basic SolarNodeOS device image files that can be deployed to a variety of hardware devices, for example the Raspberry Pi family of devices. This section will show how you can customize one of those images, for example to install/update software or make configuration changes. The outcome of the customization process is a new SolarNodeOS based image that you can then deploy to SolarNode devices.

Setup customization environment

You need to download the SolarNetwork/solarnode-os-images repository:

# download (clone) the solarnode-os-images repository
cd ~/Documents/SolarNodeOS
git clone https://github.com/SolarNetwork/solarnode-os-images.git

Next you will need to install some required software:

# install customization tools
sudo apt install bc binfmt-support btrfs-progs e2fsprogs qemu qemu-user-static \
  rsync systemd-container xz-utils

Download "vanilla" SolarNodeOS image

Download the image you want to use as the starting point for your custom SolarNodeOS image, and save that to the ~/Documents/SolarNodeOS directory. In this guide, we'll customize the solarnodeos-deb10-pi-2GB-20210303.img.xz Raspberry PI image.

The `customize.sh` script

The solarnode-os-images/debian/bin/customize.sh script is what will drive the customization process. This script will take an image file as input and then performs these basic steps:

Start up a light-weight virtual environment from the input image.
Either run a customization script or launch an interactive shell for you to customize manually.
Save the customized environment as a new image.

The customize.sh script has some other helpful tricks up its sleeve, such as the ability to grow the image filesystem if you need to install a fair bit of custom software.

To get started, you need to create a shell script that performs the customization tasks you need. This script will be executed in the virtual SolarNodeOS environment, and its job is to make any changes to SolarNodeOS you want to have in your customized image. For this guide let's start with a very basic script that simply installs the tmux command, a handy utility to help with remote node management:

#!/usr/bin/env sh

echo "!!! Begin my SolarNodeOS customizations..."

apt install -qy yasdishell

echo "!!! Finished my SolarNodeOS customizations."

Save this script as ~/Documents/SolarNodeOS/my-cust.sh. Then we can run the customization script, configuring the output image file to be named my-solarnodeos-DATE.img.xz where DATE is the current date:

sudo ~/Documents/SolarNodeOS/solarnode-os-images/debian/bin/customize.sh -v -z \
    -o ~/Documents/SolarNodeOS/my-solarnodeos-$(date '+%Y%m%d').img \
    ~/Documents/SolarNodeOS/solarnodeos-deb10-pi-2GB-20210303.img.xz \
    ~/Documents/SolarNodeOS/my-cust.sh

The script produces a lot of output, but somewhere in there you should see the output from your my-cust.sh script:

!!! Begin my SolarNodeOS customizations...
Reading package lists...
Building dependency tree...
Reading state information...
The following additional packages will be installed:
  libevent-2.1-6 libutempter0
The following NEW packages will be installed:
  libevent-2.1-6 libutempter0 tmux
0 upgraded, 3 newly installed, 0 to remove and 27 not upgraded.
Need to get 410 kB of archives.
After this operation, 962 kB of additional disk space will be used.
...
Unpacking tmux (2.8-3) ...
Setting up libevent-2.1-6:armhf (2.1.8-stable-4) ...
Setting up libutempter0:armhf (1.1.6-3) ...
Setting up tmux (2.8-3) ...
Processing triggers for libc-bin (2.28-10+rpi1) ...
!!! Finished my SolarNodeOS customizations.

What you end up with is:

The custom image file, named my-solarnodeos-DATE.img
A checksum of the image file, named my-solarnodeos-DATE.img.sha256
A compressed image file, named my-solarnodeos-DATE.img.xz
A checksum of the compressed image file, named my-solarnodeos-DATE.img.xz.sha256

$ ls -lh my-solarnodeos*
-rw-r--r-- 1 root root 1.3G Apr 11 14:47 my-solarnodeos-20210411.img
-rw-r--r-- 1 root root   94 Apr 11 14:47 my-solarnodeos-20210411.img.sha256
-rw-r--r-- 1 root root 266M Apr 11 14:50 my-solarnodeos-20210411.img.xz
-rw-r--r-- 1 root root   97 Apr 11 14:50 my-solarnodeos-20210411.img.xz.sha256

Interactive customization

During the development of your customization script, it can be handy to test out changes in a more interactive fashion. You can do this by passing the -i argument to customize.sh, which will launch a shell in the SolarNodeOS virtual environment for you. Once you've completed your changes, simply exit and customize.sh will continue and generate the customized output image for you. If you want to throw away your changes and not bother creating the output image just exit 1 to signal an error condition and customize.sh will clean up without creating the output image.

Note that you must still provide a customization script to customize.sh. That script will be copied into the SolarNodeOS virtual environment as a script named customize. This makes it convenient for troubleshooting your script, as you can then run it manually to see what problems there are. For example:

# run interactively
sudo ~/Documents/SolarNodeOS/solarnode-os-images/debian/bin/customize.sh -v -z \
    -o ~/Documents/SolarNodeOS/my-solarnodeos-$(date '+%Y%m%d').img \
    -i \
    ~/Documents/SolarNodeOS/solarnodeos-deb10-pi-2GB-20210303.img.xz \
    ~/Documents/SolarNodeOS/my-cust.sh
...
'/home/matt/Documents/SolarNodeOS/my-cust.sh' -> '/tmp/sn-CEkN7/var/tmp/sn-iYN9f/customize'
Spawning container solarnode-cust on /tmp/sn-CEkN7.
Press ^] three times within 1s to kill container.

root@solarnode-cust:/var/tmp/sn-iYN9f# ls -l
total 4
-rwxr-xr-x 1 1000 1000 144 Apr 11 14:47 customize

root@solarnode-cust:/var/tmp/sn-iYN9f# ./customize
!!! Begin my SolarNodeOS customizations...
...
!!! Finished my SolarNodeOS customizations.

root@solarnode-cust:/var/tmp/sn-iYN9f# exit 1
exit
Container solarnode-cust failed with error code 1.
!!!
!!! Error with interactive setup in container!
!!!
Restoring original /tmp/sn-CEkN7/etc/resolv.conf
removed '/tmp/sn-CEkN7/var/tmp/sn-iYN9f/customize'
removed directory '/tmp/sn-CEkN7/var/tmp/sn-iYN9f'
Enabling preload shared libs in /tmp/sn-CEkN7/etc/ld.so.preload... OK
Unmounting source SOLARBOOT filesystem /tmp/sn-CEkN7//boot.
Unmounting source SOLARNODE filesystem /tmp/sn-CEkN7.
Closing source image loop device /dev/loop0.
Deleted /tmp/img-tIpFI

Create SolarNode plugin bundle package

A good way to manage SolarNode deployments with a known set of SolarNode plugins on them is to create OS packages out of the set of SolarNode plugins you want to include on your nodes. The SolarNetwork/solarnetwork-build project contains support for you to do just that. How it works is that you'll use the SolarNetwork build system to select the plugins you want to deploy and download them all into a directory structure suitable for copying to a SolarNode device.

From there you can use your OS packaging tool of choice to create the package. For this guide, we'll use the standard make tool along with fpm to create the package.

Package software requirements

Make sure you have make and fpm available like this:

sudo apt install ant ruby ruby-dev build-essential
sudo gem install --no-ri --no-rdoc fpm

Then download (clone) the solarnetwork-build project, and we'll switch to the develop branch to access the latest & greatest:

cd ~/Documents/SolarNodeOS
git clone https://github.com/SolarNetwork/solarnetwork-build.git
cd solarnetwork-build
git checkout develop

Setup custom package environment

First create a directory for our custom package, which we'll name my-solarnode-app-core:

# create package directory
mkdir -p ~/Documents/SolarNodeOS/packages/my-solarnode-app-core/debian

# create symlink to solarnetwork-build dir
ln -s ../../../solarnetwork-build

Plugin configuration

Now we'll create an Ivy XML configuration that defines which plugins we want installed. The available plugins maintained by SolarNetwork Foundation can be seen here. For this guide, let's install support for Modbus devices. For that, we need a few plugins, which show up as <dependency> elements in the XML. Create an ivy-my-main.xml file with the following content:

<?xml version="1.0" encoding="UTF-8"?>
<ivy-module version="2.0">
	<info organisation="SolarNetwork" module="SolarNode"/>
	<configurations>
		<conf name="runtime" visibility="public" description="The Runtime"/>
	</configurations>
	<dependencies defaultconfmapping="runtime->default(runtime)">

		<!-- Modbus support -->
		<dependency org="net.solarnetwork.external" name="net.solarnetwork.external.jamod.pjc" rev="latest.release"/>
		<dependency org="net.solarnetwork.node" name="net.solarnetwork.node.io.modbus" rev="latest.release"/>
		<dependency org="net.solarnetwork.node" name="net.solarnetwork.node.io.modbus.jamod" rev="latest.release"/>
		<dependency org="net.solarnetwork.node" name="net.solarnetwork.node.datum.modbus" rev="latest.release"/>

		<!-- solarnode-app-core excludes -->
		<exclude artifact="net.solarnetwork.external.org.rxtx"/>
		<exclude artifact="net.solarnetwork.common"/>
		<exclude artifact="net.solarnetwork.common.mqtt"/>
		<exclude artifact="net.solarnetwork.common.web"/>
		<exclude artifact="net.solarnetwork.node"/>
		<exclude artifact="net.solarnetwork.node.dao.jdbc"/>
		<exclude artifact="net.solarnetwork.node.io.mqtt"/>
		<exclude artifact="net.solarnetwork.node.setup.web"/>

		<!-- Global excludes provided by the base system -->
		<exclude org="commons-(beanutils|codec|collections|digester|fileupload|io)" matcher="regexp"/>
		<exclude org="com.fasterxml.jackson.core"/>
		<exclude org="com.fasterxml.jackson.dataformat"/>
		<exclude org="com.fasterxml.jackson.datatype"/>
		<exclude org="javax.measure"/>
		<exclude org="javax.servlet"/>
		<exclude org="javax.xml.bind"/>
		<exclude org="io.netty"/>
		<exclude org="joda-time"/>
		<exclude org="net.java.dev.jna"/>
		<exclude org="net.sf.supercsv"/>
		<exclude org="org.apache.commons"/>
		<exclude org="org.apache.servicemix.bundles"/>
		<exclude org="org.apache.tomcat"/>
		<exclude org="org.eclipse.paho"/>
		<exclude org="org.eclipse.virgo.mirrored"/>
		<exclude org="org.glassfish.tyrus.bundles"/>
		<exclude org="org.osgi"/>
		<exclude org="org.quartz-scheduler"/>
		<exclude org="org.slf4j"/>
		<exclude org="org.springframework"/>
		<exclude org="org.springframework.security"/>
		<exclude org="org.mitre.dsmiley.httpproxy"/>
	</dependencies>
</ivy-module>

There's a lot of stuff packed in there, but the important lines are the <dependency> ones, which are dictating which plugins to include in the package.

`make` script

Now create a Makefile with the following content:

NAME = my-solarnode-app-core
VERSION = 1.0.0-1
SN_BUILD_ROOT = ${CURDIR}/solarnetwork-build
DEB_BUILD_ROOT = $(SN_BUILD_ROOT)/solarnode-deploy/generic/build/deb
IVY_FILE = ${CURDIR}/ivy-my-main.xml

deb : app-core table
	fpm -s dir \
	-t deb \
	-n $(NAME) \
	-v $(VERSION) \
	--description 'My SolarNode standard device support' \
	--chdir $(DEB_BUILD_ROOT) \
	-a all \
	--maintainer 'Me <me@localhost>' \
	--vendor 'Me' \
	--license 'Proprietary' \
	-f \
	-d 'solarnode-app-core (>= 1.12.0)' \
	var

clean : 
	rm $(NAME)_$(VERSION)_all.deb

app-core :
	ant -f "$(SN_BUILD_ROOT)/solarnode-deploy/generic/build.xml" \
		"-Divy.file=$(IVY_FILE)" \
		clean deb-package-assemble

table :
	java -jar "$(SN_BUILD_ROOT)/solarnetwork-osgi-lib/lib/bh.jar" \
		"$(SN_BUILD_ROOT)/solarnode-deploy/generic/build/deb"

This build script will run the build and download the latest version of each required plugin, and then print out a handy Markdown-formatted table of all the plugins that are included in the package, like this:

Name	ID	Vers
Generic Modbus Datum Source	`n.s.n.datum.modbus`	2.3.0
Java Modbus Library (PJC)	`n.s.external.jamod.pjc`	1.2.0
Modbus Communication Support (Jamod)	`n.s.n.io.modbus.jamod`	1.1.0
Modbus Communication Support API	`n.s.n.io.modbus`	3.2.0
PureJavaComm	`n.s.external.pjc`	1.0.2

The package will be named my-solarnode-app-core_1.0.0-1_all.deb and you can inspect its contents with dpkg -c like this:

$ dpkg -c my-solarnode-app-core_1.0.0-1_all.deb 
drwxr-xr-x 0/0               0 2021-04-11 15:35 ./
drwxr-xr-x 0/0               0 2021-04-11 15:35 ./usr/
drwxr-xr-x 0/0               0 2021-04-11 15:35 ./usr/share/
drwxr-xr-x 0/0               0 2021-04-11 15:35 ./usr/share/doc/
drwxr-xr-x 0/0               0 2021-04-11 15:35 ./usr/share/doc/my-solarnode-app-core/
-rw-r--r-- 0/0             148 2021-04-11 15:35 ./usr/share/doc/my-solarnode-app-core/changelog.gz
drwxr-xr-x 0/0               0 2021-04-11 15:35 ./var/
drwxr-xr-x 0/0               0 2021-04-11 15:35 ./var/lib/
drwxr-xr-x 0/0               0 2021-04-11 15:35 ./var/lib/solarnode/
drwxr-xr-x 0/0               0 2021-04-11 15:35 ./var/lib/solarnode/app/
drwxr-xr-x 0/0               0 2021-04-11 15:35 ./var/lib/solarnode/app/main/
-rw-r--r-- 0/0           36879 2021-02-26 15:17 ./var/lib/solarnode/app/main/net.solarnetwork.node.datum.modbus-2.3.0.jar
-rw-r--r-- 0/0           32175 2021-02-02 10:30 ./var/lib/solarnode/app/main/net.solarnetwork.node.io.modbus.jamod-1.1.0.jar
-rw-r--r-- 0/0          155421 2020-08-31 14:09 ./var/lib/solarnode/app/main/net.solarnetwork.external.jamod.pjc-1.2.0.rc1-SN20200831A.jar
-rw-r--r-- 0/0           56392 2021-02-02 10:30 ./var/lib/solarnode/app/main/net.solarnetwork.node.io.modbus-3.2.0.jar
-rw-r--r-- 0/0          127161 2020-08-31 14:08 ./var/lib/solarnode/app/main/net.solarnetwork.external.pjc-1.0.2.SN20200831A.jar

Customize SolarNodeOS image with custom package

This section shows how you can combine the steps you took in the previous two sections to customize a SolarNodeOS image by installing your own custom plugin package. This time when we run the customize.sh script we'll make our ~/Documents/SolarNodeOS/packages directory available to the virtual environment as /tmp/packages. That is done by passing an additional ~/Documents/SolarNodeOS/packages:/tmp/packages argument. Knowing that directory will be available, we can then update our my-cust.sh script to install the my-solarnode-app-core_1.0.0-1_all.deb package for us. Change the my-cust.sh script to this:

#!/usr/bin/env sh

echo "!!! Begin my SolarNodeOS customizations..."

dpkg -i /tmp/packages/my-solarnode-app-core/debian/my-solarnode-app-core_1.0.0-1_all.deb

echo "!!! Finished my SolarNodeOS customizations."

Now run customize.sh like this:

sudo ~/Documents/SolarNodeOS/solarnode-os-images/debian/bin/customize.sh -v -z \
    -o ~/Documents/SolarNodeOS/my-solarnodeos-$(date '+%Y%m%d').img \
    ~/Documents/SolarNodeOS/solarnodeos-deb10-pi-2GB-20210303.img.xz \
    ~/Documents/SolarNodeOS/my-cust.sh \
    ~/Documents/SolarNodeOS/packages:/tmp/packages

This time, notice the my-solarnode-app-core package is installed:

!!! Begin my SolarNodeOS customizations...
Selecting previously unselected package my-solarnode-app-core.
(Reading database ... 22383 files and directories currently installed.)
Preparing to unpack .../my-solarnode-app-core_1.0.0-1_all.deb ...
Unpacking my-solarnode-app-core (1.0.0-1) ...
Setting up my-solarnode-app-core (1.0.0-1) ...
!!! Finished my SolarNodeOS customizations.

Custom Debian package repository on S3

This section outlines how you can customize SolarNodeOS to include support for your own custom Debian package repository hosted on AWS S3 (or compatible service). This is a very nice way of maintaining the software on your nodes over time, because the nodes can access software updates via standard OS tools like apt update && apt upgrade.

S3 Debian package repository requirements

Maintaining an S3 Debian package repository is outside the scope of this guide. There are good tools available for doing this, such as aptly. Once you have the package repository available on S3, you'll need:

An S3 access token and secret with read-only access to the S3 bucket hosting the repository.
The public GPG key used to sign the packages on the S3 repo, in a GPG keyring file.

S3 package repository package

To make it easy to configure the S3 package repository into your customized SolarNodeOS image, we'll create a custom package with all the necessary configuration included, and that package can be installed via the customize.sh process. Let's call this package my-s3-repo:

# create package directory
mkdir -p ~/Documents/SolarNodeOS/packages/my-s3-repo/debian

cd ~/Documents/SolarNodeOS/packages/my-s3-repo/debian

# create package directories
mkdir -p etc/apt/sources.list.d
mkdir -p etc/apt/trusted.gpg.d

# just to be safe, ignore our s3 credentials file from being added to git
echo '/etc/apt/s3auth.conf' >.gitignore

Now create a etc/apt/sources.list.d/private-s3.list file with content similar to this, but with your S3 bucket name instead of my-debian-repo:

deb s3://my-debian-repo/ buster main

Now copy the public GPG keyring that holds the public GPG key you use to sign the packages in the S3 repository to etc/apt/trusted.gpg.d/my-packaging.gpg.

Finally, create a etc/apt/s3auth.conf file with the appropriate S3 credentials, similar to this:

AccessKeyId = MY_ACCESS_KEY
SecretAccessKey = MY_ACCESS_KEY_SECRET
Region = MY_S3_BUCKET_REGION
Token = ''

Finally, create a Makefile like this:

NAME = my-s3-repo
DIST = 
VERSION = 1.0.0-1
PATHS = etc

deb :
	fpm \
	--input-type dir \
	--output-type deb \
	--name $(NAME) \
	--version $(VERSION) \
	--architecture all \
	--maintainer 'Me <me@localhost>' \
	--vendor 'Me' \
	--description 'My private S3 package repo' \
	--license 'Proprietary' \
	--force \
	--depends 'apt-transport-s3 (>= 1.2.1)' \
	--deb-no-default-config-files \
	$(PATHS)

clean : 
	rm $(NAME)_$(VERSION)_all.deb

Now you can build your package, via make.

Integrating S3 package repository into SolarNodeOS image

Using the same process as outlined previously, you can now update the my-cust.sh script to install the new my-s3-repo_1.0.0-1_all.deb package, and assuming you've published your my-solarnode-app-core package there, can then install that afterwards, like this:

#!/usr/bin/env sh

echo "!!! Begin my SolarNodeOS customizations..."

# Install S3 repo support
dpkg -i /tmp/packages/my-solarnode-app-core/debian/my-s3-repo_1.0.0-1_all.deb

# Now install my-solarnode-app-core, which will come from the S3 repo
apt install -qy my-solarnode-app-core

echo "!!! Finished my SolarNodeOS customizations."

SolarNodeOS Managed Deploy Guide

SolarNodeOS Managed Deploy Guide

General requirements

Build environment

Customize "vanilla" SolarNodeOS image

Setup customization environment

Download "vanilla" SolarNodeOS image

The customize.sh script

Interactive customization

Create SolarNode plugin bundle package

Package software requirements

Setup custom package environment

Plugin configuration

make script

Customize SolarNodeOS image with custom package

Custom Debian package repository on S3

S3 Debian package repository requirements

S3 package repository package

Integrating S3 package repository into SolarNodeOS image

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

The `customize.sh` script

`make` script