os: Switch to bookworm, drop bullseye support (#543)

* Bump forklift and pallet-standard

* Update `forklift` invocation in OS build scripts

* Undo autoformatter regression

* Bump pallet-standard

* Stop automatically building bullseye images

* Update `CHANGELOG.md`

* Build bookworm-based fairscope-latest images

* Fix typo in `CHANGELOG.md`

* Update `CHANGELOG.md`

* Remove bullseye-specific OS build/setup code

* Update nonstandard install guide and make it lower-maintenance

* Add a bit of clarification to the nonstandard install guide

* Add a link to a tech ref doc from the nonstandard install guide

Author: ethanjli

.github/workflows/build-os-bookworm.yml

@@ -17,7 +17,7 @@ on:
       - 'software/**'
       - '!software/CHANGELOG.md'
       - '.github/workflows/build-os*.yml'
-  #merge_group:
+  merge_group:
   workflow_dispatch:
     inputs:
       git-ref:
@@ -37,6 +37,7 @@ jobs:
         variant:
           - adafruithat
           - planktoscopehat
+          - fairscope-latest
         base_variant_name:
           - lite
     permissions:

.github/workflows/build-os-bullseye-dx.yml

@@ -330,7 +330,7 @@ jobs:
             version="$(printf "%.7s" "${{ github.sha }}")"
           fi
           variant="${{ inputs.variant }}"
-          if [[ "${{ inputs.base_release_name }}" != "bullseye" ]]; then
+          if [[ "${{ inputs.base_release_name }}" != "bookworm" ]]; then
             variant="$variant.${{ inputs.base_release_name }}"
           fi
           if [[ "${{ inputs.arch }}" != "arm64" ]]; then

.github/workflows/build-os-bullseye.yml

@@ -1,20 +1,20 @@
 # Non-Standard Installation
 
-This page provides instructions for setting up non-standard versions of the PlanktoScope OS on a PlanktoScope. The PlanktoScope project also uses this same process for creating the official PlanktoScope software SD card images used in the [standard software installation process](standard-install.md).
+This page provides instructions for setting up non-standard versions of the PlanktoScope OS on a PlanktoScope. The PlanktoScope project uses an automated version of this process for creating the official PlanktoScope software SD card images used in the [standard software installation process](standard-install.md).
 
 ## Prerequisites
 
 This guide assumes that:
 
 1. You have previous experience with using the command-line terminal on the Raspberry Pi OS or another Linux distribution.
 2. You have already confirmed that your PlanktoScope works without any problems with software installed by the standard PlanktoScope software setup process.
-3. You already know how to use the PlanktoScope software.
+3. You are already familiar with the PlanktoScope software and the way the PlanktoScope project does [versioning](../../reference/software/release-process.md) of the PlanktoScope OS.
 
-If you have not used the PlanktoScope software before, you should first start with the standard software setup process in order to troubleshoot any problems with your PlanktoScope hardware; you can then try the non-standard setup process afterwards.
+If you have not used the PlanktoScope software before, you should first start with the standard software setup process in order to troubleshoot any problems with your PlanktoScope hardware; you can then try the non-standard setup process later.
 
 In order to complete the non-standard setup process, you will need all of the following:
 
-1. A Raspberry Pi computer. We only test to ensure that the PlanktoScope software works on the Raspberry Pi 4; it may or may not work on the Raspberry Pi 3, and it does not yet work on the Raspberry Pi 5.
+1. A Raspberry Pi computer. Each version of PlanktoScope OS is only compatible with certain versions of Raspberry Pi computers; for more information, please check the [software product specifications](../../reference/software/product-specs.md) for the version of PlanktoScope OS which you want to set up.
 2. A keyboard connected to your Raspberry Pi.
 3. A display connected to your Raspberry Pi.
 4. A micro-SD card for your Raspberry Pi.
@@ -25,17 +25,29 @@ In order to complete the non-standard setup process, you will need all of the fo
 
 ### Download a Raspberry Pi OS SD card image
 
-The setup scripts for the PlanktoScope OS assume that you will be setting up the PlanktoScope software on a 64-bit version of the Raspberry Pi OS with Debian version 11 (bullseye), preferably the version released on 2023-03-12. You can choose any of the following three variants of that version of the Raspberry Pi OS, depending on your needs:
+#### Choose an appropriate Raspberry Pi OS release version
 
-- ["Raspberry Pi OS with desktop"](https://downloads.raspberrypi.com/raspios_oldstable_arm64/images/raspios_oldstable_arm64-2024-03-12/2024-03-12-raspios-bullseye-arm64.img.xz)
-- ["Raspberry Pi OS with desktop and recommended software"](https://downloads.raspberrypi.com/raspios_oldstable_full_arm64/images/raspios_oldstable_full_arm64-2024-03-12/2024-03-12-raspios-bullseye-arm64-full.img.xz)
-- ["Raspberry Pi OS Lite"](https://downloads.raspberrypi.com/raspios_oldstable_lite_arm64/images/raspios_oldstable_lite_arm64-2024-03-12/2024-03-12-raspios-bullseye-arm64-lite.img.xz)
-
-The standard PlanktoScope software SD card images are built on the Raspberry Pi OS Lite image, which only provides a command-line interface, without a graphical desktop environment or web browser; because the PlanktoScope's graphical user interface must be accessed from a web browser, you might prefer to use the "Raspberry Pi OS with desktop" image in order to have a graphical desktop environment with a web browser. This would allow you to operate the PlanktoScope by plugging in a display, keyboard, and mouse to your Raspberry Pi; otherwise, you will have to connect to the PlanktoScope from another device over Ethernet or Wi-Fi in order access the PlanktoScope's graphical user interface.
+The setup scripts for the PlanktoScope OS assume that you will be setting up the PlanktoScope software on a 64-bit version of the Raspberry Pi OS with the same Raspberry Pi OS release name (e.g. `bookworm`) and release date (e.g. `2024-11-19`) as what we use in building our official SD card images of the PlanktoScope OS.
 
 !!! warning
+    The PlanktoScope OS setup scripts are very likely to work incorrectly (in obvious or subtle ways) on other versions of Raspberry Pi OS. For example, we are aware that some Raspberry Pi OS versions come with buggy or incompatible versions of system packages required by PlanktoScope OS.
+
+Here is how you can determine the appropriate release name and release date to use for the version of PlanktoScope OS which you want to set up:
+
+1. Base OS release name: check the "distro" information for the base operating system for that version of PlanktoScope OS in the [software product specifications](../../reference/software/product-specs.md). For example, the section for PlanktoScope OS v2025.0.0 lists its base distro as "Raspberry Pi OS 12 (bookworm)", so the base OS release name is `bookworm`. 
+2. Base OS release date: check the `base_release_date` field of the matching `build-os-{base OS release name}.yml` file (e.g. `build-os-bookworm.yml`) in the `.github/workflows` subdirectory of the PlanktoScope OS repository at that version of the PlanktoScope OS. For example, v2025.0.0's required base release date might be something like `2024-11-19`.
+
+If you need help determining this information, please post a message in the `#6-dev-software` channel on the PlanktoScope Slack and mention the version of the PlanktoScope OS which you want to set up with this non-standard method, so that we can help you.
+
+#### Choose an appropriate Raspberry Pi OS variant
+
+The standard PlanktoScope software SD card images are built on the Raspberry Pi OS Lite image for the appropriate release version of Raspberry Pi OS. The Lite image only provides a command-line interface, without a graphical desktop environment or web browser. Because the PlanktoScope's graphical user interface must be accessed from a web browser, you might prefer to use the "Raspberry Pi OS with desktop" image in order to have a graphical desktop environment with a web browser. This would allow you to operate the PlanktoScope by plugging in a display, keyboard, and mouse to your Raspberry Pi; otherwise, you will have to connect to the PlanktoScope from another device over Ethernet or Wi-Fi in order to access the PlanktoScope's graphical user interface.
+
+We also build and use (but we don't officially release or support) developer-focused SD card images (which we label "-dx" as an abbreviation for "developer experience") which come with the Raspberry Pi OS graphical desktop environment (which is called "Raspberry Pi OS with desktop" in the Raspberry Pi OS download page). To learn more about these `-dx` images, please post a message in the `#6-dev-software` channel on the PlanktoScope Slack.
+
+#### Download the correct Raspberry Pi OS image
 
-    The latest version of Raspberry Pi OS, with Debian version 12 (bookworm), can be downloaded from [the Raspberry Pi Operating system images page](https://www.raspberrypi.com/software/operating-systems/), but the PlanktoScope software setup scripts do not yet work on Debian version 12; that page also has links named "Archive" under the download buttons where you can find older versions with Debian version 11 (bullseye) under the "Raspberry Pi OS (Legacy)" section; those links are the same as the links we listed above.
+Once you have determined the appropriate release name, release date, and variant of Raspberry Pi OS, you should download the appropriate 64-bit version of Raspberry Pi OS from the [Raspberry Pi OS download page](https://www.raspberrypi.com/software/operating-systems/). If the appropriate release date is not shown on that page, you may need to use the "Archive" link in the section of that page corresponding to the appropriate release name and variant; with the "Archive" link, you can choose an image with the appropriate release date. If you need help, please post a message in the `#6-dev-software` channel on the PlanktoScope Slack and mention the release name, release date, and variant which you are trying to download.
 
 ### Write the OS image to an SD card
 

.github/workflows/build-os.yml

@@ -18,8 +18,9 @@ All dates in this file are given in the [UTC time zone](https://en.wikipedia.org
 
 ### Changed
 
+- (Breaking change; System) The official PlanktoScope OS images are now built on Raspberry Pi OS 12 (bookworm) instead of Raspberry Pi OS 11 (bullseye). No support will be given for running the PlanktoScope OS setup scripts on bullseye base images, and the OS setup scripts will not work on bullseye.
 - (Application: GUI) The Node-RED dashboard now initializes the Sample page's Dilution Factor field to 1.0, instead of leaving it empty.
-- (System: networking) Wi-Fi hotspot behavior and network connection management is now based on NetworkManager, in preparation for an upgrade to Raspberry Pi OS 12 (bookworm). Thus, NetworkManager is installed on bullseye-based images, while dhcpcd is now uninstalled on bullseye-based images. As part of this change the previous autohotspot service has been removed, as it's redundant with functionality provided by NetworkManager.
+- (System: networking) Wi-Fi hotspot behavior and network connection management is now based on NetworkManager, as part of an upgrade to Raspberry Pi OS 12 (bookworm). As part of this change the previous autohotspot service has been removed, as it's redundant with functionality now provided by NetworkManager.
 
 ### Removed
 

documentation/docs/setup/software/nonstandard-install.md

@@ -10,13 +10,6 @@ sudo -E apt-get install -y -o Dpkg::Progress-Fancy=0 \
   network-manager firewalld dnsmasq-base
 sudo systemctl enable NetworkManager.service
 
-# Uninstall dhcpcd if we're on bullseye
-DISTRO_VERSION_ID="$(. /etc/os-release && echo "$VERSION_ID")"
-if [ "$DISTRO_VERSION_ID" -le 11 ]; then # Support Raspberry Pi OS 11 (bullseye)
-  sudo -E apt-get purge -y -o Dpkg::Progress-Fancy=0 \
-    dhcpcd5
-fi
-
 # Set the wifi country
 # FIXME: instead have the user set the wifi country via a first-launch setup wizard, and do it
 # without using raspi-config. It should also be updated if the user changes the wifi country.

software/CHANGELOG.md

@@ -14,7 +14,7 @@
 # it harder for our project to enable running the PlanktoScope software on computers besides the
 # Raspberry Pi. So we should avoid adding more raspi-config commands.
 
-if ! command -v raspi-config &> /dev/null; then
+if ! command -v raspi-config &>/dev/null; then
   echo "Warning: raspi-config is unavailable, so no RPi-specific hardware configuration will be applied!"
   exit 0
 fi
@@ -24,17 +24,8 @@ sudo raspi-config nonint do_spi 0
 sudo raspi-config nonint do_i2c 0
 
 # The following command enables the serial port and serial port console.
-# do_serial_cons and do_serial_hw are needed for Raspberry Pi OS 12 (bookworm) and above, while
-# do_serial is needed for Raspberry Pi OS (bullseye).
-DISTRO_VERSION_ID="$(. /etc/os-release && echo "$VERSION_ID")"
-if [ $DISTRO_VERSION_ID -ge 12 ]; then # Support Raspberry Pi OS 12 (bookworm)
-  sudo raspi-config nonint do_serial_hw 0
-  sudo raspi-config nonint do_serial_cons 0
-else # Support Raspberry Pi OS 11 (bullseye)
-  sudo raspi-config nonint do_serial 0
-fi
+sudo raspi-config nonint do_serial_hw 0
+sudo raspi-config nonint do_serial_cons 0
 
-# The following command enables the camera on the 32-bit Raspberry Pi OS (ARMv7):
-sudo raspi-config nonint do_camera 0
 # The following command disables legacy camera support so that we can use libcamera:
 sudo raspi-config nonint do_legacy 1

software/distro/setup/base-os/networking/install.sh

@@ -12,12 +12,7 @@ sudo -E apt-get install -y -o Dpkg::Progress-Fancy=0 \
 # The following command enables the serial port but disables the login shell over the serial port.
 # We use this because the serial port is reserved for a GPS device.
 # Note that this overrides a setting in the base-os/platform-hardware/config.sh script.
-DISTRO_VERSION_ID="$(. /etc/os-release && echo "$VERSION_ID")"
-if [ "$DISTRO_VERSION_ID" -ge 12 ]; then # Support Raspberry Pi OS 12 (bookworm)
-  sudo raspi-config nonint do_serial_cons 1
-else # Support Raspberry Pi OS 11 (bullseye)
-  sudo raspi-config nonint do_serial 2
-fi
+sudo raspi-config nonint do_serial_cons 1
 
 # Enable automatic time update from GPSD
 file="/boot/config.txt"