Feature: permit running anything-sync-daemon as an unprivileged user

.github/workflows/nix.yml

@@ -0,0 +1,61 @@
+name: NixOS flake checks
+
+on:
+  push:
+  pull_request:
+
+jobs:
+  nix:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        system:
+          - x86_64-linux
+        check:
+          - treefmt
+          - docs
+          - anything-sync-daemon-system
+          - anything-sync-daemon-user
+          - anything-sync-daemon-home-manager
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+      - name: Install Nix
+        uses: nixbuild/nix-quick-install-action@v25
+        with:
+          # Empty `build-users-group` definition required in order to test this
+          # under `act`, at least when using "medium size" images.
+          nix_conf: |
+            system-features = benchmark big-parallel kvm nixos-test
+            keep-derivations = true
+            keep-outputs = true
+            build-users-group =
+        env:
+          # Don't bother using `gh` to fetch `flake.nix`, as it's (a)
+          # unnecessary and (b) problematic when running under `act` (`gh`
+          # bails out, saying that login is required).
+          # https://github.com/nixbuild/nix-quick-install-action/blob/d96491740839e162d9448daa5446f14dd88a52b2/read-nix-config-from-flake.sh#L17-L29
+          flake_file: "${{ github.workspace }}/flake.nix"
+      - name: Restore and cache Nix store
+        # Don't restore/cache when running locally.
+        # https://github.com/nektos/act#skipping-steps
+        if: "${{ !env.ACT }}"
+        uses: nix-community/cache-nix-action@v4
+        with:
+          key: "cache-${{ matrix.system }}-${{ hashFiles('**/*.nix', 'flake.lock') }}"
+          restore-keys: |
+            cache-${{ matrix.system }}-
+          purge-caches: true
+          purge-created: true
+          purge-key: "cache-${{ matrix.system }}-"
+      # Build the test driver for debugging purposes.  To run the driver, enter
+      # the `act` container (`docker exec act-<...> bash`) and execute
+      # `./driver/bin/nixos-test-driver -I`.
+      - name: Build NixOS test driver
+        # Only build when running locally.
+        # https://github.com/nektos/act#skipping-steps
+        if: "${{ env.ACT }}"
+        # Not all checks have a `driver` attribute; ignore errors.
+        run: "nix build -o driver -L '.#checks.${{ matrix.system }}.${{ matrix.check }}.driver' 2>/dev/null || :"
+      - name: Run flake check
+        run: "nix build -L '.#checks.${{ matrix.system }}.${{ matrix.check }}'"

.gitignore

@@ -0,0 +1,16 @@
+# Script rendered from `common/anything-sync-daemon.in`
+common/anything-sync-daemon
+
+# `nix build` and friends' output links
+result
+result-*
+repl-result-*
+
+# From using `nixos-test-driver`
+.nixos-test-history
+
+# Disk images from using `nixos-shell`
+*.qcow2
+
+# Vagrant state
+/.vagrant/

CHANGELOG.md

@@ -0,0 +1,87 @@
+# Changelog for `anything-sync-daemon`
+
+All notable changes to this project will be documented in this file.
+
+## [Unreleased]
+
+### Added
+
+- Support running as an unprivileged user.
+- Document usage in [a Markdown file](/USAGE.md) readable in GitHub's web UI.
+- Document significant environment variables in the manual page.
+- Add [a Nix derivation](/nix/packages.nix) for `anything-sync-daemon`.
+- Add [NixOS modules](/nix/nixos-modules.nix) for managing
+  `anything-sync-daemon`.
+- Support OverlayFS when running as an unprivileged user by managing mounts
+  through the new [`asd-mount-helper`](/common/asd-mount-helper) script.
+  For this to work, the user in question must be able to run `asd-mount-helper`
+  as root via `sudo` without entering a password.
+- Install the example systemd unit files to the `/user/` unit file hierarchy
+  when executing the `install-systemd` Make target.
+
+### Fixed
+
+- Only unmount bind mounts if they have the expected source, and only unmount
+  volatile paths if they are OverlayFS mountpoints.
+- Ensure that `asd.conf`'s parent directory exists before attempt to copy
+  `asd.conf` into it.
+- Deduplicate sync targets (entries in the `WHATTOSYNC` array).  This means
+  testing for string equality on said targets' canonicalized representations,
+  as determined by `realpath -m` or `readlink -m`, if those commands are
+  available and respect the `-m` ("canonicalize missing") option, or otherwise
+  by a native shell routine that simply removes trailing slashes.
+  `anything-sync-daemon` issues a non-fatal warning if it detects duplicate
+  sync targets.
+- Ensure that all intended mount points are mounted as expected when performing
+  the "ungraceful state" check.  This means checking that bind mounts exist and
+  have the expected source (e.g. `/foo/bar/.baz-backup_asd-old` has the source
+  `/foo/bar/baz`) and that (when `USE_OVERLAYFS` is in effect) overlay mount
+  targets have the same mount options as their mount sources.  This makes it
+  possible for `asd` to recover from partial mount states -- that is, to
+  sensibly re-mount intended mount points before performing re-synchronization.
+- In the "ungraceful state" check, only unmount intended mountpoints if they
+  are mounted as expected (in the manner described immediately above).  This
+  means that `asd` is no longer liable to unmount other filesystems that may
+  happen to be mounted in the relevant locations.
+- Install the `asd.conf` parent directory before attempting to copy `asd.conf`
+  into place.
+
+### Changed
+
+- **BREAKING CHANGE**: Load `asd.conf` in a subprocess. This is a breaking
+  change for configurations that attempt to inspect or modify the
+  `anything-sync-daemon` script's internal state.
+- **BREAKING CHANGE**: Enforce a timeout of configurable
+  duration on loading `asd.conf` operation.  This is a breaking change for
+  configurations that do not load within said timeout, though this can be
+  addressed by specifying a longer timeout in the `ASDCONFTIMEOUT` environment
+  variable.
+- **BREAKING CHANGE**: Use the portable shebang `#!/usr/bin/env bash` rather
+  than using `#!/bin/bash`.  This is a breaking change for any setups that (a)
+  have a `bash` under an entry in `PATH` that takes precedence over
+  `/bin/bash`, and (b) are compatible with `/bin/bash` but not the
+  higher-precedence `bash`.  It is also incompatible with setups that lack
+  `/bin` in `PATH`.
+- Respect [the `NO_COLOR` environment variable](https://no-color.org); that is,
+  do not colorize output if `NO_COLOR` is set to a non-empty value.
+- **BREAKING CHANGE**: Attempt to obtain an exclusive lock in-process (via
+  `flock -n <file-descriptor>`) rather than by re-executing when the `FLOCKER`
+  environment variable is unset or empty.  This breaks setups that bypass
+  locking by setting `FLOCKER` to a non-empty value.
+- **BREAKING CHANGE**: detect the presence of `asd.service` by running
+  `systemd list-unit-files asd.service`, in addition to checking for the unit
+  file at `/usr/lib/systemd/system/asd.service` (or, if running `asd` as a
+  non-root user, `/usr/lib/systemd/user/asd.service`).  This makes `asd` liable
+  to conclude that it is being run under systemd when previously it would have
+  concluded otherwise.
+- **BREAKING CHANGE**: detect the `INVOCATION_ID` environment variable and, if
+  it is present, determine the name of the corresponding systemd service, if
+  one exists.  If a service is detected, assume that `asd` is running under
+  systemd.  As with the item just above, this makes `asd` liable to conclude
+  that it is being run under systemd when previously it would have concluded
+  otherwise.
+
+### Removed
+
+- Removed the `asd.conf` "syntax check".  Now, `anything-sync-daemon` accepts
+  any valid Bash code in `asd.conf`.

INSTALL

@@ -1,8 +1,16 @@
 DEPENDENCIES
+ awk
  bash >=4.0
+ coreutils or busybox
  cron (only if using upstart)
  find
+ grep
+ procps (procps-ng)
+ pv
  rsync
+ tar
+ utillinux
+ zstd
 
  a supported init system
  -systemd
@@ -14,21 +22,22 @@ versions of this software before install updates. Internal changes are
 always possible from one release to another.
 
 BUILDING
- Setup the via a make.
+ Setup the script via a make.
 
  $ make
 
  Running a `make install-xxx` as root will distribute the files to the file-
- system.  Most users will want to select from three options depending on target
- init system (do not run both)!
+ system.  Most users will want to select from one or the other of the following
+ options depending on target init system (do not run both)!
 
   # make install-systemd-all
   # make install-upstart-all
 
  As of v3.13, the Makefiles for systemd-native distros such as Arch, Exherbo,
- and Fedora, no longer install the deprecated cron script using a systemd
- timer instead. Users may override and install the deprecated cron script by
- running make with `install-with-cron` instead  of `install` at this point.
+ and Fedora, no longer install the deprecated cron script, and instead install
+ a systemd timer unit. Users may override and install the deprecated cron
+ script by running make with `install-with-cron` instead  of `install` at this
+ point.
 
 USE A DISTRO PROVIDED PACKAGE
  ARCH LINUX
@@ -38,3 +47,10 @@ USE A DISTRO PROVIDED PACKAGE
  CHAKRA
  Chakra users may build the package directly with the provided PKGBUILD:
  http://chakraos.org/ccr/packages.php?ID=3750
+
+ NIX/NIXOS
+ anything-sync-daemon provides a Nix flake,
+ `github:graysky2/anything-sync-daemon`.  This flake's outputs include the
+ `anything-sync-daemon` package (`packages.<system>.anything-sync-daemon`) and
+ the `anything-sync-daemon` NixOS module (`nixosModules.anything-sync-daemon`).
+ See `doc/nixos-modules.md` for available options.

Makefile

@@ -5,6 +5,8 @@ PREFIX ?= /usr
 CONFDIR = /etc
 CRONDIR = /etc/cron.hourly
 INITDIR_SYSTEMD = /usr/lib/systemd/system
+INITDIR_SYSTEMD_SYSTEM = $(INITDIR_SYSTEMD)
+INITDIR_SYSTEMD_USER = $(dir $(INITDIR_SYSTEMD))/user
 INITDIR_UPSTART = /etc/init.d
 BINDIR = $(PREFIX)/bin
 DOCDIR = $(PREFIX)/share/doc/$(PN)
@@ -15,6 +17,7 @@ BSHDIR = $(PREFIX)/share/bash-completion/completions
 # set to anything except 0 to enable manpage compression
 COMPRESS_MAN = 1
 
+PANDOC = pandoc
 RM = rm
 SED = sed
 INSTALL = install -p
@@ -29,6 +32,9 @@ common/$(PN): Makefile common/$(PN).in
 	$(Q)echo -e '\033[1;32mSetting version\033[0m'
 	$(Q)$(SED) 's/@VERSION@/'$(VERSION)'/' common/$(PN).in > common/$(PN)
 
+doc/asd.1: USAGE.md
+	$(PANDOC) --standalone --from markdown+definition_lists+pandoc_title_block --to man -o $@ $<
+
 help: install
 
 stop-asd:
@@ -45,19 +51,21 @@ install-bin: stop-asd disable-systemd common/$(PN)
 	$(Q)echo -e '\033[1;32mInstalling main script...\033[0m'
 	$(INSTALL_DIR) "$(DESTDIR)$(BINDIR)"
 	$(INSTALL_PROGRAM) common/$(PN) "$(DESTDIR)$(BINDIR)/$(PN)"
+	$(INSTALL_PROGRAM) common/asd-mount-helper "$(DESTDIR)$(BINDIR)/asd-mount-helper"
 	ln -sf $(PN) "$(DESTDIR)$(BINDIR)/asd"
+	$(INSTALL_DIR) "$(DESTDIR)$(CONFDIR)"
 	cp -n common/asd.conf "$(DESTDIR)$(CONFDIR)/asd.conf"
 	$(INSTALL_DIR) "$(DESTDIR)$(ZSHDIR)"
 	$(INSTALL_DATA) common/zsh-completion "$(DESTDIR)/$(ZSHDIR)/_asd"
 	$(INSTALL_DIR) "$(DESTDIR)$(BSHDIR)"
 	$(INSTALL_DATA) common/bash-completion "$(DESTDIR)/$(BSHDIR)/asd"
 
-install-man:
+install-man: doc/asd.1
 	$(Q)echo -e '\033[1;32mInstalling manpage...\033[0m'
 	$(INSTALL_DIR) "$(DESTDIR)$(MANDIR)"
 	$(INSTALL_DATA) doc/asd.1 "$(DESTDIR)$(MANDIR)/asd.1"
 ifneq ($(COMPRESS_MAN),0)
-	gzip -9 "$(DESTDIR)$(MANDIR)/asd.1"
+	gzip -f -9 "$(DESTDIR)$(MANDIR)/asd.1"
 	ln -sf asd.1.gz "$(DESTDIR)$(MANDIR)/$(PN).1.gz"
 else
 	ln -sf asd.1 "$(DESTDIR)$(MANDIR)/$(PN).1"
@@ -71,10 +79,14 @@ install-cron:
 install-systemd:
 	$(Q)echo -e '\033[1;32mInstalling systemd files...\033[0m'
 	$(INSTALL_DIR) "$(DESTDIR)$(CONFDIR)"
-	$(INSTALL_DIR) "$(DESTDIR)$(INITDIR_SYSTEMD)"
-	$(INSTALL_DATA) init/asd.service "$(DESTDIR)$(INITDIR_SYSTEMD)/asd.service"
-	$(INSTALL_DATA) init/asd-resync.service "$(DESTDIR)$(INITDIR_SYSTEMD)/asd-resync.service"
-	$(INSTALL_DATA) init/asd-resync.timer "$(DESTDIR)$(INITDIR_SYSTEMD)/asd-resync.timer"
+	$(INSTALL_DIR) "$(DESTDIR)$(INITDIR_SYSTEMD_SYSTEM)"
+	$(INSTALL_DATA) init/asd.service "$(DESTDIR)$(INITDIR_SYSTEMD_SYSTEM)/asd.service"
+	$(INSTALL_DATA) init/asd-resync.service "$(DESTDIR)$(INITDIR_SYSTEMD_SYSTEM)/asd-resync.service"
+	$(INSTALL_DATA) init/asd-resync.timer "$(DESTDIR)$(INITDIR_SYSTEMD_SYSTEM)/asd-resync.timer"
+	$(INSTALL_DIR) "$(DESTDIR)$(INITDIR_SYSTEMD_USER)"
+	$(INSTALL_DATA) init/asd.service "$(DESTDIR)$(INITDIR_SYSTEMD_USER)/asd.service"
+	$(INSTALL_DATA) init/asd-resync.service "$(DESTDIR)$(INITDIR_SYSTEMD_USER)/asd-resync.service"
+	$(INSTALL_DATA) init/asd-resync.timer "$(DESTDIR)$(INITDIR_SYSTEMD_USER)/asd-resync.timer"
 
 install-upstart:
 	$(Q)echo -e '\033[1;32mInstalling upstart files...\033[0m'
@@ -97,6 +109,7 @@ install:
 uninstall-bin:
 	$(RM) "$(DESTDIR)$(BINDIR)/$(PN)"
 	$(RM) "$(DESTDIR)$(BINDIR)/asd"
+	$(RM) "$(DESTDIR)/$(BINDIR)/asd-mount-helper"
 	$(RM) "$(DESTDIR)/$(ZSHDIR)/_asd"
 	$(RM) "$(DESTDIR)/$(BSHDIR)/asd"
 
@@ -111,9 +124,12 @@ uninstall-cron:
 
 uninstall-systemd:
 	$(RM) "$(DESTDIR)$(CONFDIR)/asd.conf"
-	$(RM) "$(DESTDIR)$(INITDIR_SYSTEMD)/asd.service"
-	$(RM) "$(DESTDIR)$(INITDIR_SYSTEMD)/asd-resync.service"
-	$(RM) "$(DESTDIR)$(INITDIR_SYSTEMD)/asd-resync.timer"
+	$(RM) "$(DESTDIR)$(INITDIR_SYSTEMD_SYSTEM)/asd.service"
+	$(RM) "$(DESTDIR)$(INITDIR_SYSTEMD_SYSTEM)/asd-resync.service"
+	$(RM) "$(DESTDIR)$(INITDIR_SYSTEMD_SYSTEM)/asd-resync.timer"
+	$(RM) "$(DESTDIR)$(INITDIR_SYSTEMD_USER)/asd.service"
+	$(RM) "$(DESTDIR)$(INITDIR_SYSTEMD_USER)/asd-resync.service"
+	$(RM) "$(DESTDIR)$(INITDIR_SYSTEMD_USER)/asd-resync.timer"
 
 uninstall-upstart:
 	$(RM) "$(DESTDIR)$(CONFDIR)/asd.conf"

README.md

@@ -1,11 +1,22 @@
 # Anything-sync-daemon
-Anything-sync-daemon (asd) is a tiny pseudo-daemon designed to manage user defined dirs in tmpfs and to periodically sync back to the physical disc (HDD/SSD). This is accomplished via a symlinking step and an innovative use of rsync to maintain back-up and synchronization between the two. One of the major design goals of asd is a completely transparent user experience.
+
+Anything-sync-daemon (asd) is a tiny pseudo-daemon designed to manage user
+defined dirs in tmpfs and to periodically sync back to the physical disc
+(HDD/SSD). This is accomplished via a bind-mounting step and an innovative use
+of rsync to maintain back-up and synchronization between the two. One of the
+major design goals of asd is a completely transparent user experience.
 
 ## Documentation
-Consult the man page or the wiki page: https://wiki.archlinux.org/index.php/Anything-sync-daemon
+
+Consult the `asd(1)` man page (available [here](/USAGE.md)) or
+[the Arch Linux `asd` wiki page](https://wiki.archlinux.org/index.php/Anything-sync-daemon).
 
 ## Installation from Source
-To build from source, see the included INSTALL text document.
+
+To build from source, see [the included INSTALL](/INSTALL) text document.
 
 ## Installation from Distro Packages
-* ![logo](http://www.monitorix.org/imgs/archlinux.png "arch logo")Arch: in the [AUR](https://aur.archlinux.org/packages/anything-sync-daemon).
+
+- ![logo](http://www.monitorix.org/imgs/archlinux.png "arch logo")Arch: in the [AUR](https://aur.archlinux.org/packages/anything-sync-daemon).
+
+## Installation with the [Nix package manager](https://nixos.org)