book: Move /opt redirection guidance to building

This is where I expected to find it.

Signed-off-by: Colin Walters <[email protected]>

Author: cgwalters

docs/.gitignore

@@ -1 +1,2 @@
 book
+mermaid*.js

docs/src/building/guidance.md

@@ -138,3 +138,35 @@ for example, although this can run afoul of SELinux labeling.
 
 There is a dedicated document for [secrets](secrets.md),
 which is a special case of configuration.
+
+## Handling read-only vs writable locations
+
+The high level pattern for bootc systems is summarized again
+this way:
+
+- Put read-only data and executables in `/usr`
+- Put configuration files in `/usr` (if they're static), or `/etc` if they need to be machine-local
+- Put "data" (log files, databases, etc.) underneath `/var`
+
+However, some software installs to `/opt/examplepkg` or another
+location outside of `/usr`, and may include all three types of data
+undernath its single toplevel directory.  For example, it
+may write log files to `/opt/examplepkg/logs`.  A simple way to handle
+this is to change the directories that need to be writble to symbolic links
+to `/var`:
+
+```dockerfile
+RUN apt|dnf install examplepkg && \
+    mv /opt/examplepkg/logs && /var/log/examplepkg && \
+    ln -sr /opt/examplepkg/logs /var/log/examplepkg
+```
+
+The [Fedora/CentOS bootc puppet example](https://gitlab.com/fedora/bootc/examples/-/tree/main/opt-puppet)
+is one instance of this.
+
+Another option is to configure the systemd unit launching the service to do these mounts
+dynamically via e.g.
+
+```
+BindPaths=/var/log/exampleapp:/opt/exampleapp/logs
+```

docs/src/filesystem.md

@@ -121,18 +121,11 @@ Besides those, for other toplevel directories such as `/usr` `/opt`, they will b
 In the default suggested model of using composefs (per above) the `/opt` directory will be read-only, alongside
 other toplevels such as `/usr`.
 
-Some software expects to be able to write to its own directory in `/opt/exampleapp`.  A common
-pattern is to use a symbolic link to redirect to e.g. `/var` for things like log files:
+Some software (especially "3rd party" deb/rpm packages) expect to be able to write to
+a subdirectory of `/opt` such as `/opt/examplepkg`.
 
-```
-RUN rmdir /opt/exampleapp/logs && ln -sr /var/log/exampleapp /opt/exampleapp/logs
-```
-
-Another option is to configure the systemd unit launching the service to do these mounts
-dynamically via e.g.
-```
-BindPaths=/var/log/exampleapp:/opt/exampleapp/logs
-```
+See [building images](building/guidance.md) for recommendations on how to build
+container images and adjust the filesystem for cases like this.
 
 #### Enabling transient root
 
@@ -145,4 +138,4 @@ transient = true
 ```
 
 option in `prepare-root.conf`.  In particular this will allow software to write (transiently) to `/opt`,
-with symlinks to `/var` for content that should persist.
+with symlinks to `/var` for content that should persist.