initramfs-scripts: more elegant `canmount` property handling?

[Harry-Chen]

A debian user reported a bug that datasets with property canomunt=noauto set would also be (or at least tried to be) mounted during boot by initramfs scripts.

This behavior is described in:

zfs/contrib/initramfs/scripts/zfs

Lines 337 to 343 in 1f3444f

	# Skip filesystems with canmount=off. The root fs should not have
	# canmount=off, but ignore it for backwards compatibility just in case.
	if [ "$fs" != "${ZFS_BOOTFS}" ]
	then
	canmount=$(get_fs_value "$fs" canmount)
	[ "$canmount" = "off" ] && return 0
	fi

I understand this logic is mainly for mounting rootfs which has noauto set. Maybe we could avoid mounting other similar datasets, e.g., by comparing the dataset name with the actual ROOT?

[bar-g]

# Skip filesystems with canmount=off. The root fs should not have canmount=off, but ignore it for backwards compatibility just in case.

That comment seems confusing. It rather seems boot environments all get canmount=offnoauto set by default (EDIT: off is for placeholder/inheritance filesystems), and the bootfs exception made in the code is important to allow mounting it anyway if set as ${ZFS_BOOTFS}.

And for the problem at hand it might just be that the [ "$canmount" = "off" ] test should actually rather test if the value is "off" or "noauto" to also skip on non-root, noauto-filesystems.

---

Could there be a more elagant canmount handling?

1. can canmount=yes be used to mark the active boot environment? but there must be ever only max. one enabled to allow selectively mounting only one boot environment
1. could zpool import refuse to mount any / mount if it's already mounted (user forgot the -R option to set an altroot)? i.e. to prevent mounting a foreign rootfs / over the current rootfs / when (force) importing?

(EDIT: see below)

[Harry-Chen]

CC: original authors @rlaager @FransUrbo, and maintainers @behlendorf @tonyhutter. What's your opinion on this?

[rlaager]

It rather seems boot environments all get canmount=off set by default

On my Ubuntu daily driver, which I believe was installed using the installer's ZFS support, I have canmount=on. Looking at the HOWTOs I maintain (yes, I know I'm behind right now), the Ubuntu 22.04 instructions seem to result in the default of canmount=on. On Debian, which is not using zsys, the HOWTO explicitly uses canmount=noauto.

the exception made in the code is important to allow mounting it anyway if set as ${ZFS_BOOTFS}.

I think I would expect canmount=noauto, so the exception is important if this behavior is changed as discussed below.

And for the problem at hand it might just be that the [ "$canmount" = "off" ] test should actually rather test if the value is "off" or "noauto" to also skip on non-root, noauto-filesystems.

I think that is what is being requested in the Debian bug.

That seems reasonable to me. Fundamentally, if the filesystem is noauto, that should mean the filesystem is no(t) auto(matically) mounted.

Of course, it's tricky messing with default behavior like this because you can break people's existing systems.

[intelfx]

I recently wrote a replacement set of (systemd-based) zfs initcpio scripts for Arch, so I gave this problem some thought.

As I understand it, canmount=noauto is used mostly as a hack to allow having multiple OS installations to co-exist on the same pool in different datasets (which is a pretty cool ability of dataset-oriented filesystems; I've been using it on BTRFS previously and I'm using it on ZFS now).

The idea is that canmount=noauto is "like an off, but not off". You apply canmount=noauto to all filesystems in the alternate OS subtrees — the ones that you want to be treated as canmount=off (to prevent userspace from automounting them, because they will overmount the OS you wanted to boot into), but at the same time distinguished from true canmount=off datasets, which exist purely for organizational reasons and are never meant to be mounted at all.

With this strategy, the boot scripts must ignore canmount=noauto (treat canmount=noauto for the subtree you want to boot as canmount=on), but the rest of userspace must NOT ignore canmount=noauto (for all other subtrees).

Does this make sense?

[bar-g]

Seems to make sense, from trying to interpret things, not knowing original intentions.

It's the problem of managing different boot environments.
Made a ghostbsd test install VM and it looks like this:

# zfs get canmount
NAME                                      PROPERTY  VALUE     SOURCE
zroot                                     canmount  on        default
zroot/ROOT                                canmount  on        default
zroot/ROOT/25.02-backup-2025-11-28-11-05  canmount  noauto    local
zroot/ROOT/default                        canmount  noauto    local
zroot/ROOT/default@2025-11-28-11:05:56-0  canmount  -         -
zroot/home                                canmount  on        default
zroot/tmp                                 canmount  on        default
zroot/usr                                 canmount  off       local
zroot/usr/ports                           canmount  on        default
zroot/var                                 canmount  off       local
zroot/var/audit                           canmount  on        default
zroot/var/crash                           canmount  on        default
zroot/var/log                             canmount  on        default
zroot/var/mail                            canmount  on        default
zroot/var/tmp                             canmount  on        default

Not sure why /usr and /var are 'off', maybe because they are just too dangerous to get mounted accidentially. But the whole thing seems flawed with zpool import over-mounting the running system by default instead of shielding it with a default -R altroot.

It's way too difficult and error prone to find a safe way to properly mount an arbitrary zpool.
Maybe?:

# zpool import -N -R /inspect -f zroot
# zfs mount zroot/ROOT/default
# zfs umount -a

---

Another thing that seems wrong could be that instead of only configuring one currently active root or boot environment, zfs requires to shielding off all of them, then select one dynamically for the boot.

Could maybe sub-mounts be defined safely by relative mounting paths, instead?
zroot/home mountpoint ../ROOT/default/home # with 'default' rather called 'active'?

[FransUrbo]

I vaguely remember that when I wrote this almost a hundred years ago :), there was issues booting from an FS that had canmount=off, which is why the canmount=noauto is for - it allows root FS' to boot if you so specify. As in, booting from a snapshot..

But don't quote me on this, it was a VEEEERYYYYY long time ago :D :D. And there's nothing saying that this is still the case (as in, booting from canmount=off), but.. ??

[rlaager]

/usr and /var would just be container datasets

[bar-g]

[EDIT: Configuring classic fstab mounting mused about here would work, yet see proposed solution for zfs auto-mounting, below.]

I'm getting the impression that no single zfs-options based mount config can ever work well for multiple boot environments (i.e. root FS systems) to coexist without conflicts, because each rootfs would really need to be able to have its own config for all its "coupled system state": https://docs.zfsbootmenu.org/en/v3.0.x/general/bootenvs-and-you.html

So, basicly independent boot environments should probably just use standard /etc/fstab configurations, i.e. mountpoint=legacy (https://superuser.com/a/1557822).

For example, mounting (above) separated dataset with /var (kept unchanged accross boot environments, not rolled back with them) may work for BSD (because there /var does not contain any system level "/var/lib" stuff as in linux distros). Similarly the separate /usr dataset with installed user software may work well for BSDs (within same release), because there is no such thing as a "usrmerge" to displace system level software into /usr.

However, when separate datasets are created for /var and /usr in say Debian (https://openzfs.github.io/openzfs-docs/Getting%20Started/Debian/Debian%20Trixie%20Root%20on%20ZFS.html) this seems to go against keeping the "coupled system state" nicely together for boot alternatives and roll-backs. (At the very least, it seeems to require doing a lot of cat-and-mouse hunting for subdirs to separate into proper, boot-environment-independent datasets, if it's at all possible to do cleanly.)

[FransUrbo]

We need to keep in mind WHY we have separate file systems for the operating system in the first place.

"In the old days", hard drives where extremely (!!) expensive and small. They were also prone to failure. This was LOOONG before RAID!! And even longer before Linux :). MANY decades before!! :)

So to maximise the chance to get the system back up, "the bare minimum" (/etc which contained some system binaries and /bin where most of the "boot binaries" were, later /sbin came in) system is needed - this is what we call the "root file system" was on one filesystem (also keep in mind that the root user's home directory was there as well, all mixed in with everything else! :), and the other was split up onto other drives.

So if /usr (which originally hosted the users - usr=user) - or any other FS except the root FS which was kept as small and unchanging (changes, i.e. writes, could create inconsistencies if the machine crashed while something was written to the disk/file system) crashed, you could boot in single user, do the repairs, reboot and all good..

That was the idea, and it had a lot more success than you'd think :). I've saved countless systems that way, before I could afford my first RAID card - the very earliest versions of Linux RAID (md) wasn't very good or stable :D :D. Although, in the early days, I only had one or two drives, so had to do partitioning - a file system was more likely to crash than the disk itself.

But with RAID, and especially something as .. "error correcting" and "fault tolerant" as ZFS, there simply is no need for separate file systems anymore! This is just a .. "left over" from those, dark and rainy days.. Were it actually hailed horizontally more than actually rained :D :D. I'm actually still amazed that that's the default option on most Linux install systems. It shouldn't be imo.

But your point of multi-boot is correct. I do remember making a design decision about that, because I couldn't figure out a way to solve that in a good way. BUT, that canmount=noauto does help in that, as long as you set root=... correctly, AND (!) you have one OS per FS (as in, not split up "like we use(d) to"). That even lets you boot from a snapshot, because the whole OS (including /var, /usr etc etc) is on there/that.

But it still don't quite work .. perfectly if you're booting from several different operating systems (I tried a few versions of *BSDs and Linux'es), but it really didn't work very well. THEN, I should say, might (?) work much better now.

This was my design decision, and I still think that was the right decision. If someone have something better or smarter, that won't break this, then by all means, PR and I'm sure someone will take a look in the next few years :).

[bar-g]

We need to keep in mind WHY we have separate file systems for the operating system in the first place.

The still possible non-raid hardware (failures) seem only one aspect, but just as you say, the OS separation is still very much relevant for other reasons like multi-boot, configuration errors, startup errors, image-updates, roll-backs, ...

After thinking a bit more about canmount=noauto it seems fine to me, given the ZFS auto-mount behaviour.

The proposal in the orig. report above, comparing of dataset name with configured bootfs name, would not seem to help for non-bootfs datasets.

The thing that seems missing seems a way to associate non-bootfs filesystems to just specific boot environments.

Maybe set canmount=zpool/ROOT/DEBIAN? too restrict auto-mounting to bootfs below that path, e.g. zpool/ROOT/DEBIAN/default

[bar-g]

Proposed solutions:

• initramfs-scripts: require canmount=on for non-bootfs #17995
• canmount bootfs:path -- simple, arbitrary, conflictless, multi-OS-boot (shared zpool) layouts #17996

[FransUrbo]

The second patch is never going to happen, sorry..

It would break existing system, cause inconsistencies between installations, and between versions.. That's not where that value should be.. You put that in your boot loaders root=... command line.

The second one, that is there to automount (even though it's set to noauto!) any and all filesystem below the root FS. This is necessary (required) for multi-boots.

In a previous comment, you showed the operating system filesystems on the same level (?) as the root fs:

pool/root
pool/var
pool/usr

However, that check is there to make sure that this works (which, IF (!) you need multiple FS' for this - which I've declared you don't - I think that is a/the bug!) - see line 335:

pool/root
pool/root/var
pool/root/usr

Now, it is not completely clear to my what the Debian GNU/Linux bug report actually mean. Do they mean that the first path example here, or the second, is what they have!?

If the former, then there really IS a bug there somewhere, but then either 1) the FS option is set wrongly (for some reason - bug in the installer or user/admin error!?), or 2) the installer created the FS' wrongly (should be the second example - but it's better not have multiple filesystems in the first place).

But either way, your first patch would stop the second way of doing it working. I would, if I had any voice in this (which I don't), vote no for both patches.