Issues with stateless-system feature in custom Torizon-based distro

[threexc]

Hi,

We are building a custom distribution for an x86-64 platform, where our target image requires the torizon-docker.bb recipe, while our distro conf requires common-torizon.inc and sota.conf.inc. We also have a custom installer recipe in place so that we can use bootable USB drives to install the OS image to our test targets, but this explicitly avoids including any Torizon-specific content as it's not meant to do anything but install the image of interest to the hardware.

We're interested in using OSTree and the stateless system functionality for updates, however we've run into issues with the /etc/group, /etc/passwd files seemingly not being populated correctly (or being overwritten during the build) sometimes - this is not consistent, but seems to have something to do with the state of tmp/ and/or sstate-cache/ from previous builds. What ends up happening is that the group/passwd files get deployed without any of the user/group details except for the torizon user itself, which means things like dbus and systemd don't come up correctly, preventing the majority of the system from functioning.

I went looking in the git blame/git logs for some details on this but didn't find much about nss_files or stateless_system that was helpful to explain the interactions here.

Is this a phenomenon that's been observed in the past? If so, how can we mitigate it? And if not, what info do we need to provide for further assistance?

[threexc]

@drewmoseley ping

[drewmoseley]

@EdTheBearded is this something you can comment on?

[EdTheBearded]

We don't create or populate those files on the fly, we just maintain those in here https://github.com/torizon/meta-toradex-torizon/tree/scarthgap-7.x.y/files and use them in the final distribution (https://github.com/torizon/meta-toradex-torizon/blob/scarthgap-7.x.y/conf/distro/include/base-distro.inc#L58).

We never faced such an issue, and it's kind of hard to say what's going on. Since the files are practically empty, maybe they're being replaced by some other file someplace else, or being overwritten by some other task. But those are just guesses, hard to say what's going on.