swupd os-install failed to verify data if run within debian OS

[axet]

Describe the bug
swupd os-install failed with:

Error: File content hash mismatch for /clearlinux/var/lib/swupd/cache/https___download.clearlinux.org_update/staged/d8a2055000d9ff84e01c64622fe3ead8c324164273abce439513803345ffc4e5 (bad server data?)

I'm running Debian, with manually compiled swupd from git source. os-install failed only if os-core-update specified. Command os-install works with "os-core" bundle only and many others.

To Reproduce

• Install debian
• compile swupd
• bootstrap clear linux into folder.

sudo ./swupd os-install -n -I -F staging -u https://download.clearlinux.org/update/ /clearlinux --bundles=os-core-update --statedir=/clearlinux/var/lib/swupd

I've tried different version (specified with -V 33000) nothing helping. Something wrong with "os-core-update" bundle. If I recompile swupd with possibility to ignore verify_file errors, OS installed and executed succefully from booth: chroot and real boot after swaping folders: / (debian) <-> /clearlinux (clear linux).

[paradigm]

Bedrock Linux users with Clear Linux strata are hit by a very similar error message. It might be for the same reason.

greping through the manifests:

# grep d8a2055000d9ff84e01c64622fe3ead8c324164273abce439513803345ffc4e5 var/lib/swupd/manifest/*/* | head -n1
var/lib/swupd/manifest/34080/Manifest.os-core-webproxy:F...     d8a2055000d9ff84e01c64622fe3ead8c324164273abce439513803345ffc4e5        33900   /usr/libexec/dbus-daemon-launch-helper

It looks like the file causing the issue is /usr/libexec/dbus-daemon-launch-helper.

Per https://github.com/clearlinux/swupd-server/wiki/Hash-calculation that long hex string is a hash of both a file's content and its metadata. On a Clear Linux system, that metadata is:

# stat /usr/libexec/dbus-daemon-launch-helper | grep '^Access: ('
Access: (4750/-rwsr-x---)  Uid: (    0/    root)   Gid: (   81/messagebus)

However, on my Bedrock system, when swupd tried to stage the contents it ended up creating:

# stat var/lib/swupd/staged/d8a2055000d9ff84e01c64622fe3ead8c324164273abce439513803345ffc4e5 | grep '^Access: ('
Access: (4750/-rwsr-x---)  Uid: (    0/    root)   Gid: (   11/messagebus)

Note the GID mismatch. On Bedrock Linux systems, /etc/group is shared between strata. On my Bedrock system, some other stratum created GID 81 first and associated it with a different name. When Clear Linux tried to use it, it didn't match Clear's very carefully checked expectations.

Does your swupd-on-Debian setup share /etc/group across both Debian and Clear Linux software? Maybe Debian created a /etc/group line which mapped 81 to something other than messagebus (e.g. dbus). If so, that's likely the underlying issue. In that case you can probably change your /etc/group to map messagebus to 81 and re-run swupd. Take care, though, as doing this could have permissions side effects for Debian's part of the system.

In theory, swupd might be able to better support running on non-Clear Linux systems by translating the system's /etc/group to some swupd-internal /etc/group via the group name whenever it calculates hashes. However, it's not clear to me that our niche use case is sufficient to justify the resulting complexity for the Clear Linux developers.

[axet]

I do not share groups. Clearlinux installed into clean chroot folder. On my system it crash before staged data. Since debian has no swupd I wrote simple script which clone git swup repo and compiles it. Then after running swupd into /clearlinux it stops before creating any /etc, /var ... directories (only /var/lib/swupd/cache exists).

Script is here: https://gitlab.com/axet/homebin/-/blob/debian/clr-bootstrap

simple version is:

1. git clone https://github.com/clearlinux/swupd-client /clearlinux/tmp/swupd
2. make /clearlinux/tmp/swupd
3. sudo /clearlinux/tmp/swupd/swupd /clearlinux

result folders are:

/clearlinux/tmp/...
/clearlinux/var/lib/swupd/{ 3rd-party, bundles, cache, swupd_lock, telemetry }

no swupd/swupd folder, as in your example. But the issue looks have the same roots.