Apply disk or filesystem customization from embedded image customization

[alexlarsson]

This allows bootc images to embed a file in /usr/lib/bootc-image-builder in the image called config.json or config.toml, which specify more detailed requirements for the partitioning. From this (which is in the full blueprint format) we extract (only) the disk and/or filesystem customization part.

This is useful to either add extra partitions (like a separate /var), or to override details of the normal partitions (like uuids, labels, etc).

This is discussed in bootc-dev/bootc#926

I used the entire blueprint format so that we can reuse existing docs, and so that we can later use more customization fields if needed.

Note: This (mostly trivially) conflicts with #928

[achilleas-k]

Thank you, this is great!!

I think this change is important enough to warrant a new test container in CI.
https://github.com/osbuild/bootc-image-builder/blob/main/test/test_manifest.py
Perhaps not a full build, but at least a manifest generation and inspection test.

[mvo5]

Thank you! This is a really nice feature, we should probably add it to the README.md as well that this is now possible.

I also wonder if we should error (or warn) if there are most customizations embedded than the filesystem/disk ones. Mostly to avoid surprises when people might think adding extra stuff would work when in fact it does not. Obviously followup material.

I also included tiny suggestions for the go-code but and would love some tests but all of those I can do in a followup, I don't want to slow this (excellent) work down.

[alexlarsson]

I added some tests

[alexlarsson]

Note: The tests fail to set part_type because we lack this blueprint commit: osbuild/blueprint@bb3eead
which adds handling of PartType in the Convert() function.

Once we update the blueprint version we should extend these tests to include this, and the new options.

[alexlarsson]

I also wonder if we should error (or warn) if there are most customizations embedded than the filesystem/disk ones. Mostly to avoid surprises when people might think adding extra stuff would work when in fact it does not. Obviously followup material.

Yeah, that is probably a good idea.

[mvo5]

Thank you, this is very nice!

[alexlarsson]

Some test is failing with:

org.osbuild.bootc.install-to-filesystem: fab7420133de53329157eca54e58fbe60f081859afc46b22c14d9231befaa71e {
  "kernel-args": [
    "rw",
    "console=tty0",
    "console=ttyS0",
    "systemd.journald.forward_to_console=1"
  ],
  "target-imgref": "quay.io/fedora/fedora-bootc:43"
}
device/disk (org.osbuild.loopback): loop0 acquired (locked: False)
mount/- (org.osbuild.btrfs): mounting /dev/loop0p4 -> /store/tmp/buildroot-tmp-7wl3rl6k/mounts/
mount/boot (org.osbuild.xfs): mounting /dev/loop0p3 -> /store/tmp/buildroot-tmp-7wl3rl6k/mounts/boot
mount/boot-efi (org.osbuild.fat): mounting /dev/loop0p2 -> /store/tmp/buildroot-tmp-7wl3rl6k/mounts/boot/efi
Mount transient overlayfs for /etc/containers
Creating bind mount for run/osbuild/containers
Installing image: docker://quay.io/fedora/fedora-bootc:43
Initializing ostree layout
ERROR Installing to filesystem: Creating ostree deployment: invalid reference format
Traceback (most recent call last):

Doesn't seem related to this PR, but I don't understand how the reference format is invalid in the first place?

[mvo5]

Some test is failing with:
[..]
Doesn't seem related to this PR, but I don't understand how the reference format is invalid in the first place?

It is not related, I can reproduce this locally with:

$ git rev-parse HEAD
66049b592e1e1b8ecbce4d0b06fd6dbb2fead663
$ sudo pytest -s -vv ./test/test_build_disk.py::test_image_boots[container_ref=quay.io/fedora/fedora-bootc:43,disk_config=btrfs,image=raw,rootfs=btrfs,use_librepo=True]
...

and get the same error. So I guess we need to investigate the bootc-fedora:43 container, but we can skip the test for now to unblock you, this fedora:43 issue is outside of our controll.

[alexlarsson]

The other failure seems to be:

Resolved "registry" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
Trying to pull docker.io/library/registry:2...
time="2025-05-21T09:37:53Z" level=warning msg="Failed, retrying in 1s ... (1/3). Error: initializing source docker://registry:2: reading manifest 2 in docker.io/library/registry: toomanyrequests: You have reached your unauthenticated pull rate limit. https://www.docker.com/increase-rate-limit"
time="2025-05-21T09:38:25Z" level=warning msg="Failed, retrying in 1s ... (2/3). Error: initializing source docker://registry:2: reading manifest 2 in docker.io/library/registry: toomanyrequests: You have reached your unauthenticated pull rate limit. https://www.docker.com/increase-rate-limit"
time="2025-05-21T09:38:56Z" level=warning msg="Failed, retrying in 1s ... (3/3). Error: initializing source docker://registry:2: reading manifest 2 in docker.io/library/registry: toomanyrequests: You have reached your unauthenticated pull rate limit. https://www.docker.com/increase-rate-limit"
Error: initializing source docker://registry:2: reading manifest 2 in docker.io/library/registry: toomanyrequests: You have reached your unauthenticated pull rate limit. https://www.docker.com/increase-rate-limit
_ ERROR at setup of test_image_boots[container_ref=quay.io/fedora/fedora-bootc:42,image=qcow2,rootfs=btrfs,use_librepo=True] _

shared_tmpdir = PosixPath('/var/tmp/bib-tests/shared1')
request = <SubRequest 'registry_conf' for <Function test_image_is_generated[container_ref=quay.io/centos-bootc/centos-bootc:stream9,image=qcow2+raw+vmdk+vhd+gce,use_librepo=True,use_terminal=True]>>

[mvo5]

Thank you!

[alexlarsson]

I'm now seeing this failure:

org.osbuild.mkfs.xfs: da281d9c5d46b298ea9d4b0c174b3433a37d5881dbc378c6555c27d7994166e6 {
  "uuid": "55770db7-42a4-4002-b4da-eb41bc89bcaf",
  "label": "boot"
}
device/device (org.osbuild.loopback): loop0 acquired (locked: True)
mkfs.xfs: error - cannot set blocksize 512 on block device /dev/loop0: Inappropriate ioctl for device
Traceback (most recent call last):
  File "/run/osbuild/bin/org.osbuild.mkfs.xfs", line 24, in <module>
    ret = main(args["devices"], args["options"])
  File "/run/osbuild/bin/org.osbuild.mkfs.xfs", line 18, in main
    subprocess.run(["mkfs.xfs", "-m", f"uuid={uuid}"] + opts + [device],
  File "/usr/lib64/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['mkfs.xfs', '-m', 'uuid=55770db7-42a4-4002-b4da-eb41bc89bcaf', '-L', 'boot', '/dev/loop0']' returned non-zero exit status 1.

⏱  Duration: 3s
manifest - failed
Failed
2025/05/21 16:27:58 error: cannot run osbuild: error running osbuild: exit status 1

Which seem unrelated.

I also get from _ test_image_build_without_se_linux_denial:

E       AssertionError: denials in log May 21 16:36:37 bc697306-b54b-4422-be0c-3c1693336c71 podman[18309]: 2025-05-21 16:36:37.304979741 +0000 UTC m=+0.040396154 volume create fceff15cb99b203c409e328dd778a5ab8f136fd248af96163acc1750b90887fe
...
E         May 21 16:37:43 bc697306-b54b-4422-be0c-3c1693336c71 audit[19017]: AVC avc:  denied  { nnp_transition nosuid_transition } for  pid=19017 comm="bootc" scontext=system_u:system_r:install_t:s0:c772,c968 tcontext=system_u:system_r:container_runtime_t:s0:c772,c968 tclass=process2 permissive=0

This also seems unrelated, as we're not doing anything wrt selinux.

Also, test_manifest_disk_customization_lvm_swap gets:

/var/tmp/bib-tests/test_manifest_disk_customizati3/manifest.json has errors:

.pipelines[1].stages[10].devices.rootlv:
  could not find schema information for 'org.osbuild.lvm2.lv'
...

Which seems to indicate that the osbuild-lvm2 rpm is not installed.

[achilleas-k]

Thanks for all of this. It's great to finally have this feature!
LGTM

[cgwalters]

Doesn't seem related to this PR, but I don't understand how the reference format is invalid in the first place?

May be related to bootc-dev/bootc#1302