dracut: initramfs not compressed, somehow

[Artoria2e5]

Bug description

/boot/initramfs-6.7.5-aosc-main.img is not a compressed file, despite the intentions of our dracut.conf clearly asking for zstd.

Steps to reproduce

file /boot/initramfs-6.7.5-aosc-main.img says /boot/initramfs-6.7.5-aosc-main.img: ASCII cpio archive (SVR4 with no CRC)

binwalk /boot/initramfs-6.7.5-aosc-main.img reveals that it's actually a bunch of cpios, some carrying xz-compressed firmware, others carrying zst-compressed modules. Surely we can improve the compression somewhat by going solid: don't compress the individual files, compress the whole thing.

Update channel(s)

stable

Package and version

Package: dracut
Status: install ok installed
Section: utils
Installed-Size: 2924
Maintainer: AOSC OS Maintainers <maintainers@aosc.io>
Architecture: amd64
Version: 059-5
Depends: arping (>= 2.20-1), coreutils (>= 9.4), cpio (>= 2.13-5), gawk (>= 5.2.1), kbd (>= 2.5.1), kmod (>= 31), nfs-utils (>= 2.6.2-1), util-linux (>= 2.39.3-1), systemd (>= 1:254.6-3), bash (>= 5.2.21), parallel (>= 20191122), isomd5sum (>= 1.2.3)
Conffiles:
 /etc/dracut.conf 9c12dd66353f9ea3a6e835c8f849d4bc
Description: Generic, modular, cross-distribution initramfs generation tool
X-AOSC-ACBS-Version: 20240207.1
X-AOSC-Commit: 6cc0c6d305
X-AOSC-Packager: AOSC OS Maintainers <maintainers@aosc.io>
X-AOSC-Autobuild4-Version: 4.0.23

While we're at it: dracut has its own opinion on zstd flags when you just write compress=zstd; see https://github.com/dracutdevs/dracut/blob/5d2bda46f4e75e85445ee4d3bd3f68bf966287b9/dracut.sh#L2412C1-L2414C11

[Artoria2e5]

I am an idiot. skipcpio reveals the real file is actually zst. Binwalk just can't understand zst, and because zst finds much of the data incompressible, it ends up preserving a lot of cpio signatures to be found. We should have a separate discussion about the

Surely we can improve the compression somewhat by going solid: don't compress the individual files, compress the whole thing.

part later, but dracut is built right.

For aosc 675, the compression currently does:

-rw-r--r-- 1 a2 users 239912448 May 15 16:19 675real.img
-rw-r--r-- 1 a2 users 171053780 May 15 16:19 675real.img.zst

which is less than satisfactory, but not nothing.

[Artoria2e5]

So I experimented a bit with decompressing before putting them into initrd. The short story is that there appears to be 1.5 MB of saving, which isn't much:

drwxr-xr-x 1 root root        126 May 15 16:26 675
-rw-r--r-- 1 root root  527473132 May 15 16:35 675-decomp.cpio
-rw-r--r-- 1 a2   users 169533002 May 15 16:35 675-decomp.cpio.zst
-rw-r--r-- 1 a2   users 151227720 May 15 16:35 675-decomp.cpio.zst.15
-rw-r--r-- 1 a2   users 239912448 May 15 16:19 675real.img
-rw-r--r-- 1 a2   users 171053780 May 15 16:19 675real.img.zst

(though look at the 151M from zstd -15!)

• I expanded 675real.img with bsdtar into 675/.
• in 675/usr/lib/firmware, I ran sudo find . -name '*.xz' -exec xz -d {} \;
• in 675/usr/lib/modules, I ran sudo find . -name '*.zst' -exec zstd -d --rm {} \;
• in 675/, I ran sudo bsdtar --format cpio -cf ../675-decomp.cpio * to repack.

Then to produce the two compressed files:

zstd -15 -T0 675-decomp.cpio -o 675-decomp.cpio.zst.15
zstd -T0 675-decomp.cpio -o 675-decomp.cpio.zst

---

Since -15 seems to be responsible for more saving in the unpacked case, I experimented with doing -15 on the original zst. This time it fails to produce the expected saving:

-rw-r--r-- 1 a2   users 166652752 May 15 16:19 675real.img.zst15

So unpacking does help with compression, but zstd needs to be working hard enough for that help to matter. I doubt we want to slow down initramfs generation for that.