ephemeral: Replace systemd.volatile=overlay with fine-grained mounts #189
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request replaces the broad systemd.volatile=overlay with more granular, fine-grained tmpfs and overlayfs mounts for /var and /etc respectively. This is a well-reasoned change that solves a real-world issue with nested overlayfs when using podman inside the ephemeral VM. The implementation using systemd credentials injected via SMBIOS is clean and effective. The addition of an integration test to verify the new mount layout is excellent. My feedback is minor, focusing on improving code consistency and fixing a small typo in a comment.
cgwalters
force-pushed
the
ephemeral-prodlike
branch
3 times, most recently
from
fe6c2e0 to
b3cd27b
Compare
cgwalters
force-pushed
the
ephemeral-prodlike
branch
from
b3cd27b to
67695e6
Compare
Instead of using systemd.volatile=overlay which overlaid all of / with a single tmpfs-backed overlayfs, set up /etc and /var separately: - /etc: overlayfs with tmpfs upper (transient changes, lost on reboot) - /var: real tmpfs with content copied from image (not overlayfs) The key benefit is that /var is now a real tmpfs, allowing podman to use overlayfs for container storage inside /var/lib/containers. With the old approach, the nested overlayfs caused "too many levels of symbolic links" errors. Implementation: The initramfs units are embedded in a CPIO archive that gets appended to the existing initramfs. This uses the Linux kernel's ability to concatenate multiple CPIO archives. Services running in initramfs (before switch-root): - bcvk-etc-overlay.service: Sets up overlay on /sysroot/etc using a bind-mounted lowerdir to avoid self-referential mount issues on older kernels. Uses index=off,metacopy=off for virtiofs compat. - bcvk-var-ephemeral.service: Copies /sysroot/var to tmpfs and bind mounts it back. - bcvk-copy-units.service: Copies bcvk-journal-stream.service to /run/systemd/system/ for systemd <256 compatibility. The /run tmpfs is preserved across switch-root via MS_MOVE. For systemd 256+, the journal-stream unit is created via SMBIOS credentials (systemd.extra-unit.*). For older versions like CentOS Stream 9 (systemd 252), the unit is copied from the initramfs since credential-based unit creation isn't supported. The execute service target is changed from default.target to multi-user.target with ConditionPathExists=!/etc/initrd-release to ensure it runs after switch-root, not in the initramfs. This is Phase 1 of issue bootc-dev#22, making ephemeral VMs more bootc-like. SELinux is still disabled (selinux=0); Phase 2 will add composefs support to enable proper SELinux labeling. xref: bootc-dev#22 (Phase 1) Assisted-by: OpenCode (Sonnet 4) Signed-off-by: Colin Walters <[email protected]>
cgwalters
force-pushed
the
ephemeral-prodlike
branch
from
67695e6 to
5516e29
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Instead of using systemd.volatile=overlay which overlaid all of / with a single tmpfs-backed overlayfs, set up /etc and /var separately:
The key benefit is that /var is now a real tmpfs, allowing podman to use overlayfs for container storage inside /var/lib/containers. With the old approach, the nested overlayfs caused "too many levels of symbolic links" errors.
Implementation uses systemd credentials to inject units that run in the initramfs before switch-root:
Both units use ConditionPathExists=/etc/initrd-release to only run in the initramfs context.
This is Phase 1 of issue #22, making ephemeral VMs more bootc-like. SELinux is still disabled (selinux=0); Phase 2 will add composefs support to enable proper SELinux labeling.
xref: #22 (Phase 1)
Assisted-by: OpenCode (Sonnet 4)