Add a composefs backend #1444

cgwalters · 2025-07-24T13:05:59Z

This adds a new off-by default feature to enable
a new composefs-native backend for bootc. This
is all still a live work in progress, but
we're landing this first tranche of work to help
avoid continual issues with rebasing.

Thanks to everyone who worked on it!

xref #1190

Co-authored-by: John Eckersberg [email protected]
Co-authored-by: Robert Sturla [email protected]
Co-authored-by: Colin Walters [email protected]
Signed-off-by: Pragyan Poudyal [email protected]

gemini-code-assist

Code Review

This pull request introduces composefs as a new backend, which is a significant and extensive change. The new code paths for installation, upgrades, switching, and rollbacks for composefs are well-structured.

My review has identified a few issues, including a potential bug with a hardcoded transport value and an assert! that could lead to a panic. I've also included some suggestions for improving code style and robustness. Given this is a work-in-progress, these findings are not unexpected. Overall, this is a solid foundation for the new composefs functionality.

crates/lib/src/cli.rs

crates/lib/src/deploy.rs

crates/lib/src/cli.rs

crates/lib/src/deploy.rs

crates/lib/src/install.rs

crates/lib/src/status.rs

cgwalters · 2025-08-04T13:06:19Z

I'm going to rebase this. In the interest of preserving history I've saved a copy of the branch as composefs-branch-20250804

cgwalters · 2025-08-05T13:41:31Z

OK at least this passes the existing tests.

cgwalters · 2025-08-07T15:18:55Z

I rebased 🏄 this again (and in the process reworked the composefs karg stuff to use the new kernel_cmdline &str APIs)

cgwalters · 2025-08-18T08:02:09Z

OK I'd like to land #1525 and then rebase again

Johan-Liebert1 · 2025-08-28T11:06:15Z

Rebasing on main ends up with a bunch of conflicts again. I was thinking about moving all composefs related code to separate files/crate to minimize conflicts

cgwalters · 2025-08-28T12:44:00Z

Notes from meeting

Need to rebase (should squash most commits again)
Considering merging mostly as is with lots of followups
Gate code based on feature flag
Trust in automated tests, but also do a lot of manual testing post-merge to be sure it doesn't break the ostree path

cgwalters · 2025-08-28T19:25:25Z

@jeckersb before rebasing this would you mind also saving the existing branch? We've done that for the last few rebases

jeckersb · 2025-08-28T20:19:05Z

Worth noting that I didn't do any of the squashing during the rebase, but at least with it clean we can easily decide which ones to squash out.

Johan-Liebert1 · 2025-08-29T11:34:47Z

I'll be going ahead and rebasing again in prep for #1560

jeckersb · 2025-09-03T21:02:41Z

rebased this again (saved the old one at composefs-backend-2025-09-03) to fix a conflict and pick up #1581

cgwalters · 2025-09-04T12:34:31Z

/gemini review

gemini-code-assist

Code Review

This pull request introduces significant new functionality for composefs support, including installation, upgrades, switching, and rollbacks. The changes are extensive, touching many parts of the codebase to accommodate composefs as an alternative to the existing ostree backend. New modules, data structures, and logic have been added to handle composefs-native systems.

The review has identified a few important issues. There's a high-severity bug where a hardcoded "docker" transport is used, which will prevent switching to images from other transports. Another high-severity issue is that a crucial finalization step (install_finalize) is skipped for composefs installations, potentially leaving them in an inconsistent state. Additionally, there are some medium-severity concerns regarding robustness against non-UTF8 paths and inconsistent use of std::fs versus cap_std for filesystem operations.

Overall, this is a great step towards composefs integration, and addressing these points will improve the correctness and robustness of the new functionality.

crates/lib/src/cli.rs

crates/lib/src/install.rs

crates/lib/src/bootc_composefs/state.rs

crates/lib/src/install.rs

Johan-Liebert1 · 2025-09-09T14:33:22Z

Needs another rebase. Maybe one after #1598 lands, which is required for #1573

Johan-Liebert1 · 2025-09-10T04:01:30Z

Rebased again. Only one conflict found

<<<<<<< HEAD
etc-merge = { path = "../etc-merge" }
=======
bootc-initramfs-setup = { path = "../initramfs" }
>>>>>>> 023be101 (composefs/install: Copy /etc contents to state)

Accepted both, and saved previous branch to composefs-backend-10-09-2025

cgwalters · 2025-09-12T16:20:35Z

OK I rebased this, the conflict was from #1603 mainly. The resolutions were pretty straightforward.

cgwalters · 2025-09-12T16:21:56Z

Previous branch is backed up at https://github.com/bootc-dev/bootc/tree/composefs-backend-2025-09-12

This adds a new off-by default feature to enable a new composefs-native backend for bootc. This is all still a live work in progress, but we're landing this first tranche of work to help avoid continual issues with rebasing. Thanks to everyone who worked on it! xref #1190 Co-authored-by: John Eckersberg <[email protected]> Co-authored-by: Robert Sturla <[email protected]> Co-authored-by: Colin Walters <[email protected]> Signed-off-by: Pragyan Poudyal <[email protected]>

Johan-Liebert1 · 2025-09-15T12:53:46Z

Rebased again. Saved the previous branch to composefs-backend-15-09-2025

Conflict found

<<<<<<< HEAD
=======
    #[clap(hide(true))]
    #[cfg(feature = "docgen")]
    Man(ManOpts),
    #[cfg(feature = "composefs-backend")]
    ComposefsFinalizeStaged,
>>>>>>> a7185978 (Add a composefs backend)

resolved to

    #[cfg(feature = "composefs-backend")]
    ComposefsFinalizeStaged,

Johan-Liebert1 · 2025-09-15T13:16:26Z

Weird, the make-validate issue should've been fixed by #1606, but that change is neither present in the rebased branch nor in the branch before the rebase. It is present in https://github.com/bootc-dev/bootc/tree/composefs-backend-2025-09-12 though. Probably something went wrong during last Friday's rebase @cgwalters?

cgwalters · 2025-09-15T17:35:24Z

Hmm...ugh. Looking...

I was hitting a compliation error which *looks* like it was actaully an incremental compilation bug? Or it might have been rust-analyzer and local builds fighting over enabled features. Anyways, this ensures that we're gating on the composefs backend compiling. Signed-off-by: Colin Walters <[email protected]>

If we find UKI addons in the boot entries list, write them to ESP along with the UKI Signed-off-by: Pragyan Poudyal <[email protected]>

We don't need to write Grub menuentries for systemd-boot. For now the operation is a no-op, but later we would want to have .conf files in `ESP/loader/entries` so we can control the order of entries. Regarding that, we would also need to place the UKIs in a separate directory and not inside `ESP/EFI/Linux`, if we don't want duplicate entries, as systemd-boot will simply list all .efi files placed in EFI/Linux unconditionally Signed-off-by: Pragyan Poudyal <[email protected]>

cgwalters · 2025-09-15T18:58:22Z

OK after some auditing I think what happened is pretty simple, I probably forgot to git fetch and so rebased an old version of the branch. Pushed the 3 missing commits here.

cgwalters · 2025-09-15T18:59:27Z

Since I created this PR, I can't approve it; @jeckersb can you do so?

jeckersb

🥳

Add a comprehensive planning document for composefs integration that summarizes the current state and remaining work. The document covers the completed backend implementation (PR bootc-dev#1444), recent enhancements that made composefs the default, and outlines remaining work including fsverity upgrade paths, integrity verification, and testing gaps. This provides a roadmap for future composefs development and helps contributors understand the architecture and priorities. Signed-off-by: Ona <[email protected]> Co-authored-by: Ona <[email protected]>

gemini-code-assist bot reviewed Jul 24, 2025

View reviewed changes

cgwalters mentioned this pull request Aug 1, 2025

Composefs-native backend #1314

Merged

1 task

cgwalters force-pushed the composefs-backend branch 2 times, most recently from 33e4511 to 7d596fe Compare August 4, 2025 19:56

cgwalters force-pushed the composefs-backend branch from 7d596fe to d9b384d Compare August 7, 2025 15:18

Johan-Liebert1 force-pushed the composefs-backend branch 2 times, most recently from 828d837 to 1a7ac13 Compare August 19, 2025 09:07

jeckersb force-pushed the composefs-backend branch from 54e254c to f10ec34 Compare August 28, 2025 19:27

Johan-Liebert1 force-pushed the composefs-backend branch from f10ec34 to 67c4e8d Compare August 29, 2025 06:19

Johan-Liebert1 force-pushed the composefs-backend branch from 67c4e8d to b45ef30 Compare August 29, 2025 11:37

jeckersb force-pushed the composefs-backend branch from fd703ec to 023be10 Compare September 3, 2025 21:01

gemini-code-assist bot reviewed Sep 4, 2025

View reviewed changes

crates/lib/src/cli.rs Outdated Show resolved Hide resolved

crates/lib/src/install.rs Show resolved Hide resolved

crates/lib/src/bootc_composefs/state.rs Show resolved Hide resolved

crates/lib/src/install.rs Outdated Show resolved Hide resolved

cgwalters added the area/composefs Issues related to composefs label Sep 8, 2025

Johan-Liebert1 force-pushed the composefs-backend branch from bda61be to 389a922 Compare September 10, 2025 04:01

cgwalters force-pushed the composefs-backend branch from ab51fe7 to a718597 Compare September 12, 2025 16:19

cgwalters changed the title ~~WIP: composefs branch~~ Add a composefs backend Sep 12, 2025

cgwalters marked this pull request as ready for review September 12, 2025 16:20

bootc-bot bot requested a review from jeckersb September 12, 2025 16:20

Johan-Liebert1 force-pushed the composefs-backend branch from a718597 to 8ac9eae Compare September 15, 2025 12:52

cgwalters and others added 3 commits September 15, 2025 14:57

composefs-backend/uki: Handle UKI addons

147fcf3

If we find UKI addons in the boot entries list, write them to ESP along with the UKI Signed-off-by: Pragyan Poudyal <[email protected]>

cgwalters enabled auto-merge September 15, 2025 18:58

jeckersb approved these changes Sep 15, 2025

View reviewed changes

cgwalters merged commit 19e82be into main Sep 15, 2025
61 of 62 checks passed

cgwalters deleted the composefs-backend branch September 15, 2025 19:44

cgwalters-bot mentioned this pull request Oct 21, 2025

docs: Add composefs integration plan cgwalters/bootc#1

Open

Add a composefs backend #1444

Add a composefs backend #1444

Uh oh!

Conversation

cgwalters commented Jul 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cgwalters commented Aug 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cgwalters commented Aug 5, 2025

Uh oh!

cgwalters commented Aug 7, 2025

Uh oh!

cgwalters commented Aug 18, 2025

Uh oh!

Johan-Liebert1 commented Aug 28, 2025

Uh oh!

cgwalters commented Aug 28, 2025

Uh oh!

cgwalters commented Aug 28, 2025

Uh oh!

jeckersb commented Aug 28, 2025

Uh oh!

Johan-Liebert1 commented Aug 29, 2025

Uh oh!

jeckersb commented Sep 3, 2025

Uh oh!

cgwalters commented Sep 4, 2025

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Johan-Liebert1 commented Sep 9, 2025

Uh oh!

Johan-Liebert1 commented Sep 10, 2025

Uh oh!

cgwalters commented Sep 12, 2025

Uh oh!

cgwalters commented Sep 12, 2025

Uh oh!

Johan-Liebert1 commented Sep 15, 2025

Uh oh!

Johan-Liebert1 commented Sep 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cgwalters commented Sep 15, 2025

Uh oh!

cgwalters commented Sep 15, 2025

Uh oh!

cgwalters commented Sep 15, 2025

Uh oh!

jeckersb left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

cgwalters commented Jul 24, 2025 •

edited

Loading

cgwalters commented Aug 4, 2025 •

edited

Loading

Johan-Liebert1 commented Sep 15, 2025 •

edited

Loading