This assertion logic is a bit complex. A simpler and more idiomatic way to compare two collections for equality regardless of order is to sort them and then compare the sorted collections. This pattern is repeated for modified and removed files as well.

General question, not something specific about this PR (this looks great BTW!)

Would Bootc's ostree backend be refactored to make use of this crate in time, or is the plan for ostree to remain mostly separate? Obviously the implementation would be completely separate PR to this anyway.

This is a good question indeed. I've been really tempted to basically vendor+rewrite some of the ostree-sysroot stuff in this project. However, it has a ton of implications.

Two key things here. First I think we should incremental hashing instead of reading the whole thing into memory and then hashing the memory buffer.

But second a nice optimization to make here is handling the case where the file has fsverity enabled. Then we can efficiently just query its fsverity hash to do a diff.

It won't be the default for files in a mutable /etc to have fsverity enabled but it makes sense to do so (dir is mutable, so the write-tmpfile-then-rename pattern continues to work) and we should make things faster for those who do use it.

This is ok as is, but since we need to use raw unix stuff we might as well just directly use rustix::fs::fchmodat

Here and elsewhere, there's get_directory_opt which is intended for exactly this use case

I realised I had used get_directory as we would still want to show a diff if a file was changed to a directory and vice versa. Given, we will fail while trying to merge, but I think it makes sense to show it as a diff?

I wonder if it'd be cleaner to have added actually be a borrowed reference to the inode for the current directory.

Then it's implicit that leaf files are, well, leaves - and directories are recursive.

In order to do this, the calling code needs to make a copy of the new pristine etc already pre-populated into new_etc_fd.

But...while it's generally useful to have "compute diff without merging", won't the main use case be to actually just one-shot a combination of diff + merge?

IOW, how about renaming this function merge_with and then we have

pub fn merge(pristine_etc_fd: &Dir, current_etc_fd: &Dir, new_etc_fd: &Dir)

And inside that function we do:

• Copy pristine to new
• Compute diff
• Apply diff

This is kind of what https://github.com/ostreedev/ostree/blob/edfe02d01b78039db67c4247df6948070eae0cbe/src/libostree/ostree-sysroot-deploy.c#L511 is already doing, though https://github.com/ostreedev/ostree/blob/edfe02d01b78039db67c4247df6948070eae0cbe/src/libostree/ostree-sysroot-deploy.c#L511 does the copy beforehand.

I had the diff function separate for testing while developing, and because ostree had a command that'd just print the diff between two etcs. I think it makes sense to combine both into a single merge function.

Right. Though I'm just saying we add a new high level helper, not that we need drop the split APIs.

I see. That makes sense. Thinking about this a bit more, I think the best place to copy files/dirs in the new_etc would be in the function write_composefs_state. During a fresh install there will be no merging of etcs; only on subsequent updates will the merge happen, so we'll kinda have two places where we copy the contents of etc depending upon what operation was performed.

During a fresh install there will be no merging of etcs; only on subsequent updates will the merge happen, so we'll kinda have two places where we copy the contents of etc depending upon what operation was performed.

Yeah, in ostree the copying always happens outside of the merge function. So I'm fine having it always be separate here too.

Optional/for followup: This could probably use https://docs.rs/cap-std-ext/latest/cap_std_ext/dirext/trait.CapStdExtDirExt.html#tymethod.walk (I maintain that crate, a lot of handy helpers there)