feat: handle hermit images + config nested in image

[fogti]

This fixes the first part of #1015.

Open TODOs:

• Make sure this code actually works a expected.
  • Make sure that the mounting / reading of files from an image works
  • Make sure that config + kernel retrieval from an image works
• Document hermit-image-reader.
• Add property test for tar parser in hermit-image-reader.
• Handle the retrieval of the kernel image itself.
• Cache decompressed hermit image earlier, so that we don't have to do that twice.
  • isolation::image::Cache implements a cache for decompressed + parsed hermit images
  • Integrate into config and kernel lookup
• Ensure that writes to files from images fail. (It currently explicitly handles unlinks of files from images (by removing the entry from the image, I think that's okay))
• Where should we put the hermit-image-reader crate?
• Implement similar stuff into loader + kernel FDT handler
  • Integrate format detection logic + decompression into the loader
  • Implement config+kernel retrieval from image into the loader
  • Implement passing the decompressed tar/image file location via FDT (how should the entry be called?)
  • Implement tarfs logic in kernel (Investigate tarfs support (as part of image support) kernel#2009)
  • Implement mounting tarfs in kernel when FDT entry is present, support passing of image both from uhyve and from loader
  • After implementation in kernel, we might be able to get rid of most of the complicated image logic from uhyve

The current state is basically "At least it compiles".

Note that it was made clear in discussions that iterating over the tar file 2 or 3 files (linear scan) is considered acceptable, when necessary to keep the amount of data to be passed to the kernel at a minimum).

[fogti]

Somehow, it errornously interprets some file paths as directories now...

[fogti]

Okay, I fixed the regressions (which were basically an path.join("") invocation, which (to my initial surprise) added a trailing /.

[n0toose]

Where should we put the hermit-image-reader crate?

In the short run, I'd say that it'd make sense to put the crate in the Uhyve repository (which we already do for uhyve-interface).

I am not sure if we'd make the effort to somehow use our work in QEMU as well in the short run.

[jounathaen]

Yes, put the crate into uhyve. We can split it off later if we want to.

[fogti]

Does it make sense to put it into uhyve if we need it in hermit-os/loader?

[n0toose]

Does it make sense to put it into uhyve if we need it in hermit-os/loader?

I mean, we need uhyve-interface for the kernel, but it is still in the kernel? It's not exactly a cyclic dependency if the crates are different, right? (but the vibes are off for me as well because the repositories are cyclically depending on each other :^) )

[mkroening]

Since this is incubating in Uhyve and has a main focus on Uhyve (for now at least, as I understand it), I think putting it in this repo in the beginning would be fine. Any changes would also be more flexible, since they would not require cross-repo coordination.

Once this matures, we can reevaluate this, of course.

[codecov]

Codecov Report

❌ Patch coverage is 70.24221% with 86 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.96%. Comparing base (d74257a) to head (284e620).

Files with missing lines	Patch %	Lines
src/vm.rs	12.96%	47 Missing ⚠️
src/isolation/image.rs	76.56%	15 Missing ⚠️
src/isolation/filemap.rs	86.40%	14 Missing ⚠️
src/hypercall.rs	75.00%	9 Missing ⚠️
src/bin/uhyve.rs	75.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1135      +/-   ##
==========================================
- Coverage   76.36%   75.96%   -0.41%     
==========================================
  Files          29       30       +1     
  Lines        3613     3798     +185     
==========================================
+ Hits         2759     2885     +126     
- Misses        854      913      +59     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[n0toose]

Once this matures, we can reevaluate this, of course.

Agreed

[fogti]

The most important TODO item is now: Introduce some caching mechanism which handles decompressing all the relevant images, and later allows us to reuse the decompressed images both for finding the included config file, extracting the included kernel, and the setup of file mapping.

edit: The first part of this is solved now (I extracted a Cache implementation from what I already did in isolation::filemap), it remains to actually thread this through the code base (i.e. there should afaik be a dedicated UhyveVm constructor which takes an image path as argument, and handles config + kernel lookup, and internally creates a Cache to do that, and later passes that to the filemap constructor)

[jounathaen]

I'd propose a newtype around the Vec<u8> like tarbytes and a from(tarbytes) for the tar_parser

[fogti]

script for testing:

#!/usr/bin/env bash

uhyve="$HOME/devel/uhyve"
kernel="fs_tests"
image="$(realpath ./$kernel.hermit)"

set -e

cd "$uhyve"

echo "=== building kernel $kernel ... ==="

pushd tests/test-kernels

HERMIT_LOG_LEVEL_FILTER=Debug cargo +nightly build -Zbuild-std=std,panic_abort --target=x86_64-unknown-hermit --bin "$kernel"
KERNEL_PATH="$(realpath target/x86_64-unknown-hermit/debug/$kernel)"

popd

echo "=== building image $image ... ==="

TMPD="$(mktemp -d)"

echo "setting up temporary directory $TMPD ..."
pushd "$TMPD"

cat > "$TMPD/"hermit_config.toml <<EOF
version = "1"
kernel = "$kernel"

[input]
kernel_args = []
app_args = ["testname=open_read", "filename=/root/hello_world.txt"]
env_vars = []

[requirements]
EOF

cp -t. "$KERNEL_PATH"

mkdir "$TMPD/root"
echo "Hello, world!" > "$TMPD/root/hello_world.txt"

rm -f "$image" && tar -czvf "$image" *

popd

rm -rf "$TMPD"

echo "$image"

echo "=== running image $image ... ==="

export RUST_LOG="${RUST_LOG:-debug}"
"$uhyve"/target/release/uhyve "$image"

appears to work

[fogti]

Marked as draft because this depends on an experimental/version-controlled hermit-entry version.

[fogti]

Something breaks in strict file isolation, but it's weird...

2025-11-07T14:08:53.4376900Z ##[group]Run cargo run data/x86_64/hello_world --stats --file-isolation strict
2025-11-07T14:08:53.4377875Z �[36;1mcargo run data/x86_64/hello_world --stats --file-isolation strict�[0m
2025-11-07T14:08:53.4378750Z �[36;1mcargo run data/x86_64/rusty_demo --stats --file-isolation strict�[0m
2025-11-07T14:08:53.4379601Z �[36;1mcargo run data/x86_64/hello_c --stats --file-isolation strict�[0m
2025-11-07T14:08:53.4402379Z shell: /usr/bin/bash -e {0}
2025-11-07T14:08:53.4403019Z env:
2025-11-07T14:08:53.4403353Z   CARGO_HOME: /.cargo
2025-11-07T14:08:53.4403753Z   CARGO_INCREMENTAL: 0
2025-11-07T14:08:53.4404162Z   CARGO_TERM_COLOR: always
2025-11-07T14:08:53.4404578Z   CACHE_ON_FAILURE: false
2025-11-07T14:08:53.4404958Z ##[endgroup]
2025-11-07T14:08:53.6191078Z �[1m�[92m    Finished�[0m `dev` profile [unoptimized + debuginfo] target(s) in 0.11s
2025-11-07T14:08:53.6270663Z �[1m�[92m     Running�[0m `target/debug/uhyve data/x86_64/hello_world --stats --file-isolation strict`
2025-11-07T14:08:53.6396570Z [0][INFO] Welcome to Hermit 0.6.9
2025-11-07T14:08:53.6418274Z [0][INFO] Kernel starts at bfe40000
2025-11-07T14:08:53.6441605Z [0][INFO] BSS starts at 0xbfedb680
2025-11-07T14:08:53.6460443Z [0][INFO] tls_info = Some(
2025-11-07T14:08:53.6470027Z     TlsInfo {
2025-11-07T14:08:53.6486857Z         start: 0xbfed06f8,
2025-11-07T14:08:53.6494796Z         filesz: 0x20,
2025-11-07T14:08:53.6499813Z         memsz: 0x81,
2025-11-07T14:08:53.6504580Z         align: 0x8,
2025-11-07T14:08:53.6506290Z     },
2025-11-07T14:08:53.6506974Z )
2025-11-07T14:08:53.6552160Z [0][PANIC] panicked at src/arch/x86_64/mm/paging.rs:319:58:
2025-11-07T14:08:53.6575783Z called `Result::unwrap()` on an `Err` value: PageNotMapped
2025-11-07T14:08:53.6584651Z Number of interrupts
2025-11-07T14:08:53.6600304Z [0][INFO] shutting down with code 1
2025-11-07T14:08:53.6719581Z Run statistics:
2025-11-07T14:08:53.6722605Z VM exits:                       total     cpu0 
2025-11-07T14:08:53.6724282Z   Hypercall(Exit):                  1        1 
2025-11-07T14:08:53.6725564Z   Hypercall(Uart):                421      421 
2025-11-07T14:08:53.6726759Z CPU runtimes:
2025-11-07T14:08:53.6727514Z   cpu 0: 25.057727ms
2025-11-07T14:08:53.6728221Z 
2025-11-07T14:08:53.6762370Z ##[error]Process completed with exit code 1.

Unfortunately, I can't even reproduce this on my laptop, i.e. this is almost impossible for me to debug as-is.

[fogti]

TODO: forward Hermit image as blob to the kernel (via paging / kvm_userspace_memory_region & FDT /chosen.image_reg).

[fogti]

Blocked on #1256.