Use direct map removed guest_memfd for backing "secret free" VMs #5131
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## feature/secret-hiding #5131 +/- ##
=========================================================
- Coverage 82.86% 82.73% -0.13%
=========================================================
Files 251 251
Lines 27291 27522 +231
=========================================================
+ Hits 22614 22771 +157
- Misses 4677 4751 +74
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
roypat
force-pushed
the
secret-freedom-guest-memfd
branch
8 times, most recently
from
cf8e81b to
e593aa2
Compare
roypat
force-pushed
the
feature/secret-hiding
branch
from
bf38d65 to
cfd4d8b
Compare
roypat
force-pushed
the
secret-freedom-guest-memfd
branch
10 times, most recently
from
80824bf to
41f5cb7
Compare
Add a utility function for creating a guest_memfd and wrapping it into a `File` object. Signed-off-by: Patrick Roy <[email protected]>
There's be a lot more things that are incompatible going forward (mostly related to secret freedom), so instead of adding a ton of error variants for each pair of incompatible features, let's just have a single one where we can insert arbitrary features via a string argument. Signed-off-by: Patrick Roy <[email protected]>
roypat
force-pushed
the
secret-freedom-guest-memfd
branch
from
ceaf67f to
80411d7
Compare
roypat
changed the title
roypat
force-pushed
the
secret-freedom-guest-memfd
branch
2 times, most recently
from
476c68c to
6059353
Compare
kalyazin
reviewed
Apr 8, 2025
This will later indicate to Firecracker that guest memory should be backed by guest_memfd. Signed-off-by: Patrick Roy <[email protected]>
roypat
force-pushed
the
secret-freedom-guest-memfd
branch
from
a8c1367 to
405f22c
Compare
roypat
force-pushed
the
secret-freedom-guest-memfd
branch
2 times, most recently
from
75b4399 to
dd61db6
Compare
JackThomson2
reviewed
Apr 9, 2025
roypat
force-pushed
the
secret-freedom-guest-memfd
branch
from
dd61db6 to
5579e7a
Compare
When guest memory is backed by direct map removed guest_memfd ("secret
free"), we cannot load the guest kernel / initrd by read-ing into guest
memory, as the kernel won't be able to access guest memory. Instead
we'll need to read into a userspace buffer, and then memcpy into guest
memory. Add a newtype that wraps Read/Write and performs exactly this
operation, and use it to load guest kernel / initrd.
Signed-off-by: Patrick Roy <[email protected]>
Fall back to kvm_user_memory_region in case the 2 version of the struct isnt supported. Signed-off-by: Patrick Roy <[email protected]>
vm-memory has faulty validation logic that prevents us from mmap-ing guest_memfds, so just bypass that by calling mmap ourselves for the time being. See also rust-vmm/vm-memory#320 Signed-off-by: Patrick Roy <[email protected]>
If the `secret_free` field of the memory_config is set to true in the /machine-config endpoint, back all non-swiotlb regions using guest_memfd. For our setup, this means both setting the guest_memfd[_offset] fields in kvm_user_memory_region2, as well as mmaping the guest memory and reflecting this VMA back into the memslot's userspace_addr (which is how kvm internal accesses to guest memory will work for these guest_memfd regions, such as mmio emulation on x86). Signed-off-by: Patrick Roy <[email protected]>
To take snapshots of secret hidden VMs, we need to bounce guest memory through a userspace buffer. Reuse the `Bounce` wrapper type that is already in use for loading the guest kernel / initrd. Signed-off-by: Patrick Roy <[email protected]>
On x86, only KVM_X86_SW_PROTECTED_VM VMs support guest_memfd, so we have to explicitly specify this VM type. On Arm, all VMs support it. Signed-off-by: Patrick Roy <[email protected]>
The current version of the mmap-support patches require that on x86, memory attributes have to be set to private even if the guest_memfd VMA is short-circuited back into the memslot (on ARM, memory attributes are not even supported in this scenario). Signed-off-by: Patrick Roy <[email protected]>
Add a test that we can boot "normal" VMs on ARM with secret freedom enabled (e.g. I/O works through the swiotlb region), and test that on x86 we can boot at least an initrd (e.g. a very simple VM that doesnt have any I/O devices attached). Skip tets on m6g.metal, as currently direct map removal causes panics on this hardware. Signed-off-by: Patrick Roy <[email protected]>
Get some throughput data from our perf tests with secret freedom enabled. Signed-off-by: Patrick Roy <[email protected]>
Since we load the kernel using bounce buffers now, it will give us false-positives. Signed-off-by: Patrick Roy <[email protected]>
roypat
force-pushed
the
secret-freedom-guest-memfd
branch
from
5579e7a to
df1780d
Compare
kalyazin
approved these changes
Apr 9, 2025
JackThomson2
approved these changes
Apr 9, 2025
roypat
merged commit 1891d72
into
firecracker-microvm:feature/secret-hiding
5 of 7 checks passed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes
Add a
secret_freeparameter to the API, which indicates to Firecracker that non-swiotlb memory should be backed by guest_memfd, and that this guest_memfd should be initialized in a way such that its direct map entries are zapped.Reason
...
License Acceptance
By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache 2.0 license. For more information on following Developer
Certificate of Origin and signing off your commits, please check
CONTRIBUTING.md.PR Checklist
tools/devtool checkstyleto verify that the PR passes theautomated style checks.
how they are solving the problem in a clear and encompassing way.
in the PR.
CHANGELOG.md.Runbook for Firecracker API changes.
integration tests.
TODO.rust-vmm.