muvm: use HK_SYSMEM to set VRAM limit

By default, HK sets the heap size to be half the size of the *guest*
memory. Since commit 167744dc it's also possible to override the heap
size by setting the HK_SYSMEM environment variable.

Previously, the vram command line option was acting on the virtio-gpu
SHM window. This didn't help in most cases because userspace doesn't
really care nor is aware of it.

Here, we set the SHM window of virtio-gpu to always be as large as the
host's RAM, not because we expect VRAM to consume 100% of the RAM (which
is obviously impossible) but to account for region fragmentation. Then,
we use HK_SYSMEM to tell userspace the desired heap size (the closest
thing to VRAM we can aim for), which will be set to either the value
passed by the user throught the "vram" argument or half the size of the
*host* RAM.

Signed-off-by: Sergio Lopez <[email protected]>

Author: slp

crates/muvm/src/bin/muvm.rs

@@ -149,12 +149,22 @@ fn main() -> Result<()> {
             };
             MiB::from(guest_ram)
         };
-        // VRAM is actually the SHM window for virtio-gpu. The userspace drivers in the
-        // guest are expected to be conservative and tell applications that the amount
-        // of VRAM is just a percentage of the guest's RAM, so we can afford being more
-        // aggressive here and set the window to be as large as the system's RAM, leaving
-        // some room for dealing with potential fragmentation.
-        let vram_mib = options.vram.unwrap_or(MiB::from(ram_total_mib as u32));
+        // By default, HK sets the heap size to be half the size of the *guest* memory.
+        // Since commit 167744dc it's also possible to override the heap size by setting
+        // the HK_SYSMEM environment variable.
+        //
+        // Let's set the SHM window for virtio-gpu to be as large as the host's RAM, not
+        // because we expect VRAM to be as large as RAM, but to account for the more than
+        // likely region fragmentation.
+        //
+        // Then, let's set HK_SYSMEM to be either half the size of the *host* memory, or
+        // the value passed by the user with the "vram" argument.
+        let vram_shm_mib = MiB::from(ram_total_mib as u32);
+        let vram_mib = options.vram.unwrap_or(MiB::from(ram_total_mib as u32 / 2));
+        env.insert(
+            "HK_SYSMEM".to_owned(),
+            (u32::from(vram_mib) as u64 * 1024 * 1024).to_string(),
+        );
 
         // Bind the microVM to the specified CPU cores.
         let mut cpuset = CpuSet::new();
@@ -185,7 +195,7 @@ fn main() -> Result<()> {
             krun_set_gpu_options2(
                 ctx_id,
                 virgl_flags,
-                (u32::from(vram_mib) as u64) * 1024 * 1024,
+                (u32::from(vram_shm_mib) as u64) * 1024 * 1024,
             )
         };
         if err < 0 {

crates/muvm/src/cli_options.rs

@@ -72,12 +72,10 @@ pub fn options() -> OptionParser<Options> {
         .optional();
     let vram = long("vram")
         .help(
-            "The amount of Video RAM, in MiB, that will be available to this microVM.
-            The memory configured for the microVM will not be reserved
-            immediately. Instead, it will be provided as the guest demands
-            it, and will be returned to the host once the guest releases
-            the underlying resources.
-    [default: same as the total amount of RAM in the system]",
+            "The amount of Video RAM, in MiB, that will reported by userspace in this microVM.
+            The userspace drivers will report this amount as heap size
+            to the clients running in the microVM.
+    [default: 50% of total RAM]",
         )
         .argument("VRAM")
         .optional();