cpu templates: add T2S templates

The T2 template lets KVM populate the ARCH_CAPABILITIES MSR by using a
copy of the Host information. We sometimes want to force the information
contained in that MSR in order to make it look like a less capable
processor WRT speculative execution vulnerabilities and mitigations.
This is helpful when we want to snapshot a uVM on a newer processor and
resume on an older processor.
In this case, the software should use only the capabilities that are
available in both the newer and older host.

Create a new CPU template called T2S which is similar to the T2 template
but also sets additional information in the ARCH_CAPABILITIES MSR.
We basically set the MSR to expose to the guest a CPU that has the
security capabilities of a Skylake CPU, regardless of the host.
Also set some bits that were missing from the T2 CPUID config.

Adding the MSR information to the cpuid crate is not ideal based on
the current implementation. In the end the whole cpuid crate should
be refactored to remove the templates implementation into another crate.
The assumption that CPUID is the only place where information about the
processor is stored proves to be false so we need to extend the
templates to include at least the MSRs.

Signed-off-by: Alexandru-Cezar Sardan <[email protected]>

Author: Alexandru-Cezar Sardan

CHANGELOG.md

@@ -1,5 +1,22 @@
 # Changelog
 
+## Unreleased
+
+### Added
+
+- Add a new CPU template called `T2S`. This exposes the same CPUID as `T2`
+  to the Guest and also overwrites the `ARCH_CAPABILITIES` MSR to expose a
+  reduced set of capabilities. With regards to hardware vulnerabilities
+  and mitigations, the Guest vCPU will apear to look like a Skylake CPU,
+  making it safe to snapshot uVMs running on a newer host CPU (Cascade Lake)
+  and restore on a host that has a Skylake CPU.
+
+### Fixed
+
+- Make the `T2` template more robust by explicitly disabling additional
+  CPUID flags that should be off but were missed initially or that were
+  not available in the spec when the template was created.
+
 ## [1.1.0]
 
 ### Added

Cargo.lock

@@ -96,7 +96,7 @@ The **API endpoint** can be used to:
 - Configure the microvm by:
   - Setting the number of vCPUs (the default is 1).
   - Setting the memory size (the default is 128 MiB).
-  - [x86_64 only] Choosing a CPU template (currently, C3 and T2 are available).
+  - [x86_64 only] Choosing a CPU template (currently, C3, T2 and T2S are available).
 - Add one or more network interfaces to the microVM.
 - Add one or more read-write or read-only disks to the microVM, each represented
   by a file-backed block device.

README.md

@@ -782,6 +782,7 @@ definitions:
     enum:
       - C3
       - T2
+      - T2S
       - None
     default: "None"
 

src/api_server/swagger/firecracker.yaml

@@ -13,6 +13,7 @@ linux-loader = ">=0.4.0"
 versionize = ">=0.1.6"
 versionize_derive = ">=0.1.3"
 vm-fdt = "0.1.0"
+bitflags = ">=1.0.4"
 
 arch_gen = { path = "../arch_gen" }
 logger = { path = "../logger" }

src/arch/Cargo.toml

@@ -5,6 +5,7 @@
 use std::result;
 
 use arch_gen::x86::msr_index::*;
+use bitflags::bitflags;
 use kvm_bindings::{kvm_msr_entry, MsrList, Msrs};
 use kvm_ioctls::{Kvm, VcpuFd};
 
@@ -54,9 +55,74 @@ const MSR_KVM_POLL_CONTROL: u32 = 0x4b56_4d05;
 const MSR_KVM_ASYNC_PF_INT: u32 = 0x4b56_4d06;
 
 /// Taken from arch/x86/include/asm/msr-index.h
-const MSR_IA32_SPEC_CTRL: u32 = 0x0000_0048;
+/// Spectre mitigations control MSR
+pub const MSR_IA32_SPEC_CTRL: u32 = 0x0000_0048;
+/// Architecture capabilities MSR
+pub const MSR_IA32_ARCH_CAPABILITIES: u32 = 0x0000_010a;
+
 const MSR_IA32_PRED_CMD: u32 = 0x0000_0049;
 
+bitflags! {
+    /// Feature flags enumerated in the IA32_ARCH_CAPABILITIES MSR.
+    /// See https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/cpuid-enumeration-and-architectural-msrs.html
+    #[derive(Default)]
+    #[repr(C)]
+    pub struct ArchCapaMSRFlags: u64 {
+        /// The processor is not susceptible to Rogue Data Cache Load (RDCL).
+        const RDCL_NO               = 1 << 0;
+        /// The processor supports enhanced Indirect Branch Restriction Speculation (IBRS)
+        const IBRS_ALL              = 1 << 1;
+        /// The processor supports RSB Alternate. Alternative branch predictors may be used by RET instructions
+        /// when the RSB is empty. Software using retpoline may be affected by this behavior.
+        const RSBA                  = 1 << 2;
+        /// A value of 1 indicates the hypervisor need not flush the L1D on VM entry.
+        const SKIP_L1DFL_VMENTRY    = 1 << 3;
+        /// Processor is not susceptible to Speculative Store Bypass (SSB).
+        const SSB_NO                = 1 << 4;
+        /// Processor is not susceptible to Microarchitectural Data Sampling (MDS).
+        const MDS_NO                = 1 << 5;
+        /// The processor is not susceptible to a machine check error due to modifying the size of a code page
+        /// without TLB invalidation.
+        const IF_PSCHANGE_MC_NO     = 1 << 6;
+        /// The processor supports RTM_DISABLE and TSX_CPUID_CLEAR.
+        const TSX_CTRL              = 1 << 7;
+        /// Processor is not susceptible to Intel® Transactional Synchronization Extensions
+        /// (Intel® TSX) Asynchronous Abort (TAA).
+        const TAA_NO                = 1 << 8;
+        // Bit 9 is reserved
+        /// Processor supports IA32_MISC_PACKAGE_CTRLS MSR.
+        const MISC_PACKAGE_CTRLS    = 1 << 10;
+        /// Processor supports setting and reading IA32_MISC_PACKAGE_CTLS[0] (ENERGY_FILTERING_ENABLE) bit.
+        const ENERGY_FILTERING_CTL  = 1 << 11;
+        /// The processor supports data operand independent timing mode.
+        const DOITM                 = 1 << 12;
+        /// The processor is not affected by either the Shared Buffers Data Read (SBDR) vulnerability or the
+        /// Sideband Stale Data Propagator (SSDP).
+        const SBDR_SSDP_NO          = 1 << 13;
+        /// The processor is not affected by the Fill Buffer Stale Data Propagator (FBSDP).
+        const FBSDP_NO              = 1 << 14;
+        /// The processor is not affected by vulnerabilities involving the Primary Stale Data Propagator (PSDP).
+        const PSDP_NO               = 1 << 15;
+        // Bit 16 is reserved
+        /// The processor will overwrite fill buffer values as part of MD_CLEAR operations with the VERW instruction.
+        /// On these processors, L1D_FLUSH does not overwrite fill buffer values.
+        const FB_CLEAR              = 1 << 17;
+        /// The processor supports read and write to the IA32_MCU_OPT_CTRL MSR (MSR 123H) and to the FB_CLEAR_DIS bit
+        /// in that MSR (bit position 3).
+        const FB_CLEAR_CTRL         = 1 << 18;
+        /// A value of 1 indicates processor may have the RRSBA alternate prediction behavior,
+        /// if not disabled by RRSBA_DIS_U or RRSBA_DIS_S.
+        const RRSBA                 = 1 << 19;
+        /// A value of 1 indicates BHI_NO branch prediction behavior,
+        /// regardless of the value of IA32_SPEC_CTRL[BHI_DIS_S] MSR bit.
+        const BHI_NO                = 1 << 20;
+        // Bits 21:22 are reserved
+        /// If set, the IA32_OVERCLOCKING STATUS MSR exists.
+        const OVERCLOCKING_STATUS   = 1 << 23;
+        // Bits 24:63 are reserved
+    }
+}
+
 // Creates a MsrRange of one msr given as argument.
 macro_rules! SINGLE_MSR {
     ($msr:expr) => {
@@ -188,8 +254,8 @@ pub fn msr_should_serialize(index: u32) -> bool {
     ALLOWED_MSR_RANGES.iter().any(|range| range.contains(index))
 }
 
-// Creates and populates required MSR entries for booting Linux on X86_64.
-fn create_boot_msr_entries() -> Vec<kvm_msr_entry> {
+/// Creates and populates required MSR entries for booting Linux on X86_64.
+pub fn create_boot_msr_entries() -> Vec<kvm_msr_entry> {
     let msr_entry_default = |msr| kvm_msr_entry {
         index: msr,
         data: 0x0,
@@ -221,9 +287,8 @@ fn create_boot_msr_entries() -> Vec<kvm_msr_entry> {
 /// # Arguments
 ///
 /// * `vcpu` - Structure for the VCPU that holds the VCPU's fd.
-pub fn setup_msrs(vcpu: &VcpuFd) -> Result<()> {
-    let entry_vec = create_boot_msr_entries();
-    let msrs = Msrs::from_entries(&entry_vec).map_err(Error::FamError)?;
+pub fn set_msrs(vcpu: &VcpuFd, msr_entries: &[kvm_msr_entry]) -> Result<()> {
+    let msrs = Msrs::from_entries(&msr_entries).map_err(Error::FamError)?;
     vcpu.set_msrs(&msrs)
         .map_err(Error::SetModelSpecificRegisters)
         .and_then(|msrs_written| {
@@ -271,7 +336,8 @@ mod tests {
         let kvm = Kvm::new().unwrap();
         let vm = kvm.create_vm().unwrap();
         let vcpu = vm.create_vcpu(0).unwrap();
-        setup_msrs(&vcpu).unwrap();
+        let msr_boot_entries = create_boot_msr_entries();
+        set_msrs(&vcpu, &msr_boot_entries).unwrap();
 
         // This test will check against the last MSR entry configured (the tenth one).
         // See create_msr_entries() for details.

src/arch/src/x86_64/msr.rs

@@ -10,3 +10,4 @@ kvm-bindings = { version = ">=0.5.0", features = ["fam-wrappers"] }
 kvm-ioctls = ">=0.9.0"
 
 utils = { path = "../utils"}
+arch = { path = "../arch" }

src/cpuid/Cargo.toml

@@ -37,7 +37,9 @@ pub mod leaf_0x1 {
         // Virtual Machine Extensions
         pub const VMX_BITINDEX: u32 = 5;
         // 6 = SMX (Safer Mode Extensions)
+        pub const SMX_BITINDEX: u32 = 6;
         // 7 = EIST (Enhanced Intel SpeedStep® technology)
+        pub const EIST_BITINDEX: u32 = 7;
         // TM2 = Thermal Monitor 2
         pub const TM2_BITINDEX: u32 = 8;
         // CNXT_ID = L1 Context ID (L1 data cache can be set to adaptive/shared mode)
@@ -50,6 +52,7 @@ pub mod leaf_0x1 {
         // PDCM = Perfmon and Debug Capability
         pub const PDCM_BITINDEX: u32 = 15;
         // 18 = DCA Direct Cache Access (prefetch data from a memory mapped device)
+        pub const DCA_BITINDEX: u32 = 18;
         pub const MOVBE_BITINDEX: u32 = 22;
         pub const TSC_DEADLINE_TIMER_BITINDEX: u32 = 24;
         pub const OSXSAVE_BITINDEX: u32 = 27;
@@ -59,11 +62,13 @@ pub mod leaf_0x1 {
 
     pub mod edx {
         pub const PSN_BITINDEX: u32 = 18; // Processor Serial Number
+        pub const SSE42_BITINDEX: u32 = 20; // SSE 4.2
         pub const DS_BITINDEX: u32 = 21; // Debug Store.
         pub const ACPI_BITINDEX: u32 = 22; // Thermal Monitor and Software Controlled Clock Facilities.
         pub const SS_BITINDEX: u32 = 27; // Self Snoop
         pub const HTT_BITINDEX: u32 = 28; // Max APIC IDs reserved field is valid
         pub const TM_BITINDEX: u32 = 29; // Thermal Monitor.
+        pub const IA64_BITINDEX: u32 = 30; // IA64 processor emulating x86
         pub const PBE_BITINDEX: u32 = 31; // Pending Break Enable.
     }
 }
@@ -127,6 +132,7 @@ pub mod leaf_0x7 {
             // Intel® Resource Director Technology (Intel® RDT) Monitoring
             pub const RDT_M_BITINDEX: u32 = 12;
             // 13 = Deprecates FPU CS and FPU DS values if 1
+            pub const FPU_CS_DS_DEPRECATE_BITINDEX: u32 = 13;
             // Memory Protection Extensions
             pub const MPX_BITINDEX: u32 = 14;
             // RDT = Intel® Resource Director Technology
@@ -140,9 +146,10 @@ pub mod leaf_0x7 {
             // 20 = SMAP (Supervisor-Mode Access Prevention)
             // AVX512IFMA = AVX-512 Integer Fused Multiply-Add Instructions
             pub const AVX512IFMA_BITINDEX: u32 = 21;
-            // 21 = PCOMMIT intruction
-            // 22 reserved
-            // CLFLUSHOPT (flushing multiple cache lines in parallel within a single logical processor)
+            // 22 = PCOMMIT intruction
+            pub const PCOMMIT_BITINDEX: u32 = 22;
+            // CLFLUSHOPT (flushing multiple cache lines in parallel within a single logical
+            // processor)
             pub const CLFLUSHOPT_BITINDEX: u32 = 23;
             // CLWB = Cache Line Write Back
             pub const CLWB_BITINDEX: u32 = 24;

src/cpuid/src/cpu_leaf.rs

@@ -19,8 +19,7 @@ use crate::common::*;
 pub mod bit_helper;
 
 mod template;
-pub use crate::template::intel::c3;
-pub use crate::template::intel::t2;
+pub use crate::template::intel::{c3, t2, t2s};
 
 mod cpu_leaf;
 

src/cpuid/src/lib.rs

@@ -5,6 +5,9 @@
 pub mod c3;
 /// Follows a T2 template in setting up the CPUID.
 pub mod t2;
+/// Follows a T2 template for setting up the CPUID with additional MSRs
+/// that are speciffic to an Intel Skylake CPU.
+pub mod t2s;
 
 use crate::common::{get_vendor_id_from_host, VENDOR_ID_INTEL};
 use crate::transformer::Error;