|
| 1 | +# Trusted Device Manager Architecture |
| 2 | + |
| 3 | +This is an architecture document for the Trusted Device Manager (TDM), a |
| 4 | +confidential guest, userspace component for implementing trusted I/O. |
| 5 | + |
| 6 | +## Goals |
| 7 | + |
| 8 | +In order to achieve high performance Input/Output (I/O) operations, confidential |
| 9 | +VMs must extend their trust boundary to include a composition of directly |
| 10 | +assigned, TEE-IO-capable devices/functions. Without this ability, confidential |
| 11 | +VMs have to resort to using para-virtualized I/O using non-confidential memory |
| 12 | +regions, which has performance impacts due to memory copies and negates the use |
| 13 | +of directly assigned devices. |
| 14 | + |
| 15 | +Silicon vendors and device manufacturers provide TEE-IO compliant hardware |
| 16 | +implementations. Intel TDX Connect or AMD SEV-TIO, for example, rely on device |
| 17 | +manufacturers to support the TDISP, SPDM and PCIe IDE protocols to give |
| 18 | +confidential guests access to trusted MMIO and secure DMA between them and |
| 19 | +devices which trustworthiness is verifiable. |
| 20 | + |
| 21 | +The TEE-IO security model is built on top of this hardware support but also on |
| 22 | +the ability for confidential guests to ultimately decide if they accept or |
| 23 | +reject TEE-IO compliant devices into their TCB. The TDM, as a confidential guest |
| 24 | +component, is the TCB acceptance decision maker and is responsible for notifying |
| 25 | +the underlying Trusted Security Manager (TSM) about its decision. By accepting a |
| 26 | +TDI into the guest TCB, the TDM explicitly requests the TSM to move the device |
| 27 | +into an operational state. The TDM builds the acceptance decision after having |
| 28 | +verified and authenticated the TDI. It must also attest of the assigned device |
| 29 | +trustworthiness, through a local or remote attestation process. |
| 30 | + |
| 31 | +## Scope |
| 32 | + |
| 33 | +## Architectural Overview |
| 34 | + |
| 35 | +High-level view: |
| 36 | + |
| 37 | +``` |
| 38 | + ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ |
| 39 | + ┃ Confidential Guest ┃ |
| 40 | + ┃ ┃ |
| 41 | + ┃ ╔═══════════════════════╗ ┃ |
| 42 | + ┃ ║Trusted Device Manager ║ ┃ |
| 43 | + ┃ ║ ║ ┃ |
| 44 | + ┃ ║┌─────────────────────┐║ ┃ |
| 45 | + ┃ ║│ Attestation Module │║ ┃ |
| 46 | + ┃ ┌───▷║└─────────────────────┘║ ┃ |
| 47 | + ┃ │ ║ ║ ┃ |
| 48 | + ┃ │ ║ ║ ┃ |
| 49 | + ┃ │ ║ ╔════════╗ ╔════════╗ ║ ┃ |
| 50 | + ┃ │ ╚═╣ TEE ╠═╣ Device ╠═╝ ┃ |
| 51 | + ┃ │ ║ Plugin ║ ║ Plugin ║ ┃ |
| 52 | + ┃ udev ╚════════╝ ╚════════╝ ┃ |
| 53 | + ┃(PCIe device △ ┃ |
| 54 | + ┃ detection) │ ┃ |
| 55 | + ┃ │ Device Evidence ┃ |
| 56 | + ┃ │ Accept/Reject ┃ |
| 57 | + ┃ │ │ ┃ |
| 58 | + ┃ │ ▽ ┃ |
| 59 | + ┃ │ ┌───────┐ ┌──────────┐ ┃ |
| 60 | + ┃ │ ┌───┤TEE ABI├───┤Device ABI├─┐ ┃ |
| 61 | + ┃ │ │ └───────┘ └──────────┘ │ ┃ |
| 62 | + ┃ └──│ Linux Guest Kernel │ ┃ |
| 63 | + ┃ └────────────────────────────┘ ┃ |
| 64 | + ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛ |
| 65 | + △ |
| 66 | + │ |
| 67 | + │ |
| 68 | + TEE-IO ABI |
| 69 | + │ |
| 70 | + ▽ |
| 71 | + ┏━━━━━━━━━━━━┓ |
| 72 | +┏━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ Trusted ┃ |
| 73 | +┃ ┃ ┃ Security ┃ |
| 74 | +┃ Host VMM/Hypervisor ┃ ┃ Manager ┃ |
| 75 | +┃ (KVM+QEMU/CLH) ┃◁───────TEE ABI─────▷┃(TDX Module,┃ |
| 76 | +┃ ┃ ┃ ASP, etc) ┃ |
| 77 | +┗━━━━━━━━━━━━━━━━━━━━━━━┛ ┃ ┃ |
| 78 | + △ ┗━━━━━━━━━━━━┛ |
| 79 | + ┃ |
| 80 | + PCIe Bus |
| 81 | + ┃ |
| 82 | + ┃ |
| 83 | + ▽ |
| 84 | +┏━━━━━━━━━━━━━━━━━━━━━━━┓ |
| 85 | +┃ TEE-IO Compliant PCIe ┃ |
| 86 | +┃ Device ┃ |
| 87 | +┗━━━━━━━━━━━━━━━━━━━━━━━┛ |
| 88 | +``` |
| 89 | + |
| 90 | +In order for the TDM to achieve the above described goals, it must be able to: |
| 91 | + |
| 92 | +1. Detect VMM-assigned PCIe devices. This relies on the Linux kernel `udev` |
| 93 | + subsystem. TDM also assumes the TSM must not enable any DMA or MMIO mappings |
| 94 | + before it accepted the device. |
| 95 | +2. Authenticate the assigned device. Both authentication and attestation of the |
| 96 | + TDI relies on the ability for the TDM to retrieve a certificate chain and an |
| 97 | + attestation evidence. This is provided by the Linux kernel confidential |
| 98 | + computing ABI, which is generally exposed as a vendor-specific, ioctl-based |
| 99 | + character device. |
| 100 | +3. Verify the assigned device. Before accepting a TDI into the confidential |
| 101 | + guest TCB, the TDM must use the Linux kernel CoCo ABI to verify the following |
| 102 | + security attributes from the TSM: |
| 103 | + * Device link state |
| 104 | + * TDI state |
| 105 | + * TDI MMIO mappings |
| 106 | +4. Attest of the assigned device trustworthiness. As part of the overall device |
| 107 | + verification process, the TDM must run a local or remote attestation of the |
| 108 | + device attestation evidence: |
| 109 | + 1. For running local device attestation, the TDM needs to practically become a |
| 110 | + Relying Party. |
| 111 | + 2. In order to support remote device attestation, the TDM would rely on |
| 112 | + existing guest components like the attestation agent. |
| 113 | +5. Notify the TSM of it TDI acceptance decision. Here as well, this relies on |
| 114 | + the Linux kernel CoCo ABI. |
| 115 | + |
| 116 | +This leads to a natural split of the TDM software architecture into 4 main |
| 117 | +modules: |
| 118 | + |
| 119 | +1. The **TDM Core** runs the main TDM flows and managing confidential guest |
| 120 | + assignment with the support of the detection, ABI plugins and attestation |
| 121 | + modules. |
| 122 | +2. The **Device Detection** module traps device discovery, assignment and |
| 123 | + removal from the confidential guest PCIe emulated bus. |
| 124 | +3. The **Linux CoCo ABI plugins** abstracts the vendor specific CoCo Linux |
| 125 | + kernel ABIs in order expose a vendor-agnostic internal interface for the TDM |
| 126 | + core to consume. |
| 127 | +4. The **Device Attestation** module is responsible for verifying a device |
| 128 | + attestation evidence, which can be achieved through local or remote |
| 129 | + attestation. This module acts as a device attestation relying party to |
| 130 | + support the former, while it relies on the Attestation Agent API to support |
| 131 | + the latter case. It provides attestation results back to the TDM core. |
| 132 | + |
| 133 | +``` |
| 134 | + ┌────────────────────────┐ |
| 135 | + │ Trusted Device Manager │ |
| 136 | + ┌───────────────────────────────┴────────────────────────┴────────────────────────────────────┐ |
| 137 | + │ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ┐ │ |
| 138 | + │ ┌───────────┐ ┌─────────────┐ ┌───────────┐ │ |
| 139 | + │ │ Local │ │ │ Attester │ │ │Attestation│ │ |
| 140 | + │ │Attestation│─────▷│ Crate │──────▷│ Service │◁─┐ │ |
| 141 | + │ └───────────┘ │ └─────────────┘ │ │ Crate │ │ │ |
| 142 | + │ ┌──────────┐ ┌────────────┐ │ └───────────┘ │ │ |
| 143 | + │ │ Device │ │ Device │ │ │ │ ┌─────────┐ │ │ |
| 144 | + ┏╋▶│Detection │ │Attestation │◁──Attestation │Policies │───┘ │ |
| 145 | + ┃│ │ Module │ │ Module │ Token │ │ │ ├─────────┤ │ |
| 146 | + ┃│ └──────────┘ └────────────┘ │ │Reference│ │ |
| 147 | + ┃│ △ △ ┌───────────┐ │ ┌─────────────┐ │ │ Values │ │ |
| 148 | + ┃│ └──────┐┌─────┘ │ Remote │ │ Attestation │ └─────────┘ │ |
| 149 | + ┃│ ││ │Attestation│────┼▷│ Agent │ │ │ |
| 150 | + ┃│ ┌──────▽▽──────┐ └───────────┘ │ Crate │ │ |
| 151 | + ┃│ │ Core Module │ │ └─────────────┘ │ │ |
| 152 | + ┃│ └──────────────┘ │ |
| 153 | + ┃│ △ │ Attestation │ │ |
| 154 | + PCIe ┃│ └──────────────┐ Agent │ |
| 155 | + udev ┃│ │ └ ─ ─ ─ ─ ─ ─ ─ ─ ┘ │ |
| 156 | +events┃│ ▽ │ |
| 157 | + ┃│ ┌──────────────────────────────────────────────────────┐ │ |
| 158 | + ┃│ │ TDI Management Module │ │ |
| 159 | + ┃│ ├─────────────────────────────┬┬───────────────────────┤ │ |
| 160 | + ┃│ │TEE Plugins ││Device Vendor Plugins │ │ |
| 161 | + ┃│ │┌──────┐ ┌─────┐ ┌────┐ ┌───┐││┌──────┐ ┌──────┐ ┌───┐│ │ |
| 162 | + ┃│ ││ TDX │ │ SEV │ │ SE │ │...││││NVIDIA│ │ BCOM │ │...││ │ |
| 163 | + ┃│ │└──────┘ └─────┘ └────┘ └───┘││└──────┘ └──────┘ └───┘│ │ |
| 164 | + ┃│ └─────────────────┳───────────┘└───────────┳───────────┘ │ |
| 165 | + ┃└────────────────────╋────────────────────────╋───────────────────────────────────────────────┘ |
| 166 | + ┃ ┃ ┃ |
| 167 | + ┃ CoCo ABI Device ABI |
| 168 | + ┃ ┃ ┃ |
| 169 | + ┃ ▼ ▼ |
| 170 | + ┃ ┌──────────────────────────────────────────────────────┐ |
| 171 | + ┗━━━│ Linux Guest Kernel │ |
| 172 | + └──────────────────────────────────────────────────────┘ |
| 173 | +``` |
| 174 | + |
| 175 | +### TDM Core |
| 176 | + |
| 177 | +The TDM core module interacts with three other TDM components: |
| 178 | + |
| 179 | +1. The device detection module. It monitors udev events to detect PCI devices |
| 180 | + addition and removal from the guest emulated PCI bus. This module calls into |
| 181 | + the TDM core to notify it about both event types. The TDM core then relies on |
| 182 | + the following two modules to verify, attest and accept newly detected |
| 183 | + devices. Only TDM-accepted TDIs are visible and available to the rest of the |
| 184 | + guest software stack. |
| 185 | +2. The TDI management module. This is an abstraction over the Linux CoCo |
| 186 | + vendor-specific ABI and the TDM core calls into this module in order to |
| 187 | + communicate with the TSM. The TDI management module module abstracts the |
| 188 | + vendor-specific TSM interfaces in order for the TDM to: |
| 189 | + * Retrieve a certificate chain for any newly detected device. |
| 190 | + * Retrieve an attestation evidence for any newly detected device. This is |
| 191 | + optional as the evidence may be delivered as part of the device certificate |
| 192 | + chain. |
| 193 | + * Check the physical device link state and verify that it is confidentiality |
| 194 | + and integrity protected, as provided by the PCI IDE protocol. The TSM |
| 195 | + manages the IDE keys and must track the device physical link state for any |
| 196 | + assigned devices. |
| 197 | + * Check for the TDI state in order to verify that it is in the (TDISP) locked |
| 198 | + state. |
| 199 | + * Check that all emulated MMIO ranges are correctly (in respect to their size |
| 200 | + and order in the guest address space) mapped. |
| 201 | + * Accept or reject an assigned TDI. |
| 202 | +3. The device attestation module. Once the TDM has authenticated and verified an |
| 203 | + assigned TDI, it can attest it through local or remote attestation before |
| 204 | + finally accepting it. The TDM core fetches an attestation evidence through |
| 205 | + the ABI plugin module and forwards it to the device attestation module. This |
| 206 | + module then verifies the device evidence either locally or remotely, depending |
| 207 | + on a guest-specific TDM policy, and returns an attestation results back to the |
| 208 | + TDM core. |
| 209 | + |
| 210 | +### Device Detection |
| 211 | + |
| 212 | +### TEE ABI Plugins |
| 213 | + |
| 214 | + |
| 215 | + |
| 216 | +#### Plugin Interface |
| 217 | + |
| 218 | +A Rust trait called by the TDM core to interact with the TSM. |
| 219 | +(Not needed if Linux provides a converged TEE userspace API. |
| 220 | + |
| 221 | +``` rust |
| 222 | + |
| 223 | + |
| 224 | + |
| 225 | +``` |
| 226 | + |
| 227 | +### Device Attestation |
| 228 | + |
| 229 | +## Theory of Operations |
| 230 | + |
| 231 | +### TEE-IO Acceptance Flow |
| 232 | + |
| 233 | +#### Detection |
| 234 | + |
| 235 | +#### TDI Security Attributes Check |
| 236 | + |
| 237 | +#### Device Attestation |
| 238 | + |
| 239 | +##### Local Attestation |
| 240 | + |
| 241 | +TDM is a Relying Party. |
| 242 | +TDM to use part of the Attestation Service crates |
| 243 | + |
| 244 | +##### Remote Attestation |
| 245 | + |
| 246 | +## Glossary |
| 247 | + |
| 248 | +| Term | Acronym | Definition | |
| 249 | +|:-----|:-------:|------------| |
| 250 | +| Confidential Computing | CoCo | The protection of data in use by performing computation in a Hardware-based TEE. |
| 251 | +| Confidential VM | CVM | A confidential computing virtual machine. Same as a TVM. |
| 252 | +| Device Security Manager | DSM | A DSM is a logical entity on a TEE-IO device that enforces the TDISP security policies, attributes and states. |
| 253 | +| Integrity and Data Encryption | IDE | Extended PCIe capability for integrity, confidentiality and replay protection of PCIe Transport Layer Packets (TLP). |
| 254 | +| Security Protocol and Data Model | SPDM | A DMTF defined specification for exchanging messages between devices over a variety of transports and physical media. SPDM is used to exchange TDISP and IDE Key Management messages over PCIe DOE mailboxes. |
| 255 | +| TEE Device Interface Security Protocol | TDISP | An architecture for trusted I/O virtualization. |
| 256 | +| TEE Input and Output | TEE-IO | A PCIe-defined conceptual framework for establishing and managing trust relationships between a PCIe device and a TEE. |
| 257 | +| TEE I/O Device Interface | TDI | The unit of assignment for a trusted I/O capable device. For example, a TDI can be a Virtual Function (VF) or a Physical Function (PF). |
| 258 | +| TEE VM | TVM | A VM instantiation of an confidential workload. |
| 259 | +| Trusted Device Manager | TDM | A confidential guest TEE-IO device manager, responsible for verifying, attesting and accepting CoVE-IO devices into a TVM TCB. This is a TVM guest software stack component. |
| 260 | + |
0 commit comments