|
| 1 | +# Trusted Device Manager Architecture |
| 2 | + |
| 3 | +This is an architecture document for the Trusted Device Manager (TDM), a |
| 4 | +confidential guest, userspace component for implementing trusted I/O. |
| 5 | + |
| 6 | +## Goals |
| 7 | + |
| 8 | +In order to achieve high performance Input/Output (I/O) operations, confidential |
| 9 | +VMs must extend their trust boundary to include a composition of directly |
| 10 | +assigned, TEE-IO-capable devices/functions. Without this ability, confidential |
| 11 | +VMs have to resort to using para-virtualized I/O using non-confidential memory |
| 12 | +regions, which has performance impacts due to memory copies and negates the use |
| 13 | +of directly assigned devices. |
| 14 | + |
| 15 | +Silicon vendors and device manufacturers provide TEE-IO compliant hardware |
| 16 | +implementations. Intel TDX Connect or AMD SEV-TIO, for example, rely on device |
| 17 | +manufacturers to support the TDISP, SPDM and PCIe IDE protocols to give |
| 18 | +confidential guests access to trusted MMIO and secure DMA between them and |
| 19 | +devices which trustworthiness is verifiable. |
| 20 | + |
| 21 | +The TEE-IO security model is built on top of this hardware support but also on |
| 22 | +the ability for confidential guests to ultimately decide if they accept or |
| 23 | +reject TEE-IO compliant devices into their TCB. The TDM, as a confidential guest |
| 24 | +component, is the TCB acceptance decision maker and is responsible for notifying |
| 25 | +the underlying Trusted Security Manager (TSM) about its decision. By accepting a |
| 26 | +TDI into the guest TCB, the TDM explicitly requests the TSM to move the device |
| 27 | +into an operational state. The TDM builds the acceptance decision after having |
| 28 | +verified and authenticated the TDI. It must also attest of the assigned device |
| 29 | +trustworthiness, through a local or remote attestation process. |
| 30 | + |
| 31 | +## Scope |
| 32 | + |
| 33 | +## Architectural Overview |
| 34 | + |
| 35 | +High-level view: |
| 36 | + |
| 37 | +``` |
| 38 | +
|
| 39 | +
|
| 40 | + ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ |
| 41 | + ┃ Confidential Guest ┃ |
| 42 | + ┃ ╔═══════════════════════╗ ┃ |
| 43 | + ┃ ║Trusted Device Manager ║ ┃ |
| 44 | + ┃ ║ ║ ┃ |
| 45 | + ┃ ║┌─────────────────────┐║ ┃ |
| 46 | + ┃ ║│ Attestation Module │║ ┃ |
| 47 | + ┃ ┌───▷║│ │║ ┃ |
| 48 | + ┃ │ ║└─────────────────────┘║ ┃ |
| 49 | + ┃ │ ║ ║ ┃ |
| 50 | + ┃ │ ║ ╔════════╗ ╔════════╗ ║ ┃ |
| 51 | + ┃ │ ╚═╣ TEE ╠═╣ TEE ╠═╝ ┃ |
| 52 | + ┃ │ ║ Plugin ║ ║ Plugin ║ ┃ |
| 53 | + ┃ udev ╚════════╝ ╚════════╝ ┃ |
| 54 | + ┃ (PCIe device △ ┃ |
| 55 | + ┃ detection) │ ┃ |
| 56 | + ┃ │ Device Evidence ┃ |
| 57 | + ┃ │ Accept/Reject ┃ |
| 58 | + ┃ │ │ ┃ |
| 59 | + ┃ │ ▽ ┃ |
| 60 | + ┃ │ ┌───────┐ ┃ |
| 61 | + ┃ │ ┌───────────────┤TEE ABI├────┐ ┃ |
| 62 | + ┃ │ │ Linux Guest └───────┘ │ ┃ |
| 63 | + ┃ └──│ Kernel │ ┃ |
| 64 | + ┃ └────────────────────────────┘ ┃ |
| 65 | + ┗━━━━━━━━━━━━━━━━━━━━━━━━△━━━━━━━━━━━━━━━━━━━━━━┛ |
| 66 | + │ |
| 67 | + │ |
| 68 | + │ |
| 69 | + TEE-IO ABI |
| 70 | + │ |
| 71 | + │ |
| 72 | + ▽ |
| 73 | +┏━━━━━━━━━━━━━━━━━━━━━━━┓ ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ |
| 74 | +┃ ┃ ┃ ┃ |
| 75 | +┃ Host VMM/Hypervisor ┃ ┃ Trusted Security Manager ┃ |
| 76 | +┃ (KVM+QEMU/CLH) ┃◁──────TEE ABI─────▷┃ (TDX Module, ASP, etc) ┃ |
| 77 | +┃ ┃ ┃ ┃ |
| 78 | +┗━━━━━━━━━━━━━━━━━━━━━━━┛ ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛ |
| 79 | + △ |
| 80 | + ┃ |
| 81 | + PCIe Bus |
| 82 | + ┃ |
| 83 | + ┃ |
| 84 | + ▽ |
| 85 | +┏━━━━━━━━━━━━━━━━━━━━━━━┓ |
| 86 | +┃ TEE-IO Compliant PCIe ┃ |
| 87 | +┃ Device ┃ |
| 88 | +┗━━━━━━━━━━━━━━━━━━━━━━━┛ |
| 89 | +
|
| 90 | +``` |
| 91 | + |
| 92 | +In order for the TDM to achieve the above described goals, it must be able to: |
| 93 | + |
| 94 | +1. Detect VMM-assigned PCIe devices. This relies on the Linux kernel `udev` |
| 95 | + subsystem. TDM also assumes the TSM must not enable any DMA or MMIO mappings |
| 96 | + before it accepted the device. |
| 97 | +2. Authenticate the assigned device. Both authentication and attestation of the |
| 98 | + TDI relies on the ability for the TDM to retrieve a certificate chain and an |
| 99 | + attestation evidence. This is provided by the Linux kernel confidential |
| 100 | + computing ABI, which is generally exposed as a vendor-specific, ioctl-based |
| 101 | + character device. |
| 102 | +3. Verify the assigned device. Before accepting a TDI into the confidential |
| 103 | + guest TCB, the TDM must use the Linux kernel CoCo ABI to verify the following |
| 104 | + security attributes from the TSM: |
| 105 | + * Device link state |
| 106 | + * TDI state |
| 107 | + * TDI MMIO mappings |
| 108 | +4. Attest of the assigned device trustworthiness. As part of the overall device |
| 109 | + verification process, the TDM must run a local or remote attestation of the |
| 110 | + device attestation evidence: |
| 111 | + 1. For running local device attestation, the TDM needs to practically become a |
| 112 | + Relying Party. |
| 113 | + 2. In order to support remote device attestation, the TDM would rely on |
| 114 | + existing guest components like the attestation agent. |
| 115 | +5. Notify the TSM of it TDI acceptance decision. Here as well, this relies on |
| 116 | + the Linux kernel CoCo ABI. |
| 117 | + |
| 118 | +This leads to a natural split of the TDM software architecture into 4 main |
| 119 | +modules: |
| 120 | + |
| 121 | +1. The **TDM Core** runs the main TDM flows and managing confidential guest |
| 122 | + assignment with the support of the detection, ABI plugins and attestation |
| 123 | + modules. |
| 124 | +2. The **Device Detection** module traps device discovery, assignment and |
| 125 | + removal from the confidential guest PCIe emulated bus. |
| 126 | +3. The **Linux CoCo ABI plugins** abstracts the vendor specific CoCo Linux |
| 127 | + kernel ABIs in order expose a vendor-agnostic internal interface for the TDM |
| 128 | + core to consume. |
| 129 | +4. The **Device Attestation** module is responsible for verifying a device |
| 130 | + attestation evidence, which can be achieved through local or remote |
| 131 | + attestation. This module acts as a device attestation relying party to |
| 132 | + support the former, while it relies on the Attestation Agent API to support |
| 133 | + the latter case. It provides attestation results back to the TDM core. |
| 134 | + |
| 135 | +``` |
| 136 | +
|
| 137 | + ┌────────────────────────┐ |
| 138 | + │ Trusted Device Manager │ |
| 139 | + ┌───────────────┴────────────────────────┴────────────┐ |
| 140 | + │ ┌──────────┐ │ |
| 141 | + │ │Reference │ │ |
| 142 | + │ ┌───────────┐ ┌───────────┐ ┌─│ Values │ │ |
| 143 | + │ │ Device │ │ Device │ │ └──────────┘ │ |
| 144 | + ┏━━╋━━━━━━▶│ Detection │ │Attestation│◁┤ ┌───────────┐ │ |
| 145 | + ┃ │ │ Module │ │ Module │ │ │Attestation│ │ |
| 146 | + ┃ │ └───────────┘ └───────────┘ └─│ Policies │ │ |
| 147 | + ┃ │ △ △ └───────────┘ │ |
| 148 | + ┃ │ └──────┐┌─────┘ │ |
| 149 | + ┃ │ ▽▽ │ |
| 150 | + ┃ │ ┌──────────────┐ │ |
| 151 | + ┃ │ │ │ │ |
| 152 | + ┃ │ │ Core Module │ │ |
| 153 | + ┃ │ │ │ │ |
| 154 | + ┃ │ └──────────────┘ │ |
| 155 | + ┃ │ △ │ |
| 156 | + PCIe │ └────┐ │ |
| 157 | + udev │ │ │ |
| 158 | +events│ ▽ │ |
| 159 | + ┃ │ ┌─────────────────────────────────┐ │ |
| 160 | + ┃ │ │ ABI Plugins Module │ │ |
| 161 | + ┃ │ └─────────────────────────────────┘ │ |
| 162 | + ┃ │ ┌──────────┐┌──────────┐┌─────────┐ │ |
| 163 | + ┃ │ │TDX Plugin││SEV Plugin││SE Plugin│ │ |
| 164 | + ┃ │ └──────────┘└──────────┘└─────────┘ │ |
| 165 | + ┃ │ ┃ │ |
| 166 | + ┃ └──────────────────────────╋──────────────────────────┘ |
| 167 | + ┃ ┃ |
| 168 | + ┃ CoCo ABI |
| 169 | + ┃ ┃ |
| 170 | + ┃ ▼ |
| 171 | + ┃ ┌─────────────────────────────────────────────────────┐ |
| 172 | + ┗━━│ Linux Guest Kernel │ |
| 173 | + └─────────────────────────────────────────────────────┘ |
| 174 | +``` |
| 175 | + |
| 176 | +### TDM Core |
| 177 | + |
| 178 | +The TDM core module interacts with three other TDM components: |
| 179 | + |
| 180 | +1. The device detection module. It monitors udev events to detect PCI devices |
| 181 | + addition and removal from the guest emulated PCI bus. This module calls into |
| 182 | + the TDM core to notify it about both event types. The TDM core then relies on |
| 183 | + the following two modules to verify, attest and accept newly detected |
| 184 | + devices. Only TDM-accepted TDIs are visible and available to the rest of the |
| 185 | + guest software stack. |
| 186 | +2. The ABI plugins module. This is an abstraction over the Linux CoCo |
| 187 | + vendor-specific ABI and the TDM core calls into this module in order to |
| 188 | + communicate with the TSM. The ABI plugins module abstracts the |
| 189 | + vendor-specific TSM interfaces in order for the TDM to: |
| 190 | + * Retrieve a certificate chain for any newly detected device. |
| 191 | + * Retrieve an attestation evidence for any newly detected device. This is |
| 192 | + optional as the evidence may be delivered as part of the device certificate |
| 193 | + chain. |
| 194 | + * Check the physical device link state and verify that it is confidentiality |
| 195 | + and integrity protected, as provided by the PCI IDE protocol. The TSM |
| 196 | + manages the IDE keys and must track the device physical link state for any |
| 197 | + assigned devices. |
| 198 | + * Check for the TDI state in order to verify that it is in the (TDISP) locked |
| 199 | + state. |
| 200 | + * Check that all emulated MMIO ranges are correctly (in respect to their size |
| 201 | + and order in the guest address space) mapped. |
| 202 | + * Accept or reject an assigned TDI. |
| 203 | +3. The device attestation module. Once the TDM has authenticated and verified an |
| 204 | + assigned TDI, it can attest it through local or remote attestation before |
| 205 | + finally accepting it. The TDM core fetches an attestation evidence through |
| 206 | + the ABI plugin module and forwards it to the device attestation module. This |
| 207 | + module then verifies the device evidence either locally or remotely, depending |
| 208 | + on a guest-specific TDM policy, and returns an attestation results back to the |
| 209 | + TDM core. |
| 210 | + |
| 211 | +### Device Detection |
| 212 | + |
| 213 | +### TEE ABI Plugins |
| 214 | + |
| 215 | + |
| 216 | + |
| 217 | +#### Plugin Interface |
| 218 | + |
| 219 | +A Rust trait called by the TDM core to interact with the TSM. |
| 220 | +(Not needed if Linux provides a converged TEE userspace API. |
| 221 | + |
| 222 | +``` rust |
| 223 | + |
| 224 | + |
| 225 | + |
| 226 | +``` |
| 227 | + |
| 228 | +### Device Attestation |
| 229 | + |
| 230 | +## Theory of Operations |
| 231 | + |
| 232 | +### TEE-IO Acceptance Flow |
| 233 | + |
| 234 | +#### Detection |
| 235 | + |
| 236 | +#### TDI Security Attributes Check |
| 237 | + |
| 238 | +#### Device Attestation |
| 239 | + |
| 240 | +##### Local Attestation |
| 241 | + |
| 242 | +TDM is a Relying Party. |
| 243 | +TDM to use part of the Attestation Service crates |
| 244 | + |
| 245 | +##### Remote Attestation |
| 246 | + |
| 247 | +## Glossary |
| 248 | + |
| 249 | +| Term | Acronym | Definition | |
| 250 | +|:-----|:-------:|------------| |
| 251 | +| Confidential Computing | CoCo | The protection of data in use by performing computation in a Hardware-based TEE. |
| 252 | +| Confidential VM | CVM | A confidential computing virtual machine. Same as a TVM. |
| 253 | +| Device Interface | DI | TDISP term, Device Interface. Same as a TDI. A DI can be a Virtual Function (VF) or a Physical Function (PF). |
| 254 | +| Device Security Manager | DSM | A DSM is a logical entity on a TEE-IO device that enforces the TDISP security policies, attributes and states. |
| 255 | +| Integrity and Data Encryption | IDE | Extended PCIe capability for integrity, confidentiality and replay protection of PCIe Transport Layer Packets (TLP). |
| 256 | +| Secure Protocol and Data Model | SPDM | A DMTF defined specification for exchanging messages between devices over a variety of transports and physical media. SPDM is used to exchange TDISP and IDE Key Management messages over PCIe DOE mailboxes. |
| 257 | +| TEE Device Interface Security Protocol | TDISP | An architecture for trusted I/O virtualization. |
| 258 | +| TEE Input and Output | TEE-IO | A PCIe-defined conceptual framework for establishing and managing trust relationships between a PCIe device and a TEE. |
| 259 | +| TEE I/O Device Interface | TDI | The unit of assignment for a trusted I/O capable device. For example, a TDI can be a Virtual Function (VF) or a Physical Function (PF). |
| 260 | +| TEE VM | TVM | A VM instantiation of an confidential workload. |
| 261 | +| Trusted Device Manager | TDM | A confidential guest TEE-IO device manager, responsible for verifying, attesting and accepting CoVE-IO devices into a TVM TCB. This is a TVM guest software stack component. |
| 262 | + |
0 commit comments