Skip to content

Commit e5e716d

Browse files
Joel FernandesDanilo Krummrich
Joel Fernandes
authored and
Danilo Krummrich
committed
Documentation: gpu: nova-core: Document fwsec operation and layout
Add explanation of fwsec with diagrams. This helps clarify how the nova-core falcon boot works. Signed-off-by: Joel Fernandes <[email protected]> Signed-off-by: Alexandre Courbot <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Danilo Krummrich <[email protected]>
1 parent 952e622 commit e5e716d

File tree

2 files changed

+182
-0
lines changed

2 files changed

+182
-0
lines changed

‎Documentation/gpu/nova/core/fwsec.rst

Lines changed: 181 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,181 @@
1+
.. SPDX-License-Identifier: (GPL-2.0+ OR MIT)
2+
3+
=========================
4+
FWSEC (Firmware Security)
5+
=========================
6+
This document briefly/conceptually describes the FWSEC (Firmware Security) image
7+
and its role in the GPU boot sequence. As such, this information is subject to
8+
change in the future and is only current as of the Ampere GPU family. However,
9+
hopefully the concepts described will be useful for understanding the kernel code
10+
that deals with it. All the information is derived from publicly available
11+
sources such as public drivers and documentation.
12+
13+
The role of FWSEC is to provide a secure boot process. It runs in
14+
'Heavy-secure' mode, and performs firmware verification after a GPU reset
15+
before loading various ucode images onto other microcontrollers on the GPU,
16+
such as the PMU and GSP.
17+
18+
FWSEC itself is an application stored in the VBIOS ROM in the FWSEC partition of
19+
ROM (see vbios.rst for more details). It contains different commands like FRTS
20+
(Firmware Runtime Services) and SB (Secure Booting other microcontrollers after
21+
reset and loading them with other non-FWSEC ucode). The kernel driver only needs
22+
to perform FRTS, since Secure Boot (SB) has already completed by the time the driver
23+
is loaded.
24+
25+
The FRTS command carves out the WPR2 region (Write protected region) which contains
26+
data required for power management. Once setup, only HS mode ucode can access it
27+
(see falcon.rst for privilege levels).
28+
29+
The FWSEC image is located in the VBIOS ROM in the partition of the ROM that contains
30+
various ucode images (also known as applications) -- one of them being FWSEC. For how
31+
it is extracted, see vbios.rst and the vbios.rs source code.
32+
33+
The Falcon data for each ucode images (including the FWSEC image) is a combination
34+
of headers, data sections (DMEM) and instruction code sections (IMEM). All these
35+
ucode images are stored in the same ROM partition and the PMU table is used to look
36+
up the application to load it based on its application ID (see vbios.rs).
37+
38+
For the nova-core driver, the FWSEC contains an 'application interface' called
39+
DMEMMAPPER. This interface is used to execute the 'FWSEC-FRTS' command, among others.
40+
For Ampere, FWSEC is running on the GSP in Heavy-secure mode and runs FRTS.
41+
42+
FWSEC Memory Layout
43+
-------------------
44+
The memory layout of the FWSEC image is as follows::
45+
46+
+---------------------------------------------------------------+
47+
| FWSEC ROM image (type 0xE0) |
48+
| |
49+
| +---------------------------------+ |
50+
| | PMU Falcon Ucode Table | |
51+
| | (PmuLookupTable) | |
52+
| | +-------------------------+ | |
53+
| | | Table Header | | |
54+
| | | - version: 0x01 | | |
55+
| | | - header_size: 6 | | |
56+
| | | - entry_size: 6 | | |
57+
| | | - entry_count: N | | |
58+
| | | - desc_version:3(unused)| | |
59+
| | +-------------------------+ | |
60+
| | ... | |
61+
| | +-------------------------+ | |
62+
| | | Entry for FWSEC (0x85) | | |
63+
| | | (PmuLookupTableEntry) | | |
64+
| | | - app_id: 0x85 (FWSEC) |----|----+ |
65+
| | | - target_id: 0x01 (PMU) | | | |
66+
| | | - data: offset ---------|----|----|---+ look up FWSEC |
67+
| | +-------------------------+ | | | |
68+
| +---------------------------------+ | | |
69+
| | | |
70+
| | | |
71+
| +---------------------------------+ | | |
72+
| | FWSEC Ucode Component |<---+ | |
73+
| | (aka Falcon data) | | |
74+
| | +-------------------------+ | | |
75+
| | | FalconUCodeDescV3 |<---|--------+ |
76+
| | | - hdr | | |
77+
| | | - stored_size | | |
78+
| | | - pkc_data_offset | | |
79+
| | | - interface_offset -----|----|----------------+ |
80+
| | | - imem_phys_base | | | |
81+
| | | - imem_load_size | | | |
82+
| | | - imem_virt_base | | | |
83+
| | | - dmem_phys_base | | | |
84+
| | | - dmem_load_size | | | |
85+
| | | - engine_id_mask | | | |
86+
| | | - ucode_id | | | |
87+
| | | - signature_count | | look up sig | |
88+
| | | - signature_versions --------------+ | |
89+
| | +-------------------------+ | | | |
90+
| | (no gap) | | | |
91+
| | +-------------------------+ | | | |
92+
| | | Signatures Section |<---|-----+ | |
93+
| | | (384 bytes per sig) | | | |
94+
| | | - RSA-3K Signature 1 | | | |
95+
| | | - RSA-3K Signature 2 | | | |
96+
| | | ... | | | |
97+
| | +-------------------------+ | | |
98+
| | | | |
99+
| | +-------------------------+ | | |
100+
| | | IMEM Section (Code) | | | |
101+
| | | | | | |
102+
| | | Contains instruction | | | |
103+
| | | code etc. | | | |
104+
| | +-------------------------+ | | |
105+
| | | | |
106+
| | +-------------------------+ | | |
107+
| | | DMEM Section (Data) | | | |
108+
| | | | | | |
109+
| | | +---------------------+ | | | |
110+
| | | | Application | |<---|----------------+ |
111+
| | | | Interface Table | | | |
112+
| | | | (FalconAppifHdrV1) | | | |
113+
| | | | Header: | | | |
114+
| | | | - version: 0x01 | | | |
115+
| | | | - header_size: 4 | | | |
116+
| | | | - entry_size: 8 | | | |
117+
| | | | - entry_count: N | | | |
118+
| | | | | | | |
119+
| | | | Entries: | | | |
120+
| | | | +-----------------+ | | | |
121+
| | | | | DEVINIT (ID 1) | | | | |
122+
| | | | | - id: 0x01 | | | | |
123+
| | | | | - dmemOffset X -|-|-|----+ |
124+
| | | | +-----------------+ | | | |
125+
| | | | +-----------------+ | | | |
126+
| | | | | DMEMMAPPER(ID 4)| | | | |
127+
| | | | | - id: 0x04 | | | | Used only for DevInit |
128+
| | | | | (NVFW_FALCON_ | | | | application (not FWSEC) |
129+
| | | | | APPIF_ID_DMEMMAPPER) | |
130+
| | | | | - dmemOffset Y -|-|-|----|-----+ |
131+
| | | | +-----------------+ | | | | |
132+
| | | +---------------------+ | | | |
133+
| | | | | | |
134+
| | | +---------------------+ | | | |
135+
| | | | DEVINIT Engine |<|----+ | Used by FWSEC |
136+
| | | | Interface | | | | app. |
137+
| | | +---------------------+ | | | |
138+
| | | | | | |
139+
| | | +---------------------+ | | | |
140+
| | | | DMEM Mapper (ID 4) |<|----+-----+ |
141+
| | | | (FalconAppifDmemmapperV3) | |
142+
| | | | - signature: "DMAP" | | | |
143+
| | | | - version: 0x0003 | | | |
144+
| | | | - Size: 64 bytes | | | |
145+
| | | | - cmd_in_buffer_off | |----|------------+ |
146+
| | | | - cmd_in_buffer_size| | | | |
147+
| | | | - cmd_out_buffer_off| |----|------------|-----+ |
148+
| | | | - cmd_out_buffer_sz | | | | | |
149+
| | | | - init_cmd | | | | | |
150+
| | | | - features | | | | | |
151+
| | | | - cmd_mask0/1 | | | | | |
152+
| | | +---------------------+ | | | | |
153+
| | | | | | | |
154+
| | | +---------------------+ | | | | |
155+
| | | | Command Input Buffer|<|----|------------+ | |
156+
| | | | - Command data | | | | |
157+
| | | | - Arguments | | | | |
158+
| | | +---------------------+ | | | |
159+
| | | | | | |
160+
| | | +---------------------+ | | | |
161+
| | | | Command Output |<|----|------------------+ |
162+
| | | | Buffer | | | |
163+
| | | | - Results | | | |
164+
| | | | - Status | | | |
165+
| | | +---------------------+ | | |
166+
| | +-------------------------+ | |
167+
| +---------------------------------+ |
168+
| |
169+
+---------------------------------------------------------------+
170+
171+
.. note::
172+
This is using an GA-102 Ampere GPU as an example and could vary for future GPUs.
173+
174+
.. note::
175+
The FWSEC image also plays a role in memory scrubbing (ECC initialization) and VPR
176+
(Video Protected Region) initialization as well. Before the nova-core driver is even
177+
loaded, the FWSEC image is running on the GSP in heavy-secure mode. After the devinit
178+
sequence completes, it does VRAM memory scrubbing (ECC initialization). On consumer
179+
GPUs, it scrubs only part of memory and then initiates 'async scrubbing'. Before this
180+
async scrubbing completes, the unscrubbed VRAM cannot be used for allocation (thus DRM
181+
memory allocators need to wait for this scrubbing to complete).

‎Documentation/gpu/nova/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,3 +30,4 @@ vGPU manager VFIO driver and the nova-drm driver.
3030
core/todo
3131
core/vbios
3232
core/devinit
33+
core/fwsec

0 commit comments

Comments
 (0)