Skip to content

Commit 7853615

Browse files
rgantoisrgantois
committed
docs: Provide some performance indicators for Snagfactory
Provide some recent Snagfactory benchmark results to give potential users an idea of expected performances. Signed-off-by: Romain Gantois <[email protected]>
1 parent 0f5d5c3 commit 7853615

File tree

2 files changed

+128
-0
lines changed

2 files changed

+128
-0
lines changed

‎README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -135,6 +135,8 @@ For recovering and flashing large batches of boards efficiently, you may use the
135135

136136
Note that Snagfactory support is only included in the "gui" package variant: `pip install snagboot[gui]`
137137

138+
Some [benchmark results](https://github.com/bootlin/snagboot/blob/main/docs/snagfactory_benchmarks.md) are provided in the Snagfactory docs.
139+
138140
If you encounter issues, please take a look at the
139141
[troubleshooting](https://github.com/bootlin/snagboot/blob/main/docs/troubleshooting.md) section.
140142

‎docs/snagfactory_benchmarks.md‎

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
# Snagfactory benchmarks
2+
3+
To give an idea of how Snagfactory can perform in terms of flashing speed and parallelization of tasks, some benchmarks results are provided below. The time taken by Snagfactory to complete a factory flashing task depends on many different factors, including but not limited to:
4+
5+
- Processing capabilities of the host machine
6+
- Current system load
7+
- Processing capabilities of the target boards
8+
- Speeds of the USB links from host to targets
9+
- Total amount of image data to write to each target
10+
- USB bus topology
11+
- Type of target storage used
12+
13+
The following benchmarks are meant to provide some basic performance indicators for various types of storage devices.
14+
15+
# Measurement protocol:
16+
17+
Host machine CPU: Intel i5-13500
18+
19+
Host machine memory: 8.00 GB
20+
21+
Operating system: Windows 11
22+
23+
Snagboot version: 2.1 with a minor patch to raise loglevels to "debug" and increase snagflash USB transfer timeouts to 8 minutes
24+
25+
1. Snagfactory is used to flash a randomly generated 64MB image to the chosen storage device on the target.
26+
2. Log timestamps are read from the standard logfiles generated by Snagfactory, and the following durations are computed:
27+
28+
- Total Snagfactory runtime: from `Start` to `BoardPhase.FLASHER -> BoardPhase.DONE`
29+
- Time spent running snagrecover: from `Installing firmware tiboot3` to `Done installing firmware u-boot`
30+
- Time spent running snagflash: from `Fastboot object` to last occurrence of `[DEBUG ] fastboot OKAY`
31+
- Time spent erasing flash blocks (for SPI-NAND and SPI-NOR only): from last occurrence of `erasing flash area` to following occurence of `fastboot OKAY`
32+
- Time spent transmitting image data over USB: from `flashing file random.img` to following occurrence of `fastboot OKAY`
33+
- Time spent writing data to storage medium: from preceding log line to following occurence of `fastboot OKAY`
34+
35+
3. Throughput values are then derived in the following manner:
36+
37+
- transmission throughput: size of flashed image / time spent transmitting over USB
38+
- write throughput: size of flashed image / (time spent writing to storage + time spent erasing storage)
39+
40+
41+
For each type of storage, this measurement protocol was performed three times
42+
with different boards of the same model.
43+
44+
45+
A series of additional total runtime measurements were also performed for the
46+
SPI-NAND case with increasing numbers of boards, to showcase Snagfactory's
47+
ability to parallelize tasks.
48+
49+
# eMMC
50+
51+
Board: AM62x SKEVM
52+
53+
Storage: Micron MTFC16GAPALBH-IT 16GB eMMC
54+
55+
|total runtime (s)|snagrecover runtime (s)|snagflash runtime (s)|
56+
|---|---|---|
57+
|18|9|4|
58+
|18|9|4|
59+
|18|9|4|
60+
61+
62+
63+
NB: The discrepancy between total Snagfactory runtime, and the sum of the other two runtimes is mostly due to the USB reenumeration delay after U-Boot proper is launched on the target
64+
65+
66+
67+
|transmission throughput (MB/s)| write throughput (MB/s)|
68+
|---|---|
69+
|24.701|56.288|
70+
|24.559|57.971|
71+
|24.578|56.437|
72+
73+
# SPI-NAND
74+
75+
Board: AM62x Low-power SKEVM
76+
77+
Storage: Winbond W35N01JWTBAG 128MB OSPI-NAND
78+
79+
|total runtime (s)|snagrecover runtime (s)|snagflash runtime (s)|
80+
|---|---|---|
81+
|25|10|11|
82+
|25|9|11|
83+
|24|9|10|
84+
85+
86+
|transmission throughput (MB/s)| write throughput, including erase cycles (MB/s)|
87+
|---|---|
88+
|24.568|8.854|
89+
|24.530|8.443|
90+
|24.682|9.100|
91+
92+
# OSPI-NOR
93+
94+
Board: AM62x SKEVM
95+
96+
Storage: Infineon S28HS512TGABHM010 64MB OSPI-NOR
97+
98+
|total runtime (s)|snagrecover runtime (s)|snagflash runtime (s)|
99+
|---|---|---|
100+
|426|9|411|
101+
|401|9|387|
102+
|390|9|375|
103+
104+
105+
|transmission throughput (MB/s)| write throughput, including erase cycles (MB/s)|
106+
|---|---|
107+
|25.257|0.157|
108+
|24.625|0.167|
109+
|24.806|0.172|
110+
111+
# Parallelization
112+
113+
Board: AM62x Low-power SKEVM
114+
115+
Storage: Winbond W35N01JWTBAG 128MB OSPI-NAND
116+
117+
|number of boards|total runtime (s)|
118+
|---|---|
119+
|1|25|
120+
|3|25|
121+
|4|26|
122+
|7|27|
123+
|8|27|
124+
125+
NB: these results were obtained with "native" USB ports.
126+

0 commit comments

Comments
 (0)