Verified Hardware Builds and Benches #101

nothingismagick · 2020-12-23T10:21:16Z

nothingismagick
Dec 23, 2020

This Discussion is a Tracker for the verification that Stronghold compiles and benches on a matrix of hardware.

Please use this as a template but replace with your details and results:

| Device | CPU | OS & Version | Notes |
| --- | --- | --- | --- |
| MacBook Pro (15-inch, 2018) | 2,2 GHz 6-Core Intel Core i7 | macOS 10.15.7 (19H15) | Stable |

Rust Stable Version: rustc 1.49.0 (e1884a8e3 2020-12-29)
Branch: dev

Results from cargo bench:

write to stronghold while creating vaults
                        time:   [11.697 ms 12.211 ms 12.735 ms]

read from stronghold    time:   [841.30 us 858.65 us 877.47 us]
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe

write to stronghold while initializing records
                        time:   [12.049 ms 12.480 ms 12.929 ms]
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild

Write to snapshot       time:   [16.151 ms 17.071 ms 18.021 ms]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild

Read from snapshot      time:   [1.1005 ms 1.1212 ms 1.1480 ms]
Found 7 outliers among 100 measurements (7.00%)
  4 (4.00%) high mild

gallegogt · 2021-01-22T12:03:45Z

gallegogt
Jan 22, 2021

Device	CPU	OS & Version	Notes
MacBook Pro (13-inch, 2016)	2,9 GHz Dual-Core Intel Core i5	macOS 11.1 (20C69)	Stable

Rust Stable Version: rustc 1.49.0 (e1884a8e3 2020-12-29)
Branch: dev

Results from cargo bench:

Gnuplot not found, using plotters backend
write to stronghold while creating vaults                                                                             
                        time:   [11.184 ms 11.708 ms 12.236 ms]
                        change: [-4.3469% +1.7950% +8.2257%] (p = 0.57 > 0.05)
                        No change in performance detected.

write to stronghold while initializing records                                                                             
                        time:   [10.788 ms 11.244 ms 11.719 ms]
                        change: [-14.629% -9.1287% -3.0907%] (p = 0.00 < 0.05)
                        Performance has improved.

Write to snapshot       time:   [13.016 ms 13.226 ms 13.435 ms]                              
                        change: [-1.2199% +1.0851% +3.3844%] (p = 0.37 > 0.05)
                        No change in performance detected.

Benchmarking Read from snapshot: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 6.3s, enable flat sampling, or reduce sample count to 60.
Read from snapshot      time:   [1.1644 ms 1.1778 ms 1.1937 ms]                                
                        change: [+0.3959% +2.6632% +5.4205%] (p = 0.04 < 0.05)
                        Change within noise threshold.
Found 14 outliers among 100 measurements (14.00%)
  6 (6.00%) high mild
  8 (8.00%) high severe

Bench write to store    time:   [166.94 us 170.10 us 173.08 us]                                 
                        change: [-5.3944% -2.9013% -0.2785%] (p = 0.03 < 0.05)
                        Change within noise threshold.
Found 14 outliers among 100 measurements (14.00%)
  1 (1.00%) low severe
  8 (8.00%) low mild
  2 (2.00%) high mild
  3 (3.00%) high severe

Bench read from store   time:   [169.55 us 173.95 us 178.55 us]                                  
                        change: [-12.113% -7.9100% -3.5956%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
  4 (4.00%) high mild

0 replies

yolierrbd · 2021-01-22T12:21:25Z

yolierrbd
Jan 22, 2021

Device	CPU	OS & Version	Notes
Lenovo Thinkpad (2020)	Intel® Xeon(R) E-2276M CPU @ 2.80GHz × 12	Manjaro Linux; Build ID: rolling	Stable

Rust Stable Version: rustc 1.49.0 (e1884a8e3 2020-12-29)
Branch: dev

Results from cargo bench:**

write to stronghold while creating vaults                                                                            
                        time:   [8.4778 ms 9.1605 ms 9.8620 ms]

write to stronghold while initializing records                                                                            
                        time:   [7.4244 ms 8.1053 ms 8.8184 ms]

Write to snapshot       time:   [6.6276 ms 6.7171 ms 6.8000 ms]                              
Found 11 outliers among 100 measurements (11.00%)
  4 (4.00%) low severe
  1 (1.00%) low mild
  4 (4.00%) high mild
  2 (2.00%) high severe

Read from snapshot      time:   [2.5727 ms 2.7506 ms 2.9113 ms]                                
Found 30 outliers among 100 measurements (30.00%)
  19 (19.00%) low severe
  11 (11.00%) high severe

Bench write to store    time:   [57.038 us 57.247 us 57.589 us]                                 
Found 7 outliers among 100 measurements (7.00%)
  2 (2.00%) low mild
  3 (3.00%) high mild
  2 (2.00%) high severe

Bench read from store   time:   [56.783 us 57.053 us 57.423 us]                                  
Found 11 outliers among 100 measurements (11.00%)
  1 (1.00%) low mild
  4 (4.00%) high mild
  6 (6.00%) high severe

0 replies

gallegogt · 2021-01-23T03:03:05Z

gallegogt
Jan 23, 2021

Device	CPU	OS & Version	Notes
RaspberryPi Zero W	1GHz, single-core CPU	2021-01-11-raspios-buster-armhf-lite	Stable

Rust Stable Version: rustc 1.49.0 (e1884a8e3 2020-12-29)
Branch: dev

Gnuplot not found, using plotters backend
write to stronghold while creating vaults                                                                            
                        time:   [50.509 ms 51.831 ms 53.160 ms]

write to stronghold while initializing records                                                                            
                        time:   [53.221 ms 54.584 ms 55.955 ms]

Write to snapshot       time:   [28.629 ms 32.118 ms 38.398 ms]                              
Found 5 outliers among 100 measurements (5.00%)
  5 (5.00%) high severe

Read from snapshot      time:   [18.405 ms 18.445 ms 18.486 ms]                               
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild

Bench write to store    time:   [3.2762 ms 3.2840 ms 3.2946 ms]                                  
Found 7 outliers among 100 measurements (7.00%)
  2 (2.00%) high mild
  5 (5.00%) high severe

Bench read from store   time:   [3.2833 ms 3.2938 ms 3.3050 ms]                                   
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild

0 replies

nothingismagick · 2021-02-04T15:20:13Z

nothingismagick
Feb 4, 2021
Author

Device	CPU	OS & Version	Notes
MacBook Air (13-inch, 2020)	8-Core M1	macOS 11.1	Stable

Rust Stable Version: rustc 1.49.0 (e1884a8e3 2020-12-29)
Branch: dev

write to stronghold while creating vaults                        
                        time:   [12.583 ms 12.904 ms 13.226 ms]
                        
write to stronghold while initializing records                        
                        time:   [12.519 ms 12.797 ms 13.074 ms]

Write to snapshot       time:   [22.257 ms 22.524 ms 22.835 ms]     

Read from snapshot      time:   [669.55 us 672.25 us 675.80 us] 

Bench write to store    time:   [73.377 us 73.925 us 74.634 us]

Bench read from store   time:   [73.901 us 74.038 us 74.180 us]

compress and decompress data                        
                        time:   [79.428 us 79.920 us 80.472 us]
                        
compress data           time:   [44.608 us 45.129 us 45.675 us] 

decompress data         time:   [29.292 us 29.736 us 30.352 us]     

Write to cache          time:   [59.239 ns 59.361 ns 59.488 ns]     

Read from cache         time:   [106.88 ns 108.64 ns 110.50 ns] 

Read from Cached function                        
                        time:   [57.180 ns 57.334 ns 57.514 ns]

write to engine         time:   [12.901 us 12.984 us 13.087 us]

0 replies

elenaf9 · 2021-02-25T00:22:20Z

elenaf9
Feb 25, 2021

Device	CPU	OS & Version	Notes
F-Secure USB Armory Mk II	NXP i.MX6ULZ ARM® Cortex™-A7 900 MHz	Debian GNU/Linux 10 (buster)

Rust Stable Version: rustc 1.49.0 (e1884a8e3 2020-12-29)
Branch: dev

    Running target/arm-unknown-linux-gnueabihf/release/deps/benchmark-25f665d1409175b1

write to stronghold while creating vaults  
                        time:   [37.795 ms 39.194 ms 40.596 ms]

write to stronghold while initializing records       
                        time:   [39.184 ms 40.569 ms 41.961 ms]

Write to snapshot       time:   [27.736 ms 28.931 ms 30.395 ms]     

Read from snapshot      time:   [20.182 ms 20.398 ms 20.636 ms]   

Bench write to store    time:   [2.2996 ms 2.3318 ms 2.3699 ms] 

Bench read from store   time:   [2.2856 ms 2.3132 ms 2.3439 ms]
  
     Running target/arm-unknown-linux-gnueabihf/release/deps/benchmark-43ad07bb8b895c8e

compress and decompress data 
                        time:   [1.3948 ms 1.4131 ms 1.4336 ms]

compress data           time:   [1.0684 ms 1.0825 ms 1.0991 ms]

decompress data         time:   [312.06 us 316.01 us 320.82 us]

     Running target/arm-unknown-linux-gnueabihf/release/deps/benchmark-cc9f728e851dda17

Write to cache          time:   [1.6128 us 1.6326 us 1.6555 us]  

Read from cache         time:   [1.9984 us 2.0229 us 2.0506 us]

Read from Cached function                                                                             
                        time:   [1.6094 us 1.6280 us 1.6492 us]

     Running target/arm-unknown-linux-gnueabihf/release/deps/benchmark-990d1fc0755de6bf

write to engine         time:   [135.84 us 137.65 us 139.77 us]

Additional Notes:

compiling the benchmarks on the USB Armory for the first time with all dependencies can take up to a full day
executing the bench command takes ~1h.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Verified Hardware Builds and Benches #101

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 5 comments

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Select a reply

Uh oh!

Verified Hardware Builds and Benches #101

Uh oh!

Uh oh!

nothingismagick Dec 23, 2020

Replies: 5 comments

Uh oh!

Uh oh!

gallegogt Jan 22, 2021

Uh oh!

yolierrbd Jan 22, 2021

Uh oh!

gallegogt Jan 23, 2021

Uh oh!

nothingismagick Feb 4, 2021 Author

Uh oh!

Uh oh!

elenaf9 Feb 25, 2021

nothingismagick
Dec 23, 2020

gallegogt
Jan 22, 2021

yolierrbd
Jan 22, 2021

gallegogt
Jan 23, 2021

nothingismagick
Feb 4, 2021
Author

elenaf9
Feb 25, 2021