Weekly benchmark report - 2023

[fcollonval]

This issue gathers the results of the weekly scheduled benchmark tests.

Those test are run against JupyterLab master comparing the HEAD to the commit one week older than the HEAD; so every benchmark test is the evolution compared to the previous test.

For more information please refer to the documentation.

Older reports are in: #85

[github-actions]

Benchmark report

The execution time (in milliseconds) are grouped by test file, test type and browser.
For each case, the following values are computed: min <- [1st quartile - median - 3rd quartile] -> max.

The mean relative comparison is computed with 95% confidence.

Results table

Test file	errorOutputs - 10x100 errors output collected
open
chromium
head	466 <- [492 - 503 - 515] -> 660
expected	461 <- [486 - 501 - 515] -> 674
Mean relative change	0.1% ± 1.6%
switch-from-copy
chromium
head	118 <- [142 - 156 - 217] -> 398
expected	110 <- [146 - 164 - 215] -> 411
Mean relative change	-1.7% ± 4.6%
switch-to-copy
chromium
head	503 <- [506 - 509 - 514] -> 541
expected	503 <- [506 - 509 - 514] -> 535
Mean relative change	0.0% ± 0.2%
switch-from-txt
chromium
head	76 <- [97 - 105 - 114] -> 202
expected	65 <- [100 - 110 - 122] -> 187
Mean relative change	-3.9% ± 2.4%
switch-to-txt
chromium
head	91 <- [111 - 121 - 141] -> 279
expected	95 <- [111 - 119 - 137] -> 270
Mean relative change	1.3% ± 3.7%
search
chromium
head	729 <- [794 - 805 - 829] -> 903
expected	756 <- [793 - 807 - 830] -> 918
Mean relative change	-0.2% ± 1.0%
close
chromium
head	144 <- [172 - 186 - 201] -> 267
expected	159 <- [177 - 187 - 198] -> 283
Mean relative change	-0.9% ± 3.9%

Changes are computed with expected as reference.

Expected = 8d42fc0af0e5579aea881680d6e169a24fa56c69 | Actual = 282ec9129964ccefec362c61eaef6340f9274eb2
Go to action log
Changelog covered

❗ Test metadata have changed

--- /dev/fd/63	2023-01-15 02:52:48.088779107 +0000
+++ /dev/fd/62	2023-01-15 02:52:48.088779107 +0000
@@ -1,7 +1,7 @@
 {
   "benchmark": {
     "BENCHMARK_OUTPUTFILE": "lab-benchmark.json",
-    "BENCHMARK_REFERENCE": "head"
+    "BENCHMARK_REFERENCE": "actual"
   },
   "browsers": {
     "chromium": "109.0.5414.46"

[github-actions]

Benchmark report

The execution time (in milliseconds) are grouped by test file, test type and browser.
For each case, the following values are computed: min <- [1st quartile - median - 3rd quartile] -> max.

The mean relative comparison is computed with 95% confidence.

Results table

Test file	large_md_100_notebook
open
chromium
head	729 <- [827 - 862 - 893] -> 1322
expected	722 <- [829 - 851 - 871] -> 944
Mean relative change	2.8% ± 2.2%
switch-from-copy
chromium
head	353 <- [383 - 427 - 604] -> 1090
expected	346 <- [386 - 432 - 590] -> 1013
Mean relative change	0.1% ± 5.2%
switch-to-copy
chromium
head	511 <- [527 - 537 - 545] -> 728
expected	504 <- [524 - 533 - 545] -> 644
Mean relative change	0.6% ± 0.6%
switch-from-txt
chromium
head	142 <- [169 - 183 - 308] -> 558
expected	140 <- [168 - 181 - 312] -> 502
Mean relative change	-0.0% ± 5.7%
switch-to-txt
chromium
head	260 <- [294 - 314 - 422] -> 684
expected	259 <- [296 - 317 - 436] -> 634
Mean relative change	-0.7% ± 3.9%
search
chromium
head	1166 <- [1300 - 1360 - 1429] -> 1600
expected	1128 <- [1287 - 1360 - 1420] -> 1571
Mean relative change	0.6% ± 1.9%
close
chromium
head	239 <- [265 - 277 - 296] -> 457
expected	242 <- [261 - 273 - 287] -> 384
Mean relative change	3.3% ± 3.4%

Changes are computed with expected as reference.

Expected = 8d42fc0af0e5579aea881680d6e169a24fa56c69 | Actual = 282ec9129964ccefec362c61eaef6340f9274eb2
Go to action log
Changelog covered

❗ Test metadata have changed

--- /dev/fd/63	2023-01-15 03:15:18.327388232 +0000
+++ /dev/fd/62	2023-01-15 03:15:18.327388232 +0000
@@ -1,7 +1,7 @@
 {
   "benchmark": {
     "BENCHMARK_OUTPUTFILE": "lab-benchmark.json",
-    "BENCHMARK_REFERENCE": "head"
+    "BENCHMARK_REFERENCE": "actual"
   },
   "browsers": {
     "chromium": "109.0.5414.46"

[github-actions]

Benchmark report

The execution time (in milliseconds) are grouped by test file, test type and browser.
For each case, the following values are computed: min <- [1st quartile - median - 3rd quartile] -> max.

The mean relative comparison is computed with 95% confidence.

Results table

Test file	large_code_100_notebook
open
chromium
head	567 <- [617 - 637 - 660] -> 784
expected	577 <- [616 - 635 - 661] -> 748
Mean relative change	0.2% ± 1.5%
switch-from-copy
chromium
head	415 <- [453 - 497 - 956] -> 1688
expected	412 <- [457 - 485 - 971] -> 1724
Mean relative change	-0.5% ± 8.4%
switch-to-copy
chromium
head	516 <- [593 - 613 - 631] -> 717
expected	525 <- [598 - 612 - 632] -> 730
Mean relative change	-0.4% ± 0.8%
switch-from-txt
chromium
head	130 <- [175 - 192 - 222] -> 651
expected	142 <- [179 - 193 - 228] -> 458
Mean relative change	-0.9% ± 5.2%
switch-to-txt
chromium
head	210 <- [243 - 256 - 272] -> 434
expected	214 <- [243 - 254 - 271] -> 450
Mean relative change	0.9% ± 1.9%
search
chromium
head	823 <- [987 - 1016 - 1046] -> 1332
expected	836 <- [971 - 995 - 1035] -> 1289
Mean relative change	0.8% ± 2.1%
close
chromium
head	227 <- [263 - 280 - 299] -> 406
expected	233 <- [268 - 283 - 298] -> 391
Mean relative change	-0.7% ± 2.7%

Changes are computed with expected as reference.

Expected = 8d42fc0af0e5579aea881680d6e169a24fa56c69 | Actual = 282ec9129964ccefec362c61eaef6340f9274eb2
Go to action log
Changelog covered

❗ Test metadata have changed

--- /dev/fd/63	2023-01-15 03:28:01.297523434 +0000
+++ /dev/fd/62	2023-01-15 03:28:01.297523434 +0000
@@ -1,7 +1,7 @@
 {
   "benchmark": {
     "BENCHMARK_OUTPUTFILE": "lab-benchmark.json",
-    "BENCHMARK_REFERENCE": "head"
+    "BENCHMARK_REFERENCE": "actual"
   },
   "browsers": {
     "chromium": "109.0.5414.46"

[github-actions]

Benchmark report

The execution time (in milliseconds) are grouped by test file, test type and browser.
For each case, the following values are computed: min <- [1st quartile - median - 3rd quartile] -> max.

The mean relative comparison is computed with 95% confidence.

Results table

Test file	large_metadata_1000x100_notebook
open
chromium
head	3957 <- [4023 - 4045 - 4072] -> 4145
expected	4030 <- [4082 - 4101 - 4126] -> 4517
Mean relative change	-1.5% ± 0.3%
switch-from-copy
chromium
head	150 <- [204 - 229 - 338] -> 726
expected	145 <- [208 - 244 - 343] -> 725
Mean relative change	-3.7% ± 6.5%
switch-to-copy
chromium
head	503 <- [504 - 505 - 508] -> 803
expected	503 <- [505 - 505 - 508] -> 796
Mean relative change	0.3% ± 1.4%
switch-from-txt
chromium
head	67 <- [103 - 111 - 119] -> 144
expected	46 <- [101 - 109 - 117] -> 144
Mean relative change	1.7% ± 1.9%
switch-to-txt
chromium
head	30 <- [50 - 65 - 103] -> 302
expected	37 <- [52 - 75 - 106] -> 136
Mean relative change	-4.9% ± 6.1%
close
chromium
head	251 <- [276 - 288 - 304] -> 354
expected	258 <- [279 - 290 - 302] -> 375
Mean relative change	-0.5% ± 1.9%

Changes are computed with expected as reference.

Expected = 8d42fc0af0e5579aea881680d6e169a24fa56c69 | Actual = 282ec9129964ccefec362c61eaef6340f9274eb2
Go to action log
Changelog covered

❗ Test metadata have changed

--- /dev/fd/63	2023-01-15 03:30:27.193000827 +0000
+++ /dev/fd/62	2023-01-15 03:30:27.193000827 +0000
@@ -1,7 +1,7 @@
 {
   "benchmark": {
     "BENCHMARK_OUTPUTFILE": "lab-benchmark.json",
-    "BENCHMARK_REFERENCE": "head"
+    "BENCHMARK_REFERENCE": "actual"
   },
   "browsers": {
     "chromium": "109.0.5414.46"

[github-actions]

Benchmark report

The execution time (in milliseconds) are grouped by test file, test type and browser.
For each case, the following values are computed: min <- [1st quartile - median - 3rd quartile] -> max.

The mean relative comparison is computed with 95% confidence.

Results table

Test file	longOutput - A single output with 100x100 divs
open
chromium
head	1230 <- [1349 - 1373 - 1446] -> 1752
expected	1252 <- [1336 - 1374 - 1407] -> 1771
Mean relative change	1.1% ± 1.7%
switch-from-copy
chromium
head	753 <- [823 - 879 - 938] -> 1186
expected	768 <- [818 - 866 - 940] -> 1246
Mean relative change	0.4% ± 1.5%
switch-to-copy
chromium
head	864 <- [971 - 1013 - 1048] -> 1252
expected	879 <- [975 - 1010 - 1046] -> 1231
Mean relative change	-0.1% ± 1.0%
switch-from-txt
chromium
head	166 <- [187 - 196 - 207] -> 248
expected	171 <- [188 - 197 - 209] -> 298
Mean relative change	-1.1% ± 1.4%
switch-to-txt
chromium
head	681 <- [742 - 780 - 863] -> 1070
expected	695 <- [742 - 787 - 865] -> 1035
Mean relative change	-0.1% ± 1.6%
search
chromium
head	23472 <- [25974 - 26349 - 27063] -> 29663
expected	23605 <- [26240 - 26504 - 27133] -> 29441
Mean relative change	-0.2% ± 1.5%
close
chromium
head	290 <- [317 - 330 - 345] -> 411
expected	285 <- [314 - 328 - 341] -> 376
Mean relative change	1.2% ± 1.8%

Changes are computed with expected as reference.

Expected = 8d42fc0af0e5579aea881680d6e169a24fa56c69 | Actual = 282ec9129964ccefec362c61eaef6340f9274eb2
Go to action log
Changelog covered

❗ Test metadata have changed

--- /dev/fd/63	2023-01-15 05:39:36.469497147 +0000
+++ /dev/fd/62	2023-01-15 05:39:36.469497147 +0000
@@ -1,7 +1,7 @@
 {
   "benchmark": {
     "BENCHMARK_OUTPUTFILE": "lab-benchmark.json",
-    "BENCHMARK_REFERENCE": "head"
+    "BENCHMARK_REFERENCE": "actual"
   },
   "browsers": {
     "chromium": "109.0.5414.46"

[github-actions]

Benchmark report

The execution time (in milliseconds) are grouped by test file, test type and browser.
For each case, the following values are computed: min <- [1st quartile - median - 3rd quartile] -> max.

The mean relative comparison is computed with 95% confidence.

Results table

Test file	errorOutputs - 10x100 errors output collected
open
chromium
head	461 <- [494 - 506 - 518] -> 651
expected	460 <- [487 - 500 - 515] -> 690
Mean relative change	0.8% ± 1.9%
switch-from-copy
chromium
head	107 <- [148 - 167 - 221] -> 454
expected	123 <- [145 - 160 - 211] -> 473
Mean relative change	6.1% ± 5.3%
switch-to-copy
chromium
head	502 <- [506 - 510 - 513] -> 544
expected	503 <- [506 - 511 - 515] -> 572
Mean relative change	-0.3% ± 0.2%
switch-from-txt
chromium
head	68 <- [99 - 111 - 119] -> 174
expected	51 <- [96 - 105 - 115] -> 199
Mean relative change	2.1% ± 2.5%
switch-to-txt
chromium
head	91 <- [108 - 116 - 125] -> 184
expected	94 <- [111 - 120 - 135] -> 258
Mean relative change	-9.7% ± 2.9%
search
chromium
head	746 <- [796 - 809 - 829] -> 941
expected	670 <- [798 - 813 - 834] -> 900
Mean relative change	-0.1% ± 1.1%
close
chromium
head	151 <- [181 - 194 - 219] -> 310
expected	148 <- [180 - 186 - 207] -> 260
Mean relative change	3.4% ± 4.0%

Changes are computed with expected as reference.

Expected = d4688fe784c2b7d8cb9af8a153af88c91fd8dcc8 | Actual = 6658ffc9d893c993e339932e9e25433f92341673
Go to action log
Changelog covered

❗ Test metadata have changed

--- /dev/fd/63	2023-01-22 02:52:11.425919841 +0000
+++ /dev/fd/62	2023-01-22 02:52:11.425919841 +0000
@@ -1,7 +1,7 @@
 {
   "benchmark": {
     "BENCHMARK_OUTPUTFILE": "lab-benchmark.json",
-    "BENCHMARK_REFERENCE": "head"
+    "BENCHMARK_REFERENCE": "actual"
   },
   "browsers": {
     "chromium": "109.0.5414.46"

[github-actions]

Benchmark report

The execution time (in milliseconds) are grouped by test file, test type and browser.
For each case, the following values are computed: min <- [1st quartile - median - 3rd quartile] -> max.

The mean relative comparison is computed with 95% confidence.

Results table

Test file	large_code_100_notebook
open
chromium
head	406 <- [442 - 461 - 478] -> 571
expected	416 <- [439 - 452 - 465] -> 588
Mean relative change	1.6% ± 1.8%
switch-from-copy
chromium
head	246 <- [298 - 336 - 619] -> 1424
expected	259 <- [298 - 330 - 585] -> 1366
Mean relative change	4.7% ± 9.1%
switch-to-copy
chromium
head	503 <- [508 - 513 - 519] -> 641
expected	503 <- [509 - 512 - 516] -> 591
Mean relative change	0.5% ± 0.4%
switch-from-txt
chromium
head	92 <- [120 - 132 - 151] -> 399
expected	76 <- [117 - 126 - 144] -> 272
Mean relative change	4.2% ± 4.0%
switch-to-txt
chromium
head	134 <- [155 - 168 - 182] -> 268
expected	138 <- [152 - 161 - 177] -> 286
Mean relative change	4.2% ± 2.4%
search
chromium
head	767 <- [891 - 909 - 945] -> 1075
expected	849 <- [885 - 904 - 943] -> 1201
Mean relative change	0.1% ± 1.6%
close
chromium
head	152 <- [173 - 186 - 203] -> 312
expected	145 <- [173 - 188 - 200] -> 343
Mean relative change	1.4% ± 4.6%

Changes are computed with expected as reference.

Expected = d4688fe784c2b7d8cb9af8a153af88c91fd8dcc8 | Actual = 6658ffc9d893c993e339932e9e25433f92341673
Go to action log
Changelog covered

❗ Test metadata have changed

--- /dev/fd/63	2023-01-22 03:00:26.998047711 +0000
+++ /dev/fd/62	2023-01-22 03:00:26.998047711 +0000
@@ -1,7 +1,7 @@
 {
   "benchmark": {
     "BENCHMARK_OUTPUTFILE": "lab-benchmark.json",
-    "BENCHMARK_REFERENCE": "head"
+    "BENCHMARK_REFERENCE": "actual"
   },
   "browsers": {
     "chromium": "109.0.5414.46"

[github-actions]

Benchmark report

The execution time (in milliseconds) are grouped by test file, test type and browser.
For each case, the following values are computed: min <- [1st quartile - median - 3rd quartile] -> max.

The mean relative comparison is computed with 95% confidence.

Results table

Test file	large_md_100_notebook
open
chromium
head	674 <- [751 - 778 - 794] -> 967
expected	652 <- [750 - 768 - 789] -> 892
Mean relative change	0.8% ± 1.4%
switch-from-copy
chromium
head	286 <- [323 - 368 - 533] -> 1006
expected	293 <- [323 - 358 - 531] -> 870
Mean relative change	2.8% ± 5.9%
switch-to-copy
chromium
head	503 <- [510 - 515 - 521] -> 696
expected	503 <- [509 - 514 - 519] -> 565
Mean relative change	0.6% ± 0.4%
switch-from-txt
chromium
head	114 <- [141 - 152 - 248] -> 441
expected	117 <- [140 - 150 - 249] -> 326
Mean relative change	2.1% ± 5.5%
switch-to-txt
chromium
head	216 <- [245 - 267 - 374] -> 567
expected	216 <- [249 - 268 - 350] -> 503
Mean relative change	3.3% ± 3.9%
search
chromium
head	1078 <- [1163 - 1211 - 1260] -> 1448
expected	1103 <- [1192 - 1231 - 1293] -> 1398
Mean relative change	-1.9% ± 1.5%
close
chromium
head	187 <- [215 - 228 - 246] -> 319
expected	184 <- [211 - 223 - 239] -> 292
Mean relative change	3.8% ± 3.1%

Changes are computed with expected as reference.

Expected = d4688fe784c2b7d8cb9af8a153af88c91fd8dcc8 | Actual = 6658ffc9d893c993e339932e9e25433f92341673
Go to action log
Changelog covered

❗ Test metadata have changed

--- /dev/fd/63	2023-01-22 03:05:53.126936353 +0000
+++ /dev/fd/62	2023-01-22 03:05:53.126936353 +0000
@@ -1,7 +1,7 @@
 {
   "benchmark": {
     "BENCHMARK_OUTPUTFILE": "lab-benchmark.json",
-    "BENCHMARK_REFERENCE": "head"
+    "BENCHMARK_REFERENCE": "actual"
   },
   "browsers": {
     "chromium": "109.0.5414.46"

[github-actions]

Benchmark report

The execution time (in milliseconds) are grouped by test file, test type and browser.
For each case, the following values are computed: min <- [1st quartile - median - 3rd quartile] -> max.

The mean relative comparison is computed with 95% confidence.

Results table

Test file	large_metadata_1000x100_notebook
open
chromium
head	4023 <- [4081 - 4105 - 4146] -> 4340
expected	4005 <- [4085 - 4115 - 4160] -> 4273
Mean relative change	0.0% ± 0.4%
switch-from-copy
chromium
head	143 <- [209 - 229 - 362] -> 1133
expected	145 <- [213 - 251 - 361] -> 795
Mean relative change	0.8% ± 8.3%
switch-to-copy
chromium
head	503 <- [505 - 506 - 513] -> 852
expected	503 <- [505 - 506 - 509] -> 784
Mean relative change	0.4% ± 1.7%
switch-from-txt
chromium
head	65 <- [104 - 114 - 123] -> 143
expected	62 <- [105 - 113 - 123] -> 167
Mean relative change	0.3% ± 1.9%
switch-to-txt
chromium
head	32 <- [53 - 78 - 108] -> 150
expected	40 <- [61 - 101 - 112] -> 151
Mean relative change	-12.3% ± 4.7%
close
chromium
head	262 <- [284 - 295 - 313] -> 408
expected	257 <- [289 - 301 - 315] -> 538
Mean relative change	-0.8% ± 2.7%

Changes are computed with expected as reference.

Expected = d4688fe784c2b7d8cb9af8a153af88c91fd8dcc8 | Actual = 6658ffc9d893c993e339932e9e25433f92341673
Go to action log
Changelog covered

❗ Test metadata have changed

--- /dev/fd/63	2023-01-22 03:30:50.065877999 +0000
+++ /dev/fd/62	2023-01-22 03:30:50.065877999 +0000
@@ -1,7 +1,7 @@
 {
   "benchmark": {
     "BENCHMARK_OUTPUTFILE": "lab-benchmark.json",
-    "BENCHMARK_REFERENCE": "head"
+    "BENCHMARK_REFERENCE": "actual"
   },
   "browsers": {
     "chromium": "109.0.5414.46"

[github-actions]

Benchmark report

The execution time (in milliseconds) are grouped by test file, test type and browser.
For each case, the following values are computed: min <- [1st quartile - median - 3rd quartile] -> max.

The mean relative comparison is computed with 95% confidence.

Results table

Test file	longOutput - A single output with 100x100 divs
open
chromium
head	1329 <- [1466 - 1528 - 1599] -> 2035
expected	1309 <- [1442 - 1503 - 1567] -> 1948
Mean relative change	1.4% ± 2.1%
switch-from-copy
chromium
head	811 <- [913 - 968 - 1062] -> 1846
expected	797 <- [901 - 962 - 1050] -> 1379
Mean relative change	1.7% ± 1.9%
switch-to-copy
chromium
head	922 <- [1065 - 1132 - 1199] -> 1421
expected	874 <- [990 - 1081 - 1158] -> 1387
Mean relative change	4.6% ± 1.6%
switch-from-txt
chromium
head	171 <- [198 - 210 - 221] -> 277
expected	165 <- [193 - 204 - 219] -> 345
Mean relative change	1.8% ± 1.6%
switch-to-txt
chromium
head	755 <- [863 - 919 - 1003] -> 1260
expected	738 <- [838 - 893 - 969] -> 1249
Mean relative change	3.3% ± 1.8%
search
chromium
head	21428 <- [24236 - 25432 - 26272] -> 28669
expected	21151 <- [23799 - 24869 - 25942] -> 28858
Mean relative change	1.7% ± 1.8%
close
chromium
head	311 <- [337 - 353 - 370] -> 424
expected	298 <- [327 - 341 - 357] -> 418
Mean relative change	3.1% ± 2.0%

Changes are computed with expected as reference.

Expected = d4688fe784c2b7d8cb9af8a153af88c91fd8dcc8 | Actual = 6658ffc9d893c993e339932e9e25433f92341673
Go to action log
Changelog covered

❗ Test metadata have changed

--- /dev/fd/63	2023-01-22 05:47:30.580082844 +0000
+++ /dev/fd/62	2023-01-22 05:47:30.580082844 +0000
@@ -1,7 +1,7 @@
 {
   "benchmark": {
     "BENCHMARK_OUTPUTFILE": "lab-benchmark.json",
-    "BENCHMARK_REFERENCE": "head"
+    "BENCHMARK_REFERENCE": "actual"
   },
   "browsers": {
     "chromium": "109.0.5414.46"

[github-actions]

Benchmark report

The execution time (in milliseconds) are grouped by test file, test type and browser.
For each case, the following values are computed: min <- [1st quartile - median - 3rd quartile] -> max.

The mean relative comparison is computed with 95% confidence.

Results table

Test file	large_code_100_notebook
open
chromium
head	454 <- [480 - 492 - 504] -> 642
expected	399 <- [454 - 460 - 472] -> 514
Mean relative change	⚠️ 7.4% ± 1.5%
switch-from-copy
chromium
head	311 <- [378 - 403 - 943] -> 1306
expected	206 <- [287 - 321 - 614] -> 1026
Mean relative change	⚠️ 34.4% ± 11.4%
switch-to-copy
chromium
head	503 <- [506 - 511 - 517] -> 553
expected	502 <- [510 - 515 - 521] -> 563
Mean relative change	⚠️ -0.6% ± 0.3%
switch-from-txt
chromium
head	87 <- [127 - 140 - 151] -> 191
expected	76 <- [124 - 134 - 149] -> 271
Mean relative change	⚠️ -2.1% ± 3.2%
switch-to-txt
chromium
head	210 <- [241 - 252 - 263] -> 387
expected	131 <- [151 - 161 - 170] -> 239
Mean relative change	⚠️ 56.8% ± 2.8%
search
chromium
head	974 <- [983 - 987 - 991] -> 1051
expected	975 <- [979 - 983 - 988] -> 1007
Mean relative change	⚠️ 0.6% ± 0.3%
close
chromium
head	181 <- [221 - 245 - 305] -> 434
expected	186 <- [209 - 219 - 237] -> 347
Mean relative change	⚠️ 18.3% ± 6.8%

Changes are computed with expected as reference.

Expected = ac2807fdea9e2d431dce10419c82336cd23ebd17 | Actual = 63f32800e989c0113d1b8caec34d4f530e094e74
Go to action log
Changelog covered

❗ Test metadata have changed

--- /dev/fd/63	2023-02-12 03:02:10.752941328 +0000
+++ /dev/fd/62	2023-02-12 03:02:10.752941328 +0000
@@ -1,7 +1,7 @@
 {
   "benchmark": {
     "BENCHMARK_OUTPUTFILE": "lab-benchmark.json",
-    "BENCHMARK_REFERENCE": "head"
+    "BENCHMARK_REFERENCE": "actual"
   },
   "browsers": {
     "chromium": "110.0.5481.38"

[github-actions]

Benchmark report

The execution time (in milliseconds) are grouped by test file, test type and browser.
For each case, the following values are computed: min <- [1st quartile - median - 3rd quartile] -> max.

The mean relative comparison is computed with 95% confidence.

Results table

Test file	large_md_100_notebook
open
chromium
head	690 <- [766 - 779 - 799] -> 1047
expected	624 <- [675 - 697 - 724] -> 898
Mean relative change	10.8% ± 2.0%
switch-from-copy
chromium
head	282 <- [341 - 361 - 718] -> 1010
expected	206 <- [270 - 294 - 565] -> 874
Mean relative change	20.6% ± 8.4%
switch-to-copy
chromium
head	503 <- [515 - 522 - 532] -> 618
expected	502 <- [512 - 516 - 522] -> 549
Mean relative change	2.1% ± 0.5%
switch-from-txt
chromium
head	103 <- [137 - 151 - 163] -> 228
expected	116 <- [137 - 148 - 263] -> 363
Mean relative change	-19.6% ± 3.6%
switch-to-txt
chromium
head	245 <- [271 - 283 - 295] -> 432
expected	129 <- [155 - 212 - 240] -> 433
Mean relative change	38.5% ± 4.3%
search
chromium
head	1117 <- [1136 - 1144 - 1158] -> 1282
expected	1133 <- [1162 - 1176 - 1245] -> 1380
Mean relative change	-3.8% ± 1.0%
close
chromium
head	205 <- [229 - 241 - 253] -> 358
expected	194 <- [221 - 235 - 249] -> 303
Mean relative change	4.2% ± 2.7%

Changes are computed with expected as reference.

Expected = ac2807fdea9e2d431dce10419c82336cd23ebd17 | Actual = 63f32800e989c0113d1b8caec34d4f530e094e74
Go to action log
Changelog covered

❗ Test metadata have changed

--- /dev/fd/63	2023-02-12 03:07:35.378009258 +0000
+++ /dev/fd/62	2023-02-12 03:07:35.378009258 +0000
@@ -1,7 +1,7 @@
 {
   "benchmark": {
     "BENCHMARK_OUTPUTFILE": "lab-benchmark.json",
-    "BENCHMARK_REFERENCE": "head"
+    "BENCHMARK_REFERENCE": "actual"
   },
   "browsers": {
     "chromium": "110.0.5481.38"

[github-actions]

Benchmark report

The execution time (in milliseconds) are grouped by test file, test type and browser.
For each case, the following values are computed: min <- [1st quartile - median - 3rd quartile] -> max.

The mean relative comparison is computed with 95% confidence.

Results table

Test file	errorOutputs - 10x100 errors output collected
open
chromium
head	386 <- [652 - 684 - 733] -> 996
expected	574 <- [637 - 660 - 700] -> 937
Mean relative change	2.9% ± 3.7%
switch-from-copy
chromium
head	149 <- [203 - 229 - 326] -> 619
expected	138 <- [183 - 203 - 297] -> 531
Mean relative change	9.6% ± 5.6%
switch-to-copy
chromium
head	503 <- [510 - 515 - 523] -> 560
expected	503 <- [509 - 515 - 523] -> 556
Mean relative change	0.1% ± 0.3%
switch-from-txt
chromium
head	104 <- [140 - 153 - 166] -> 225
expected	84 <- [122 - 131 - 142] -> 200
Mean relative change	16.2% ± 2.6%
switch-to-txt
chromium
head	131 <- [153 - 166 - 181] -> 235
expected	96 <- [131 - 141 - 153] -> 216
Mean relative change	16.9% ± 2.5%
search
chromium
head	945 <- [957 - 965 - 975] -> 1053
expected	948 <- [959 - 965 - 974] -> 1024
Mean relative change	0.0% ± 0.5%
close
chromium
head	235 <- [275 - 293 - 328] -> 463
expected	224 <- [262 - 283 - 317] -> 424
Mean relative change	5.1% ± 4.7%

Changes are computed with expected as reference.

Expected = ac2807fdea9e2d431dce10419c82336cd23ebd17 | Actual = 63f32800e989c0113d1b8caec34d4f530e094e74
Go to action log
Changelog covered

❗ Test metadata have changed

--- /dev/fd/63	2023-02-12 03:09:23.543372550 +0000
+++ /dev/fd/62	2023-02-12 03:09:23.543372550 +0000
@@ -1,7 +1,7 @@
 {
   "benchmark": {
     "BENCHMARK_OUTPUTFILE": "lab-benchmark.json",
-    "BENCHMARK_REFERENCE": "head"
+    "BENCHMARK_REFERENCE": "actual"
   },
   "browsers": {
     "chromium": "110.0.5481.38"

[github-actions]

Benchmark report

The execution time (in milliseconds) are grouped by test file, test type and browser.
For each case, the following values are computed: min <- [1st quartile - median - 3rd quartile] -> max.

The mean relative comparison is computed with 95% confidence.

Results table

Test file	large_metadata_1000x100_notebook
open
chromium
head	396 <- [466 - 481 - 498] -> 636
expected	352 <- [430 - 453 - 502] -> 600
Mean relative change	⚠️ 4.2% ± 4.6%
switch-from-copy
chromium
head	202 <- [259 - 321 - 455] -> 752
expected	190 <- [234 - 252 - 464] -> 747
Mean relative change	⚠️ 12.9% ± 11.0%
switch-to-copy
chromium
head	505 <- [511 - 517 - 523] -> 770
expected	503 <- [510 - 519 - 535] -> 749
Mean relative change	⚠️ -1.4% ± 2.5%
switch-from-txt
chromium
head	104 <- [132 - 143 - 158] -> 203
expected	81 <- [106 - 119 - 128] -> 171
Mean relative change	⚠️ 24.0% ± 4.3%
switch-to-txt
chromium
head	101 <- [121 - 129 - 139] -> 175
expected	82 <- [104 - 112 - 122] -> 291
Mean relative change	⚠️ 14.6% ± 4.1%
close
chromium
head	321 <- [348 - 360 - 373] -> 409
expected	310 <- [344 - 364 - 387] -> 865
Mean relative change	⚠️ -6.4% ± 8.6%

Changes are computed with expected as reference.

Expected = ac2807fdea9e2d431dce10419c82336cd23ebd17 | Actual = 63f32800e989c0113d1b8caec34d4f530e094e74
Go to action log
Changelog covered

❗ Test metadata have changed

--- /dev/fd/63	2023-02-12 03:27:16.122769979 +0000
+++ /dev/fd/62	2023-02-12 03:27:16.122769979 +0000
@@ -1,7 +1,7 @@
 {
   "benchmark": {
     "BENCHMARK_OUTPUTFILE": "lab-benchmark.json",
-    "BENCHMARK_REFERENCE": "head"
+    "BENCHMARK_REFERENCE": "actual"
   },
   "browsers": {
     "chromium": "110.0.5481.38"