Skip to content

Commit 92aff27

Browse files
kernckernc
committed
Add benchmark
1 parent 5f68b26 commit 92aff27

File tree

1 file changed

+263
-0
lines changed

1 file changed

+263
-0
lines changed

‎benchmark.txt‎

Lines changed: 263 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,263 @@
1+
numpy 1.26.4
2+
scipy 1.14.1
3+
scikit-learn 1.5.1
4+
scikit-optimize 0.10.2
5+
hyperopt 0.2.7
6+
7+
Test function Method N Evals Error % Duration
8+
————————————————————————————————————————————————————————————————————————————————
9+
6-hump-camelback/2 SLSQP 20 0 0.00
10+
6-hump-camelback/2 COBYQA 36 0 0.08
11+
6-hump-camelback/2 shgo 36 0 0.00
12+
6-hump-camelback/2 COBYLA 40 0 0.00
13+
6-hump-camelback/2 sambo.minimize(shgo) 50 0 0.01
14+
6-hump-camelback/2 CG† 51 0 0.00
15+
6-hump-camelback/2 TNC† 66 0 0.00
16+
6-hump-camelback/2 Nelder-Mead† 72 0 0.00
17+
6-hump-camelback/2 Powell† 80 0 0.00
18+
6-hump-camelback/2 scikit-optimize 111 0 2.86
19+
6-hump-camelback/2 trust-constr 135 0 0.16
20+
6-hump-camelback/2 sambo.minimize(smbo) 321 0 3.76
21+
6-hump-camelback/2 differential_evolution 348 0 0.03
22+
6-hump-camelback/2 sambo.minimize(sceua) 362 0 0.04
23+
6-hump-camelback/2 hyperopt 776 0 4.56
24+
6-hump-camelback/2 basinhopping 948 0 0.12
25+
6-hump-camelback/2 nevergrad 2000 0 7.50
26+
6-hump-camelback/2 direct† 2007 0 0.02
27+
6-hump-camelback/2 dual_annealing† 4028 0 0.21
28+
bird/2 COBYQA 34 0 0.13
29+
bird/2 SLSQP 35 0 0.01
30+
bird/2 COBYLA 40 0 0.00
31+
bird/2 Powell† 40 0 0.00
32+
bird/2 CG† 53 0 0.01
33+
bird/2 Nelder-Mead† 67 0 0.00
34+
bird/2 TNC† 129 0 0.01
35+
bird/2 sambo.minimize(shgo) 137 0 0.04
36+
bird/2 trust-constr 150 0 0.16
37+
bird/2 scikit-optimize 258 0 47.10
38+
bird/2 sambo.minimize(sceua) 299 0 0.04
39+
bird/2 sambo.minimize(smbo) 316 0 5.49
40+
bird/2 differential_evolution 459 0 0.15
41+
bird/2 hyperopt 1243 0 9.91
42+
bird/2 nevergrad 1805 0 6.30
43+
bird/2 direct† 2013 0 0.03
44+
bird/2 dual_annealing† 4019 0 0.23
45+
bird/2 shgo 31* 48 0.01
46+
bird/2 basinhopping 66* 100 0.01
47+
branin-hoo/2 SLSQP 23 0 0.00
48+
branin-hoo/2 COBYQA 40 0 0.10
49+
branin-hoo/2 COBYLA 46 0 0.00
50+
branin-hoo/2 shgo 55 0 0.01
51+
branin-hoo/2 trust-constr 63 0 0.14
52+
branin-hoo/2 CG† 66 0 0.01
53+
branin-hoo/2 Nelder-Mead† 84 0 0.00
54+
branin-hoo/2 Powell† 95 0 0.00
55+
branin-hoo/2 sambo.minimize(shgo) 103 0 0.02
56+
branin-hoo/2 TNC† 138 0 0.01
57+
branin-hoo/2 scikit-optimize 148 0 12.37
58+
branin-hoo/2 sambo.minimize(smbo) 327 0 5.02
59+
branin-hoo/2 sambo.minimize(sceua) 476 0 0.06
60+
branin-hoo/2 basinhopping 495 0 0.06
61+
branin-hoo/2 differential_evolution 555 0 0.05
62+
branin-hoo/2 hyperopt 1638 0 18.60
63+
branin-hoo/2 nevergrad 1780 0 6.45
64+
branin-hoo/2 direct† 2009 0 0.02
65+
branin-hoo/2 dual_annealing† 4031 0 0.23
66+
eggholder/2 sambo.minimize(sceua) 759 0 0.08
67+
eggholder/2 direct† 2011 0 0.02
68+
eggholder/2 dual_annealing† 4076 0 0.23
69+
eggholder/2 scikit-optimize 293 1 51.73
70+
eggholder/2 differential_evolution 741* 3 0.06
71+
eggholder/2 hyperopt 1286* 3 12.52
72+
eggholder/2 sambo.minimize(shgo) 128* 7 0.02
73+
eggholder/2 nevergrad 2000* 7 6.53
74+
eggholder/2 sambo.minimize(smbo) 311* 9 2.18
75+
eggholder/2 TNC† 117* 12 0.01
76+
eggholder/2 shgo 94* 20 0.01
77+
eggholder/2 Nelder-Mead† 108* 35 0.00
78+
eggholder/2 COBYQA 53* 37 0.13
79+
eggholder/2 COBYLA 129* 37 0.01
80+
eggholder/2 trust-constr 141* 37 0.18
81+
eggholder/2 CG† 57* 38 0.01
82+
eggholder/2 SLSQP 47* 43 0.00
83+
eggholder/2 basinhopping 1269* 44 0.15
84+
eggholder/2 Powell† 135* 48 0.01
85+
gomez-levy/2 COBYQA 39 0 0.16
86+
gomez-levy/2 COBYLA 45 0 0.00
87+
gomez-levy/2 sambo.minimize(shgo) 75 0 0.02
88+
gomez-levy/2 scikit-optimize 115 0 5.50
89+
gomez-levy/2 shgo 298 0 0.03
90+
gomez-levy/2 sambo.minimize(smbo) 313 0 5.10
91+
gomez-levy/2 differential_evolution 423 0 0.10
92+
gomez-levy/2 sambo.minimize(sceua) 550 0 0.08
93+
gomez-levy/2 hyperopt 808 0 5.62
94+
gomez-levy/2 SLSQP 1104 0 0.12
95+
gomez-levy/2 nevergrad 1631 0 6.18
96+
gomez-levy/2 direct† 2015 0 0.02
97+
gomez-levy/2 trust-constr 3231 0 1.90
98+
gomez-levy/2 dual_annealing† 4061 0 0.24
99+
gomez-levy/2 Nelder-Mead† 133 1 0.01
100+
gomez-levy/2 Powell† 78 2 0.00
101+
gomez-levy/2 TNC† 174 2 0.01
102+
gomez-levy/2 basinhopping 802* 3 0.08
103+
gomez-levy/2 CG† 32* 10 0.00
104+
griewank/2 shgo 39 0 0.00
105+
griewank/2 sambo.minimize(shgo) 56 0 0.01
106+
griewank/2 Powell† 118 0 0.01
107+
griewank/2 scikit-optimize 283 0 46.94
108+
griewank/2 sambo.minimize(smbo) 319 0 3.53
109+
griewank/2 direct† 461 0 0.01
110+
griewank/2 sambo.minimize(sceua) 569 0 0.07
111+
griewank/2 hyperopt 759 0 5.14
112+
griewank/2 basinhopping 1065 0 0.15
113+
griewank/2 nevergrad 1321 0 5.20
114+
griewank/2 differential_evolution 1392 0 0.14
115+
griewank/2 dual_annealing† 4109 0 0.34
116+
griewank/2 Nelder-Mead† 102 1 0.01
117+
griewank/2 SLSQP 18* 10 0.00
118+
griewank/2 CG† 24* 10 0.00
119+
griewank/2 COBYQA 33* 10 0.08
120+
griewank/2 trust-constr 33* 10 0.10
121+
griewank/2 COBYLA 35* 10 0.00
122+
griewank/2 TNC† 105* 10 0.01
123+
hartman/6 SLSQP 96 0 0.01
124+
hartman/6 COBYLA 118 0 0.01
125+
hartman/6 trust-constr 147 0 0.15
126+
hartman/6 sambo.minimize(shgo) 154 0 0.03
127+
hartman/6 Powell† 161 0 0.01
128+
hartman/6 shgo 168 0 0.01
129+
hartman/6 CG† 252 0 0.02
130+
hartman/6 Nelder-Mead† 422 0 0.02
131+
hartman/6 TNC† 616 0 0.04
132+
hartman/6 direct† 733 0 0.02
133+
hartman/6 differential_evolution 1787 0 0.16
134+
hartman/6 nevergrad 2000 0 16.97
135+
hartman/6 dual_annealing† 12120 0 0.89
136+
hartman/6 basinhopping 12376 0 1.26
137+
hartman/6 sambo.minimize(sceua) 593 1 0.08
138+
hartman/6 hyperopt 931 1 15.17
139+
hartman/6 COBYQA 222* 4 0.54
140+
hartman/6 sambo.minimize(smbo) 980* 4 27.12
141+
hartman/6 scikit-optimize 135* 61 0.08
142+
rastrigin/2 sambo.minimize(shgo) 21 0 0.01
143+
rastrigin/2 shgo 26 0 0.01
144+
rastrigin/2 SLSQP 42 0 0.01
145+
rastrigin/2 direct† 313 0 0.01
146+
rastrigin/2 sambo.minimize(smbo) 316 0 65.69
147+
rastrigin/2 sambo.minimize(sceua) 491 0 0.33
148+
rastrigin/2 basinhopping 828 0 0.12
149+
rastrigin/2 differential_evolution 1972 0 0.41
150+
rastrigin/2 nevergrad 2000 0 6.99
151+
rastrigin/2 dual_annealing† 4088 0 0.28
152+
rastrigin/2 COBYQA 37 2 0.20
153+
rastrigin/2 COBYLA 40 2 0.00
154+
rastrigin/2 hyperopt 500 2 2.72
155+
rastrigin/2 scikit-optimize 222* 3 255.27
156+
rastrigin/2 trust-constr 1161* 5 0.67
157+
rastrigin/2 CG† 3* 100 0.00
158+
rastrigin/2 TNC† 3* 100 0.00
159+
rastrigin/2 Nelder-Mead† 47* 100 0.00
160+
rastrigin/2 Powell† 51* 100 0.00
161+
rosenbrock/10 direct† 413 0 0.01
162+
rosenbrock/10 SLSQP 637 0 0.07
163+
rosenbrock/10 sambo.minimize(shgo) 664 0 2.26
164+
rosenbrock/10 shgo 708 0 1.83
165+
rosenbrock/10 COBYQA 914 0 5.13
166+
rosenbrock/10 COBYLA 1000 0 0.07
167+
rosenbrock/10 TNC† 1100 0 0.07
168+
rosenbrock/10 sambo.minimize(sceua) 1382 0 0.26
169+
rosenbrock/10 trust-constr 1485 0 0.50
170+
rosenbrock/10 Nelder-Mead† 2000 0 0.14
171+
rosenbrock/10 nevergrad 2000 0 11.04
172+
rosenbrock/10 Powell† 2758 0 0.17
173+
rosenbrock/10 CG† 4272 0 0.28
174+
rosenbrock/10 basinhopping 20901 0 1.41
175+
rosenbrock/10 dual_annealing† 24489 0 1.60
176+
rosenbrock/10 differential_evolution 150652 0 19.98
177+
rosenbrock/10 sambo.minimize(smbo) 1551 2 46.63
178+
rosenbrock/10 hyperopt 500* 3 10.43
179+
rosenbrock/10 scikit-optimize 209* 9 0.99
180+
rosenbrock/2 sambo.minimize(shgo) 45 0 0.02
181+
rosenbrock/2 COBYQA 100 0 0.54
182+
rosenbrock/2 shgo 176 0 0.03
183+
rosenbrock/2 scikit-optimize 191 0 29.15
184+
rosenbrock/2 Powell† 224 0 0.02
185+
rosenbrock/2 Nelder-Mead† 282 0 0.03
186+
rosenbrock/2 sambo.minimize(smbo) 328 0 9.49
187+
rosenbrock/2 sambo.minimize(sceua) 386 0 0.07
188+
rosenbrock/2 hyperopt 500 0 2.92
189+
rosenbrock/2 COBYLA 1000 0 0.07
190+
rosenbrock/2 SLSQP 1124 0 0.14
191+
rosenbrock/2 nevergrad 1698 0 6.96
192+
rosenbrock/2 direct† 2011 0 0.04
193+
rosenbrock/2 trust-constr 2988 0 2.10
194+
rosenbrock/2 differential_evolution 3504 0 2.55
195+
rosenbrock/2 dual_annealing† 4283 0 0.28
196+
rosenbrock/2 TNC† 93 1 0.01
197+
rosenbrock/2 basinhopping 534 1 0.06
198+
rosenbrock/2 CG† 29 2 0.01
199+
schwefel/2 sambo.minimize(shgo) 84 0 0.02
200+
schwefel/2 scikit-optimize 279 0 43.98
201+
schwefel/2 sambo.minimize(sceua) 603 0 0.07
202+
schwefel/2 direct† 665 0 0.01
203+
schwefel/2 hyperopt 1819 0 21.23
204+
schwefel/2 dual_annealing† 4046 0 0.27
205+
schwefel/2 differential_evolution 4719 0 0.43
206+
schwefel/2 sambo.minimize(smbo) 311 1 2.34
207+
schwefel/2 nevergrad 2000* 7 7.30
208+
schwefel/2 shgo 34* 21 0.00
209+
schwefel/2 Powell† 54* 25 0.00
210+
schwefel/2 SLSQP 24* 34 0.00
211+
schwefel/2 trust-constr 24* 34 0.09
212+
schwefel/2 COBYLA 44* 34 0.00
213+
schwefel/2 COBYQA 44* 34 0.11
214+
schwefel/2 CG† 69* 34 0.01
215+
schwefel/2 Nelder-Mead† 82* 34 0.00
216+
schwefel/2 TNC† 153* 34 0.01
217+
schwefel/2 basinhopping 768* 50 0.10
218+
simionescu/2 sambo.minimize(shgo) 29 0 0.01
219+
simionescu/2 COBYQA 52* 10 0.21
220+
simionescu/2 sambo.minimize(sceua) 107* 10 0.02
221+
simionescu/2 Nelder-Mead† 218* 10 0.01
222+
simionescu/2 sambo.minimize(smbo) 318* 10 7.70
223+
simionescu/2 differential_evolution 981* 10 0.45
224+
simionescu/2 direct† 2013* 10 0.02
225+
simionescu/2 dual_annealing† 4163* 10 0.26
226+
simionescu/2 hyperopt 500* 11 2.61
227+
simionescu/2 trust-constr 3063* 11 2.10
228+
simionescu/2 scikit-optimize 101* 14 0.82
229+
simionescu/2 Powell† 91* 20 0.00
230+
simionescu/2 TNC† 96* 39 0.01
231+
simionescu/2 CG† 65* 41 0.01
232+
simionescu/2 nevergrad 1512* 42 6.35
233+
simionescu/2 SLSQP 21* 43 0.01
234+
simionescu/2 shgo 1249* 43 0.12
235+
simionescu/2 basinhopping 547* 55 0.06
236+
simionescu/2 COBYLA 47* 100 0.00
237+
238+
239+
Method Correct N Evals Error % Duration
240+
————————————————————————————————————————————————————————————
241+
sambo.minimize(shgo) 92% 129 1 0.04
242+
sambo.minimize(sceua) 92% 548 1 0.24
243+
direct 92% 1389 1 0.03
244+
dual_annealing 92% 6459 1 0.84
245+
differential_evolution 83% 13961 1 2.34
246+
sambo.minimize(smbo) 75% 476 2 44.68
247+
hyperopt 75% 938 2 18.26
248+
nevergrad 75% 1812 5 10.69
249+
scikit-optimize 67% 195 7 51.27
250+
COBYLA 67% 215 15 0.06
251+
shgo 67% 243 11 0.11
252+
SLSQP 67% 266 11 0.12
253+
Nelder-Mead 67% 301 15 0.03
254+
Powell 67% 324 16 0.02
255+
COBYQA 58% 134 8 0.54
256+
TNC 58% 232 16 0.04
257+
trust-constr 58% 1052 8 2.08
258+
basinhopping 58% 3383 21 1.15
259+
CG 50% 414 20 0.02
260+
261+
262+
* Did not finish / unexpected result.
263+
† Non-constrained method.

0 commit comments

Comments
 (0)