BENCH: Benchmarks with meson optimization levels

[Smit-create]

Hi @rgommers,
I have written a CI script for benchmarking and uploading the results to GitHub Actions with meson optimization levels(#36). See: https://github.com/Smit-create/scipy/blob/for_run/.github/workflows/linux.yml

I am uploading results here too: https://drive.google.com/drive/folders/1x9sBoOGLKMf6VWV0jNj0_p6Jgm35HDAM?usp=sharing

I can see many failing/timeout tests in them: https://drive.google.com/file/d/1mpqNEdzy9eml8FFVNMf3v0r_EMv_NMJ-/view?usp=sharing

Should I open an issue in scipy about these tests?

[rgommers]

Yes that would be useful. If you can add the way to recreate the failure without running the whole test suite (e.g. python runtests.py --no-build --bench BenchThatsFailing), that would be useful.

[rgommers]

Saving some runbench.py current results here. The benchmark timings need an update, because those are now the asv runtime, rather than the sum over the actual benchmark results. But the build time and installed size results should be reliable.

(1) Ran it with python runbench.py linalg while building on 10 cores (-j 10) and got:

Optimization Level      Build Time            Benchmarks Time            Size 
0               100.49949049949646              308.7487030029297               245M
1               104.61960482597351              295.6740379333496               316M
2               118.19833755493164              295.3264682292938               330M
3               123.51068782806396              295.2923581600189               336M
g               103.1662015914917               294.93110060691833              317M
s               104.83046507835388              295.1928594112396               281M

(2) Ran all but the optimize benchmarks (because those take forever) while building with -j 1:

$ git st
On branch meson
Your branch is up to date with 'origin/meson'.

Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        deleted:    benchmarks/benchmarks/optimize.py
        deleted:    benchmarks/benchmarks/optimize_lap.py
        deleted:    benchmarks/benchmarks/optimize_linprog.py
        deleted:    benchmarks/benchmarks/optimize_qap.py
        deleted:    benchmarks/benchmarks/optimize_zeros.py

$ python runbench.py
Optimization Level      Build Time            Benchmarks Time            Size 
0               647.6344661712646               5090.681086778641               245M
1               809.6512916088104               4388.309802532196               316M
2               940.309764623642                4323.488268852234               330M
3               985.0790905952454               4275.624203443527               336M
g               759.3073546886444               4382.525142908096               317M
s               792.8293855190277               4426.311350107193               281M