Parallel execution of benchmarks #80
Description
Is your feature request related to a problem? Please describe.
Evaluating algorithmic ideas is slow due to linear execution of embarassingly parallel benchmarks
Describe the solution you'd like
Use MultiProcessing to launch all benchmark settings in parallel
Describe alternatives you've considered
One can run multiple times benchmark.py with different setitngs -- but that is also impractical since benchmark does not take keyword arguments to setup benchmarks -- it is in python -- so manually we have to keep track of what is run and what is not
Use case
Policy experiments -- the time can reduce from 2hr -> 5 minutes
Implementation suggestions
Multiprocessing
Would you be willing to contribute this feature?
Yes, i can work on it