Commit aa3b0e1
Process TFM_MIG entries in parallel; add --profile flag (#338)
This PR adds a `--profile` flag to run_benchmarks.py that produces a
cProfile file `xl2times.prof` that can be visualized with e.g.
[snakeviz](https://jiffyclub.github.io/snakeviz/). It also parallelizes
the querying and update creation of TFM_MIG rows in order to speed up
the tool on TIMES-GEO.
Before (on `208aa4e`):
```
Ran TIMES-GEO in 838.50s. 96.1% (823412 correct, 14985 additional).
```
After (on `616ae9a`):
```
Ran TIMES-GEO in 228.88s. 96.1% (823412 correct, 14985 additional).
```
A 3.7x speedup on my 2.3 GHz 8-Core Intel Core i9 laptop!
(I created a new test file to test different approaches to speeding up
the querying. It's not really testing anything at the moment, but I've
kept it in because it'll be useful if we need to experiment with other
optimizations for the querying and speed this up even further.)
---------
Co-authored-by: Olexandr Balyk <ob@facilitate.energy>1 parent 84d0e22 commit aa3b0e1
File tree
7 files changed
+516
-56
lines changed- .github/workflows
- tests
- utils
- xl2times
7 files changed
+516
-56
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
24 | | - | |
| 24 | + | |
25 | 25 | | |
26 | 26 | | |
27 | 27 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| 14 | + | |
| 15 | + | |
14 | 16 | | |
15 | 17 | | |
16 | 18 | | |
| |||
62 | 64 | | |
63 | 65 | | |
64 | 66 | | |
| 67 | + | |
| 68 | + | |
65 | 69 | | |
66 | 70 | | |
67 | 71 | | |
| |||
0 commit comments