Conversation
Codecov Report
@@ Coverage Diff @@
## trunk #69 +/- ##
==========================================
- Coverage 75.31% 73.74% -1.58%
==========================================
Files 9 8 -1
Lines 1203 1131 -72
==========================================
- Hits 906 834 -72
Misses 297 297
Flags with carried forward coverage won't be shown. Click here to find out more. Continue to review full report at Codecov.
|
|
FWIW, there is a parallelization implementation here #65, albeit the heavy-lifting is shifted to the C++ file instead of the pyx one. |
|
Thanks for mentioning that. I still prefer to split up the work into big chunks instead of applying OpenMP to the inner loop, since this minimizes overhead and avoids any contention or lack of reentrancy in the gdist algorithm itself. In any case, I ran out of memory and could not compute the whole thing it one go, so yet another reason for the functions in the PR (that allow computing block of rows at a time). |
This enable parallelization of the gdist calculation across CPU cores (6-core CPU drops expected runtime from 8 hours to 1h30). It also enables a progress bar and keyboard interrupt support.