Commit 21a5092
committed
[SYSTEMDS-3849] Perftest: Fix MVSM performance regression (spark ctx)
This patch fixes a perftest performance regression of
runMSVM_10k_1k_dense_k5 which ran in 36s instead of few seconds in
earlier releases. The reason was unnecessary spark context creation
during parfor optimization. We now handle theses cluster info requests
more carefully, which now avoids this unnecessary spark context creation
and reduced the total runtime back to 5.9s.1 parent 4d6c6b4 commit 21a5092
File tree
2 files changed
+8
-3
lines changed- src/main/java/org/apache/sysds
- runtime/controlprogram/context
- utils/stats
2 files changed
+8
-3
lines changedLines changed: 4 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
340 | 340 | | |
341 | 341 | | |
342 | 342 | | |
| 343 | + | |
343 | 344 | | |
344 | | - | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
345 | 348 | | |
346 | 349 | | |
347 | 350 | | |
| |||
Lines changed: 4 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
183 | 183 | | |
184 | 184 | | |
185 | 185 | | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
186 | 189 | | |
187 | 190 | | |
188 | 191 | | |
189 | | - | |
| 192 | + | |
190 | 193 | | |
191 | 194 | | |
192 | 195 | | |
| |||
198 | 201 | | |
199 | 202 | | |
200 | 203 | | |
201 | | - | |
202 | 204 | | |
203 | 205 | | |
204 | 206 | | |
| |||
0 commit comments