RF: Rearranging matmul order and using hermitian flag in ICC_rep_anova

[kristoferm94]

Summary

See #3450

List of changes proposed in this PR (pull-request)

Calculation of predicted_Y refactored such that:

• numpy.dot is replaced with the @ operator
• The pseudo-inverse of X.T@X is now calculated with the hermitian flag set to True (pinv uses a more efficient algorithm when the matrix input is assumed to be hermitian, which will always be the case with X.T@X in this function).
• The matrix multiplication order is changed in a way that we don't end up multiplying as many fat matrices as before.

[effigies]

Nice! I ran some quick tests just to see what this looks like. The reordering seems to dominate, but both changes seem well worth making.

[effigies]

For a simulated Y of shape 50 x 10, I get:

In []: %timeit dot(dot(dot(X, pinv(dot(X.T, X))), X.T), Y.flatten("F"))
1.42 ms ± 128 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

Convert to matmul:

In []: %timeit (X @ pinv(X.T @ X) @ X.T) @ Y.flatten("F")
1.4 ms ± 188 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

No change. Reorder multiplications:

In []: %timeit X @ (pinv(X.T @ X) @ (X.T @ Y.flatten("F")))
838 µs ± 43.3 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

41% speedup. Use hermetian optimization:

In []: %timeit X @ (pinv(X.T @ X, hermitian=True) @ (X.T @ Y.flatten("F")))
785 µs ± 17.8 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

6% relative speedup for total of 45%.

[effigies]

I think you are going to want to remove the dot import from the top of the file, since it's no longer used. Otherwise looks good to me.

[codecov]

Codecov Report

Merging #3453 (9cb58c3) into master (8d065ca) will not change coverage.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master    #3453   +/-   ##
=======================================
  Coverage   65.24%   65.24%           
=======================================
  Files         309      309           
  Lines       40833    40833           
  Branches     5376     5376           
=======================================
  Hits        26641    26641           
  Misses      13118    13118           
  Partials     1074     1074           

Flag	Coverage Δ
unittests	65.02% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
nipype/algorithms/icc.py	57.53% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8d065ca...9cb58c3. Read the comment docs.

[kristoferm94]

I removed the numpy.dot import

[effigies]

Thanks!