✨ linalg: improved basic solve* annotations

[jorenham]

This improves the following "basic" scipy.linalg functions:

• solve
• solve_triangular
• solve_banded
• solveh_banded
• solve_toeplitz
• solve_circulant

A total of 240 type-tests have been added for these functions.

[jorenham]

This somehow causes the type-tests for scipy.stats._entropy to fail, but only on Pyright. These tests, and their respective stub definitions, are completely independent. The only conclusion I can draw is that this "spooky action at a distance" is caused by a bug in Pyright.

As can be seen in the CI output, this affects both pyright (the lint job) and basedpyright (the typetest jobs) in the exact same way.

For reference, here's the full pyright output:

pyright 1.1.403, node v20.19.1, pyright-action 2.3.2
Working directory: /home/runner/work/scipy-stubs/scipy-stubs
Running: /home/runner/actions-runner/cached/externals/node20/bin/node /opt/hostedtoolcache/pyright/1.1.403/x64/package/index.js --outputjson
/home/runner/work/scipy-stubs/scipy-stubs/tests/stats/test__entropy.pyi:180:13 - error: "assert_type" mismatch: expected "floating[_16Bit]" but received "floating[_16Bit] | ndarray[tuple[Any, ...], dtype[floating[_16Bit]]]" (reportAssertTypeFailure)
Error: "assert_type" mismatch: expected "floating[_16Bit]" but received "floating[_16Bit] | ndarray[tuple[Any, ...], dtype[floating[_16Bit]]]" (reportAssertTypeFailure)
/home/runner/work/scipy-stubs/scipy-stubs/tests/stats/test__entropy.pyi:181:13 - error: "assert_type" mismatch: expected "ndarray[tuple[int], dtype[floating[_16Bit]]]" but received "floating[_16Bit] | ndarray[tuple[Any, ...], dtype[floating[_16Bit]]]" (reportAssertTypeFailure)
Error: "assert_type" mismatch: expected "ndarray[tuple[int], dtype[floating[_16Bit]]]" but received "floating[_16Bit] | ndarray[tuple[Any, ...], dtype[floating[_16Bit]]]" (reportAssertTypeFailure)
/home/runner/work/scipy-stubs/scipy-stubs/tests/stats/test__entropy.pyi:187:13 - error: "assert_type" mismatch: expected "floating[_32Bit]" but received "floating[_32Bit] | ndarray[tuple[Any, ...], dtype[floating[_32Bit]]]" (reportAssertTypeFailure)
Error: "assert_type" mismatch: expected "floating[_32Bit]" but received "floating[_32Bit] | ndarray[tuple[Any, ...], dtype[floating[_32Bit]]]" (reportAssertTypeFailure)
/home/runner/work/scipy-stubs/scipy-stubs/tests/stats/test__entropy.pyi:188:13 - error: "assert_type" mismatch: expected "ndarray[tuple[int], dtype[floating[_32Bit]]]" but received "floating[_32Bit] | ndarray[tuple[Any, ...], dtype[floating[_32Bit]]]" (reportAssertTypeFailure)
Error: "assert_type" mismatch: expected "ndarray[tuple[int], dtype[floating[_32Bit]]]" but received "floating[_32Bit] | ndarray[tuple[Any, ...], dtype[floating[_32Bit]]]" (reportAssertTypeFailure)
/home/runner/work/scipy-stubs/scipy-stubs/tests/stats/test__entropy.pyi:194:13 - error: "assert_type" mismatch: expected "float64" but received "float64 | ndarray[tuple[Any, ...], dtype[float64]]" (reportAssertTypeFailure)
Error: "assert_type" mismatch: expected "float64" but received "float64 | ndarray[tuple[Any, ...], dtype[float64]]" (reportAssertTypeFailure)
/home/runner/work/scipy-stubs/scipy-stubs/tests/stats/test__entropy.pyi:195:13 - error: "assert_type" mismatch: expected "ndarray[tuple[int], dtype[float64]]" but received "float64 | ndarray[tuple[Any, ...], dtype[float64]]" (reportAssertTypeFailure)
Error: "assert_type" mismatch: expected "ndarray[tuple[int], dtype[float64]]" but received "float64 | ndarray[tuple[Any, ...], dtype[float64]]" (reportAssertTypeFailure)
/home/runner/work/scipy-stubs/scipy-stubs/tests/stats/test__entropy.pyi:201:13 - error: "assert_type" mismatch: expected "floating[_64Bit | _96Bit | _128Bit]" but received "floating[_64Bit | _96Bit | _128Bit] | ndarray[tuple[Any, ...], dtype[floating[_64Bit | _96Bit | _128Bit]]]" (reportAssertTypeFailure)
Error: "assert_type" mismatch: expected "floating[_64Bit | _96Bit | _128Bit]" but received "floating[_64Bit | _96Bit | _128Bit] | ndarray[tuple[Any, ...], dtype[floating[_64Bit | _96Bit | _128Bit]]]" (reportAssertTypeFailure)
/home/runner/work/scipy-stubs/scipy-stubs/tests/stats/test__entropy.pyi:202:13 - error: "assert_type" mismatch: expected "ndarray[tuple[int], dtype[floating[_64Bit | _96Bit | _128Bit]]]" but received "floating[_64Bit | _96Bit | _128Bit] | ndarray[tuple[Any, ...], dtype[floating[_64Bit | _96Bit | _128Bit]]]" (reportAssertTypeFailure)
Error: "assert_type" mismatch: expected "ndarray[tuple[int], dtype[floating[_64Bit | _96Bit | _128Bit]]]" but received "floating[_64Bit | _96Bit | _128Bit] | ndarray[tuple[Any, ...], dtype[floating[_64Bit | _96Bit | _128Bit]]]" (reportAssertTypeFailure)
/home/runner/work/scipy-stubs/scipy-stubs/tests/stats/test__entropy.pyi:208:13 - error: "assert_type" mismatch: expected "complexfloating[_32Bit, _32Bit]" but received "complexfloating[_32Bit, _32Bit] | ndarray[tuple[Any, ...], dtype[complexfloating[_32Bit, _32Bit]]]" (reportAssertTypeFailure)
Error: "assert_type" mismatch: expected "complexfloating[_32Bit, _32Bit]" but received "complexfloating[_32Bit, _32Bit] | ndarray[tuple[Any, ...], dtype[complexfloating[_32Bit, _32Bit]]]" (reportAssertTypeFailure)
/home/runner/work/scipy-stubs/scipy-stubs/tests/stats/test__entropy.pyi:209:13 - error: "assert_type" mismatch: expected "ndarray[tuple[int], dtype[complexfloating[_32Bit, _32Bit]]]" but received "complexfloating[_32Bit, _32Bit] | ndarray[tuple[Any, ...], dtype[complexfloating[_32Bit, _32Bit]]]" (reportAssertTypeFailure)
Error: "assert_type" mismatch: expected "ndarray[tuple[int], dtype[complexfloating[_32Bit, _32Bit]]]" but received "complexfloating[_32Bit, _32Bit] | ndarray[tuple[Any, ...], dtype[complexfloating[_32Bit, _32Bit]]]" (reportAssertTypeFailure)
10 errors, 0 warnings, 0 informations

Note that pyright tests/linalg/test__sketches.pyi reports 0 errors. So these errors also don't appear in VSCode (with either the pylance or basedpyright plugin).

---

update

Replacing the contents of tests/linalg/test_basic.pyi with the following will also trigger this bug:

import numpy as np
from scipy.linalg import solve

i8_2d: np.ndarray[tuple[int, int], np.dtype[np.int8]]

solve(i8_2d, i8_2d)

---

update 2

Running pyright --stats --verbose . shows the following per-file analysis times:

2949ms: file:///home/joren/Workspace/scipy-stubs/tests/linalg/test__basic.pyi
1520ms: file:///home/joren/Workspace/scipy-stubs/scipy-stubs/linalg/_basic.pyi
671ms: file:///home/joren/Workspace/scipy-stubs/tests/stats/test__entropy.pyi
356ms: file:///home/joren/Workspace/scipy-stubs/scipy-stubs/sparse/_construct.pyi
340ms: file:///home/joren/Workspace/scipy-stubs/scipy-stubs/special/_ufuncs.pyi
333ms: file:///home/joren/Workspace/scipy-stubs/scipy-stubs/stats/_distribution_infrastructure.pyi
302ms: file:///home/joren/Workspace/scipy-stubs/tests/sparse/test_construct.pyi
281ms: file:///home/joren/Workspace/scipy-stubs/scipy-stubs/sparse/_base.pyi
280ms: file:///home/joren/Workspace/scipy-stubs/scipy-stubs/stats/_entropy.pyi
243ms: file:///home/joren/Workspace/scipy-stubs/tests/special/test_logsumexp.pyi
210ms: file:///home/joren/Workspace/scipy-stubs/scipy-stubs/special/_logsumexp.pyi
207ms: file:///home/joren/Workspace/scipy-stubs/tests/sparse/test_csr.pyi
188ms: file:///home/joren/Workspace/scipy-stubs/scipy-stubs/sparse/_dok.pyi
188ms: file:///home/joren/Workspace/scipy-stubs/tests/linalg/test__decomp_svd.pyi
159ms: file:///home/joren/Workspace/scipy-stubs/scipy-stubs/stats/__init__.pyi

So the changes in this PR are problematic for pyright's runtime. My guess is that this is what's triggering this bug, especially considering that the false-positives are reported in test__entropy.pyi, which is the third slowest on the list.

But to be honest, I have no idea how timing could trigger this bug when we're not using --threads 🤷🏻. Perhaps there's a confounder, i.e. something that causes both the slow analysis and the bug?

---

update 3

After several hours of looking for a workaround, I almost decided to give up, but then ran pyright --threads=2 as hail Mary. This option is known to be unstable, and in the past it was causing false posives in scipy-stubs to be reported. This is a known issue, so I expected it would report at least the same false-positives as it did before. But as it turns out; timing-based pyright bugs cancel out, because with --threads=2 no errors are reported:

$ uv run pyright --threads=2
0 errors, 0 warnings, 0 informations
$ basedpyright --threads=2
0 errors, 0 warnings, 0 notes

This suggests that it less likely that there's a confounder, i.e. that the slow analysis is what's triggering the bug.

But there's one thing I know for sure: The more I look into this, the less I understand about it 😅

[jorenham]

I'll try to get pyright's analysis time down first, as it's bad for DX. Perhaps by doing that, the bug will "solve" itself. But if not, then there's still the --threads=2 workaround (assuming that it'll still work at that point).

[jorenham]

I don't how why exactly, but but removing the overloads for the float32 and complex64 (which aren't used very often in practice), and by simplifying some of the type aliases, I manages to bring down Pyright's analysis time of linalg/_basic.pyi. This somehow also made the false positives disappear. I still don't know whether the bug is triggered by the long analysis of _basic.pyi, or whether there's an unknown confounder that causes both.