Integrating scipy-stubs into SciPy

[RoyiAvital]

Is there a reason scipy-stubs is not integrated into SciPy?

[jorenham]

Thanks for asking; I should probably put this in the FAQ 🤔.

Is there a reason scipy-stubs is not integrated into SciPy?

There are a bunch of reasons, actually. These are the most important ones:

Separation of concerns

If scipy-stubs would be part of SciPy, then that would require every contributor to also be familiar with the stubs. Not everyone would like that, which could hurt the general willingness to contribute to SciPy. It might sound arrogant, but writing stubs is very difficult and requires a lot of expertise to do right (example). I don't think it's a good idea expect every SciPy dev to become a typing expert all of sudden. And the one to review all typing changes would realistically be me, which will probably be inefficient in terms of time. So as much as I would like to see more contributors work on typing, I don't think that this is something we should force upon them.

Opt-in

A large portion of SciPy users simply don't care about typing. Without scipy-stubs installed, type-checkers won't type-check scipy-related code. If you're doing things that are technically type-unsafe, but that work fine in practice, then installing scipy-stubs will cause a bunch of squiggly lines to appear. If scipy-stubs would be part of SciPy, then users wouldn't have a choice.

Independent releases

The versioning scheme uses 4 version numbers, of which the first three match the SciPy semver version, and the last one is specific to scipy-stubs releases, which tend to occur more frequently than those of SciPy.

CI decoupling

The SciPy CI pipeline is very elaborate, and takes a long time to run (>30 minutes). The scipy-stubs CI is very lightweight, and runs in under 4 minutes. I'm sure you can imagine the impact of this on the development workflow.