`mypy` nightly crashes with `ModuleNotFoundError: No module named 'native_internal'`

[cmp0xff]

Crash Report

mypy nightly crashed upon starting, see this GitHub CI pipeline for an example. It is also reproducible on my local machine.

Traceback

Traceback (most recent call last):
  File "/home/runner/.cache/pypoetry/virtualenvs/pandas-stubs-Si3tVypI-py3.11/bin/mypy", line 3, in <module>
    from mypy.__main__ import console_entry
  File "/home/runner/.cache/pypoetry/virtualenvs/pandas-stubs-Si3tVypI-py3.11/lib/python3.11/site-packages/mypy/__main__.py", line 9, in <module>
    from mypy.main import main, process_options
  File "/home/runner/.cache/pypoetry/virtualenvs/pandas-stubs-Si3tVypI-py3.11/lib/python3.11/site-packages/mypy/main.py", line 16, in <module>
    from mypy import build, defaults, state, util
  File "/home/runner/.cache/pypoetry/virtualenvs/pandas-stubs-Si3tVypI-py3.11/lib/python3.11/site-packages/mypy/build.py", line 41, in <module>
    import mypy.semanal_main
  File "/home/runner/.cache/pypoetry/virtualenvs/pandas-stubs-Si3tVypI-py3.11/lib/python3.11/site-packages/mypy/semanal_main.py", line 37, in <module>
    from mypy.checker import FineGrainedDeferredNode
  File "/home/runner/.cache/pypoetry/virtualenvs/pandas-stubs-Si3tVypI-py3.11/lib/python3.11/site-packages/mypy/checker.py", line 23, in <module>
    import mypy.checkexpr
  File "/home/runner/.cache/pypoetry/virtualenvs/pandas-stubs-Si3tVypI-py3.11/lib/python3.11/site-packages/mypy/checkexpr.py", line 16, in <module>
    from mypy import applytype, erasetype, join, message_registry, nodes, operators, types
  File "/home/runner/.cache/pypoetry/virtualenvs/pandas-stubs-Si3tVypI-py3.11/lib/python3.11/site-packages/mypy/applytype.py", line 6, in <module>
    import mypy.subtypes
  File "/home/runner/.cache/pypoetry/virtualenvs/pandas-stubs-Si3tVypI-py3.11/lib/python3.11/site-packages/mypy/subtypes.py", line 9, in <module>
    import mypy.constraints
  File "/home/runner/.cache/pypoetry/virtualenvs/pandas-stubs-Si3tVypI-py3.11/lib/python3.11/site-packages/mypy/constraints.py", line 10, in <module>
    import mypy.typeops
  File "/home/runner/.cache/pypoetry/virtualenvs/pandas-stubs-Si3tVypI-py3.11/lib/python3.11/site-packages/mypy/typeops.py", line 14, in <module>
    from mypy.checker_state import checker_state
  File "/home/runner/.cache/pypoetry/virtualenvs/pandas-stubs-Si3tVypI-py3.11/lib/python3.11/site-packages/mypy/checker_state.py", line 7, in <module>
    from mypy.checker_shared import TypeCheckerSharedApi
  File "/home/runner/.cache/pypoetry/virtualenvs/pandas-stubs-Si3tVypI-py3.11/lib/python3.11/site-packages/mypy/checker_shared.py", line 13, in <module>
    from mypy.errors import ErrorWatcher
  File "/home/runner/.cache/pypoetry/virtualenvs/pandas-stubs-Si3tVypI-py3.11/lib/python3.11/site-packages/mypy/errors.py", line 15, in <module>
    from mypy.nodes import Context
  File "/home/runner/.cache/pypoetry/virtualenvs/pandas-stubs-Si3tVypI-py3.11/lib/python3.11/site-packages/mypy/nodes.py", line 17, in <module>
    from mypy.cache import (
  File "/home/runner/.cache/pypoetry/virtualenvs/pandas-stubs-Si3tVypI-py3.11/lib/python3.11/site-packages/mypy/cache.py", line 7, in <module>
    from native_internal import (
ModuleNotFoundError: No module named 'native_internal'

To Reproduce

poetry run python -m pip install --upgrade --find-links https://github.com/mypyc/mypy_mypyc-wheels/releases/ mypy
poetry run mypy .

Your Environment

• Mypy version used: 1.19.0+dev
• Mypy command-line flags:
• Mypy configuration options from mypy.ini (and other config files):
• Python version used:
• Operating system and version:

[A5rocks]

Maybe the wheels being built in https://github.com/mypyc/mypy_mypyc-wheels are incorrect...?

What OS/Python version are you running? Does MYPYC_OPT_LEVEL=0 MYPY_USE_MYPYC=1 pip install . in a git clone of mypy build a mypy that fails in the same way? (it shouldn't as that is tested in CI, but just checking)

[cmp0xff]

Hi @A5rocks , thank you for triaging the PR.

On my local machine, I am on Debian 12, and when I just clone mypy, I cannot reproduce the error. I did not use MYPYC_OPT_LEVEL=0 MYPY_USE_MYPYC=1. If that's necessary, could you give me more hints?

[sterliakov]

Yes, looks like smth is wrong with the nightly wheels: I just downloaded a recent whl and it lacks the necessary .so.

cd "$(mktemp -d)"
pip download 'mypy==1.18.2' --no-deps
mv mypy-*.whl old.whl
pip download 'mypy' --find-links 'https://github.com/mypyc/mypy_mypyc-wheels/releases/' --no-deps
mv mypy-*.whl new.whl

echo 'Stable:'
unzip -l old.whl | grep native_internal

echo 'Nightly:'
unzip -l new.whl | grep native_internal

Gives me

Collecting mypy==1.18.2
  Using cached mypy-1.18.2-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (2.2 kB)
Using cached mypy-1.18.2-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (13.3 MB)
Saved ./mypy-1.18.2-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Successfully downloaded mypy
renamed 'mypy-1.18.2-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl' -> 'old.whl'
Looking in links: https://github.com/mypyc/mypy_mypyc-wheels/releases/
Collecting mypy
  Using cached https://github.com/mypyc/mypy_mypyc-wheels/releases/download/v1.19.0%2Bdev.de2f375eb0da1c446e2aab1474c28b8d7707a848/mypy-1.19.0%2Bdev-py3-none-any.whl (2.4 MB)
Saved ./mypy-1.19.0+dev-py3-none-any.whl
Successfully downloaded mypy
renamed 'mypy-1.19.0+dev-py3-none-any.whl' -> 'new.whl'
Stable:
   135464  2025-09-18 17:57   native_internal.cpython-312-x86_64-linux-gnu.so
      611  2025-09-18 17:57   mypy/typeshed/stubs/mypy-native/native_internal.pyi
    19938  2025-09-18 17:57   mypyc/lib-rt/native_internal.c
     2587  2025-09-18 17:57   mypyc/lib-rt/native_internal.h
Nightly:
      611  2025-10-04 02:16   mypy/typeshed/stubs/librt/native_internal.pyi
    19938  2025-10-04 02:16   mypyc/lib-rt/native_internal.c
     2587  2025-10-04 02:16   mypyc/lib-rt/native_internal.h

Note that native_internal.cpython-312-x86_64-linux-gnu.so is what can actually be imported at runtime.

[sterliakov]

Furthermore, that's not specific to some platform or python version, none of the wheels in the latest nightly build have this shared lib.

gh -R mypyc/mypy_mypyc-wheels release download -p "*" -D v1.19.0+dev.fe313349ee9895d49e7b63ec096e3c7f76d96f2e
for f in ./*.whl; do unzip -l "$f" | grep 'native_internal.*.so'; done

cc @ilevkivskyi - is this supposed to fail now?

UPD: that shouldn't be related at all since librt is published and installed, sorry. There's something wrong with poetry installs specifically and only after certain sequence steps, I'll try to take a look tonight. Installing the same wheels with uv or plain pip works fine.

Poetry seems to use some weird global cache and package storage system (and that's why I stopped using it myself), this must be relevant. Creating a new poetry project (poetry init and hit Enter until it stops asking something) followed by poetry add https://github.com/mypyc/mypy_mypyc-wheels/releases/download/v1.19.0%2Bdev.de2f375eb0da1c446e2aab1474c28b8d7707a848/mypy-1.19.0%2Bdev-py3-none-any.whl produces a working mypy installation, poetry run mypy -c 'import math' passes.

[sterliakov]

Okay, I do not think there's any issue on mypy end, I probably know what exactly is wrong here, and I do not have any feasible workaround to offer.

This works:

cd "$(mktemp -d)"
python -m venv .venv
. .venv/bin/activate
python -m pip install mypy -U -f https://github.com/mypyc/mypy_mypyc-wheels/releases
python -m mypy -c ''

This fails:

cd "$(mktemp -d)"
python -m venv .venv
. .venv/bin/activate
python -m pip install 'mypy==1.18.2'
python -m mypy -c ''
# OK
python -m pip install mypy -U -f https://github.com/mypyc/mypy_mypyc-wheels/releases
python -m mypy -c ''
# CRASH

The problem is that mypy==1.18.2 ships its own native_internal shared library, but newer versions (nightly) do not and use one provided by librt instead. So when new mypy is installed on top of the existing installation, native_internal*.so is removed (as it's now gone from mypy), but librt is installed before doing that (because it's a direct dependency added in the new version), so pip ends up removing the native_internal*.so provided by librt and is obviously unaware of that fact.

Again, I don't have any trivial workaround to offer: apparently installing mypy==1.19.0 into an environment containing mypy==1.18.2 needs an extra step. Either drop and recreate the venv (probably difficult with the mess Poetry makes of its filesystem), or install librt explicitly again after installing mypy, or reinstall mypy once again on top of that:

python -m pip install mypy -U -f https://github.com/mypyc/mypy_mypyc-wheels/releases
python -m pip install librt --force-reinstall
# or
# python -m pip install mypy -U -f https://github.com/mypyc/mypy_mypyc-wheels/releases --force-reinstall

Either of these options will avoid the crash.

[sterliakov]

BTW I think we should report this to pip because removing files that do not belong to the package being uninstalled is a questionable outcome of installing some package.

[sterliakov]

Oh, no, we shouldn't - that's very similar to pypa/pip#13591 and pypa/pip#12211. We hardly ship any namespace packages, but a standalone top-level file should not be too different.

[sterliakov]

I left a comment with a repro on the linked pip issue. Please do not close this ticket yet - we'll get a lot of angry reports about this when we release 1.19.0...

[A5rocks]

Can we ship native_internal*.so in a different place for librt and lookup both? (Just brainstorming on how to avoid breakage)

[sterliakov]

Hm, we could ship its copy (native_internal_mypy.so) in librt, and mypy will import native_internal_mypy in next 2-3 versions. Then we'll be able to deprecate the alternative name and move back to native_internal, documenting that upgrading through multiple minor versions may cause such issues. It's much better than pip install -U silently failing when upgrading from the latest release, thanks!

[A5rocks]

That + try/except the ImportError to say "librt is corrupted, please reinstall it" would make sense to me.

[ilevkivskyi]

FWIW I would simply rename the module, native_internal is kind of bad name anyway (as it follows the old idea of using native for librt). @JukkaL any opinions on the new name? FWIW I would call it librt_internal.