Unable to build pandas from source on Windows

[clin1234]

pip install pandas
Collecting pandas
  Using cached pandas-2.2.2.tar.gz (4.4 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [32 lines of output]
      + meson setup C:\Users\ךינשגכהד\AppData\Local\Temp\pip-install-12u0d8nj\pandas_dbfe103136b3466e8b5b75d423059088 C:\Users\ךינשגכהד\AppData\Local\Temp\pip-install-12u0d8nj\pandas_dbfe103136b3466e8b5b75d423059088\.mesonpy-ta2thafr\build -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --vsenv --native-file=C:\Users\ךינשגכהד\AppData\Local\Temp\pip-install-12u0d8nj\pandas_dbfe103136b3466e8b5b75d423059088\.mesonpy-ta2thafr\build\meson-python-native-file.ini
      The Meson build system
      Version: 1.2.1
      Source dir: C:\Users\ךינשגכהד\AppData\Local\Temp\pip-install-12u0d8nj\pandas_dbfe103136b3466e8b5b75d423059088
      Build dir: C:\Users\ךינשגכהד\AppData\Local\Temp\pip-install-12u0d8nj\pandas_dbfe103136b3466e8b5b75d423059088\.mesonpy-ta2thafr\build
      Build type: native build
      Project name: pandas
      Project version: 2.2.2
      Activating VS 17.11.0 Preview 5.0
      C compiler for the host machine: sccache cl (msvc 19.41.34117 "Microsoft (R) C/C++ Optimizing Compiler Version 19.41.34117 for x64")
      C linker for the host machine: link link 14.41.34117.0
      C++ compiler for the host machine: sccache cl (msvc 19.41.34117 "Microsoft (R) C/C++ Optimizing Compiler Version 19.41.34117 for x64")
      C++ linker for the host machine: link link 14.41.34117.0
      Cython compiler for the host machine: cython (cython 3.0.5)
      Host machine cpu family: x86_64
      Host machine cpu: x86_64
      Program python found: YES (C:\Users\ךינשגכהד\scoop\apps\python-alpha\current\python.exe)

      ..\..\pandas\meson.build:1:15: ERROR: Command `C:\Users\ךינשגכהד\scoop\apps\python-alpha\current\python.exe -c "
      import os
      import numpy as np
      try:
          # Check if include directory is inside the pandas dir
          # e.g. a venv created inside the pandas dir
          # If so, convert it to a relative path
          incdir = os.path.relpath(np.get_include())
      except Exception:
          incdir = np.get_include()
      print(incdir)
           "` failed with status 3221225477.

      A full log can be found at C:\Users\ךינשגכהד\AppData\Local\Temp\pip-install-12u0d8nj\pandas_dbfe103136b3466e8b5b75d423059088\.mesonpy-ta2thafr\build\meson-logs\meson-log.txt
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

python -c "import numpy;print(numpy.version.version)"
2.1.0.dev0+git20240728.fdcbd0e

[rhshadrach]

Can you try running the program that failed and share the output / stacktrace.

      import os
      import numpy as np
      try:
          # Check if include directory is inside the pandas dir
          # e.g. a venv created inside the pandas dir
          # If so, convert it to a relative path
          incdir = os.path.relpath(np.get_include())
      except Exception:
          incdir = np.get_include()
      print(incdir)

[clin1234]

~/src/OpenBBTerminal $ python << EOF
> import os
> import numpy as np
> try:
>   incdir = os.path.relpath(np.get_include())
> except Exception:
>   incdir = np.get_include()
> print(incdir)
> EOF
..\..\scoop\apps\python-alpha\current\Lib\site-packages\numpy\_core\include
~/src/OpenBBTerminal $ echo $?
0

[WillAyd]

Which shell are you using? This is a complete guess but I wonder if the non-ascii characters in your username are the culprit.

Our build system is essentially providing the path where numpy is located back to your shell for use in a command - maybe the path that gets resolved isn't usable as a literal string in the shell?

[clin1234]

Tried both cmd.exe and bash

[clin1234]

Return value of 3221225477 might be related to 0xC000005 (Access violation)

[rgommers]

If you do pip install pandas -Cbuild-dir=build, then you'll be able to obtain the more detailed logging in build\meson-logs\meson-log.txt which may give more insight.

[clin1234]

Build log:
meson-log.txt

[rgommers]

So that says:

--- stderr ---
<frozen importlib._bootstrap>:488: Warning: Numpy built with MINGW-W64 on Windows 64 bits is experimental, and only available for 
testing. You are advised not to use it for production. 

CRASHES ARE TO BE EXPECTED - PLEASE REPORT THEM TO NUMPY DEVELOPERS



..\..\..\..\..\ךינשגכהד\AppData\Local\Temp\pip-install-rd40svcp\pandas_7614d43e4c2e4321ad6e33440bb60e0d\pandas\meson.build:1:15: ERROR: Command `C:\Users\ךינשגכהד\scoop\apps\python-alpha\current\python.exe -c "
import os
import numpy as np
try:
    # Check if include directory is inside the pandas dir
    # e.g. a venv created inside the pandas dir
    # If so, convert it to a relative path
    incdir = os.path.relpath(np.get_include())
except Exception:
    incdir = np.get_include()
print(incdir)
     "` failed with status 3221225477.

I'd recommend not using MinGW, but then again it's probably not the issue.

It's some issue related to your local environment (drives, access permissions, virus scanner, or some such thing). Seems fairly common (reports like https://learn.microsoft.com/en-us/answers/questions/1036492/dll-crash-on-exit-with-error-3221225477), but it's not really possible to say much more based on just an error code.

You're also running python-alpha from a fairly obscure install method (https://scoop.sh/#/apps?q=python-alpha, currently at Python 3.13.b4), with a numpy that was built with a non-recommended compiler. That's too many non-standard things together - I think you have a choice of debugging that yourself locally, or using something that is better tested.

[clin1234]

Is there a way for pdb to interface with a meson.build file?

[dnicolodi]

pdb would not tell much about the issue: the issue is not caused by anything happening at the Python level. It is most likely caused by some ABI mismatch or other much lower level issue. You can run the Python code snippet run by Meson independently of Meson. It should fail in exactly the same way. If you want to debug the issue, you would need to run Python in a binary level debugger and see what exactly fails.

[clin1234]

Update: issue went away by rebuilding numpy and pandas using the MSVC compiler and lld-link. Btw, is there documentation recommending only using compilers targeting the MS abi due to numpy's very experimental minge64 support?