BUG: Segmentation Fault when importing Pandas in python 3.10.14

[bhawmik]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

sudiptabhawmik@Sudiptas-iPro gippss % python3
Python 3.10.14 (main, May  6 2024, 14:47:20) [Clang 14.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas
zsh: segmentation fault  python3

Issue Description

In MacOS (10.15.7) in a conda created virtual env with Python 3.10.14 installed,
when I open a python3 shell and try importing pandas, I get this segmentation fault.
When I try to import Pandas in Python 3.10.14 in MacOS zsh, python3 crashes with segmentation fault.

sudiptabhawmik@Sudiptas-iPro gippss % python3
Python 3.10.14 (main, May 6 2024, 14:47:20) [Clang 14.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.

import pandas
zsh: segmentation fault python3

I tried Pandas from version 2.2.2 down to 1.5.n
I tried changing zsh to bash and csh, didn't work. I tried multiple versions of Pandas from 2.2.2 to 1.5.n but failed.
Works fine with Python 3.7.n and Pandas pandas==1.1.5

Expected Behavior

Expected behavior should be no segmentation fault in MacOS when Pandas. is imported.

Installed Versions

MacOS (10.15.7) Python 3.10.14 Pandas 2.0.3

[lithomas1]

Hi, thanks for the report.

Can you try attaching the output from lldb so we can better see what's going on?

e.g. run

lldb -- python -c "import pandas as pd"

and then enter "r" into lldb to execute that when you enter the lldb session.

After that, enter "bt" to get the backtrace, and copy that here.

(If you don't have lldb, you might need to install the Xcode command line tools).

[bhawmik]

I could resolve the problem by installing both python and pandas using conda (conda install). Earlier I installed python with conda but used pip for pandas installation. It seems these two installations are not compatible.

[lithomas1]

Thanks for the update!

[sric0880]

after run lldb -- python -c "import pandas as pd" I find it's the libarrow throws EXC_BAD_ACCESS error.
everything goes ok after I uninstall pyarrow!

$ lldb -- python -c "import pandas as pd"
(lldb) target create "python"
Current executable set to 'python' (x86_64).
(lldb) settings set -- target.run-args  "-c" "import pandas as pd"
(lldb) r
Process 72983 launched: '/Users/qiong/opt/miniconda3/bin/python' (x86_64)
Process 72983 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x20a9b54)
    frame #0: 0x00007fff72f4b5ad libsystem_pthread.dylib`pthread_mutex_lock
libsystem_pthread.dylib`pthread_mutex_lock:
->  0x7fff72f4b5ad <+0>:  cmpq   $0x4d55545a, (%rdi)       ; imm = 0x4D55545A 
    0x7fff72f4b5b4 <+7>:  jne    0x7fff72f4b625            ; <+120>
    0x7fff72f4b5b6 <+9>:  movl   $0x1c0, %eax              ; imm = 0x1C0 
    0x7fff72f4b5bb <+14>: andl   0xc(%rdi), %eax
Target 0: (python) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x20a9b54)
  * frame #0: 0x00007fff72f4b5ad libsystem_pthread.dylib`pthread_mutex_lock
    frame #1: 0x00007fff7004a539 libc++.1.dylib`std::__1::mutex::lock() + 9
    frame #2: 0x00007fff7004b245 libc++.1.dylib`std::__1::__shared_mutex_base::lock() + 31
    frame #3: 0x0000000117742212 libarrow.1700.dylib`arrow::fs::FileSystemFactoryRegistry::RegisterFactory(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, arrow::fs::FileSystemFactory, std::__1::function<void ()>, bool) + 82
    frame #4: 0x000000011774272d libarrow.1700.dylib`arrow::fs::FileSystemRegistrar::FileSystemRegistrar(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, arrow::fs::FileSystemFactory, std::__1::function<void ()>) + 285
    frame #5: 0x00000001177897ca libarrow.1700.dylib`arrow::Result<std::__1::shared_ptr<arrow::fs::HadoopFileSystem> >::Result(arrow::Status const&) + 426
    frame #6: 0x00000001003db353 dyld`ImageLoaderMachO::doModInitFunctions(ImageLoader::LinkContext const&) + 535
    frame #7: 0x00000001003db75e dyld`ImageLoaderMachO::doInitialization(ImageLoader::LinkContext const&) + 40

[hualei2016]

after run lldb -- python -c "import pandas as pd" I find it's the libarrow throws EXC_BAD_ACCESS error. everything goes ok after I uninstall pyarrow!

$ lldb -- python -c "import pandas as pd"
(lldb) target create "python"
Current executable set to 'python' (x86_64).
(lldb) settings set -- target.run-args  "-c" "import pandas as pd"
(lldb) r
Process 72983 launched: '/Users/qiong/opt/miniconda3/bin/python' (x86_64)
Process 72983 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x20a9b54)
    frame #0: 0x00007fff72f4b5ad libsystem_pthread.dylib`pthread_mutex_lock
libsystem_pthread.dylib`pthread_mutex_lock:
->  0x7fff72f4b5ad <+0>:  cmpq   $0x4d55545a, (%rdi)       ; imm = 0x4D55545A 
    0x7fff72f4b5b4 <+7>:  jne    0x7fff72f4b625            ; <+120>
    0x7fff72f4b5b6 <+9>:  movl   $0x1c0, %eax              ; imm = 0x1C0 
    0x7fff72f4b5bb <+14>: andl   0xc(%rdi), %eax
Target 0: (python) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x20a9b54)
  * frame #0: 0x00007fff72f4b5ad libsystem_pthread.dylib`pthread_mutex_lock
    frame #1: 0x00007fff7004a539 libc++.1.dylib`std::__1::mutex::lock() + 9
    frame #2: 0x00007fff7004b245 libc++.1.dylib`std::__1::__shared_mutex_base::lock() + 31
    frame #3: 0x0000000117742212 libarrow.1700.dylib`arrow::fs::FileSystemFactoryRegistry::RegisterFactory(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, arrow::fs::FileSystemFactory, std::__1::function<void ()>, bool) + 82
    frame #4: 0x000000011774272d libarrow.1700.dylib`arrow::fs::FileSystemRegistrar::FileSystemRegistrar(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, arrow::fs::FileSystemFactory, std::__1::function<void ()>) + 285
    frame #5: 0x00000001177897ca libarrow.1700.dylib`arrow::Result<std::__1::shared_ptr<arrow::fs::HadoopFileSystem> >::Result(arrow::Status const&) + 426
    frame #6: 0x00000001003db353 dyld`ImageLoaderMachO::doModInitFunctions(ImageLoader::LinkContext const&) + 535
    frame #7: 0x00000001003db75e dyld`ImageLoaderMachO::doInitialization(ImageLoader::LinkContext const&) + 40

This for the info. It helped a lot. I have been blocked by this issue for hours ...

[saraalrumih]

I was getting the same error and uninstalling the pyarrow package was not helpful for my case.
I solved it by downgrading pyarrow to 15.0.0 version!
Kudos to toymodel333 (https://discuss.streamlit.io/t/streamlit-multipage-app-crashes-on-import-pandas-or-import-plotly-express-statement/70594/16)

I then got a "HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out." error and managed to solve it by using admin rights (sudo pip install pyarrow==15.0.0)