failed to import pytorch with EXC_BAD_INSTRUCTION (code=1, subcode=0x4a03000) error

[sunya1989]

operating system: macos 13.0.1 (with m1 chip)
python version:3.9.15
pyo3 version: 0.18.2

I encountered this error when i run following code:

PyModule::import(py, "torch").unwrap();

I also failed on importing numpy, pandas but i can import some other packages like sys, argparse sucessfully.

[davidhewitt]

Can you give us any more information about how you installed torch and numpy, made even a sample project with a script that contains all the install steps?

[sunya1989]

Can you give us any more information about how you installed torch and numpy, made even a sample project with a script that contains all the install steps?

I use conda to install pytorch (conda version 22.11.1)

conda create -n test_pyo3 python=3.9.15 && conda activate test_pyo3 && conda install pytorch torchvision torchaudio -c pytorch

Then I wrote following rust codes:

#[test]
fn test_pyo3(){
unsafe {
let pyhome = WideCString::from_str("/Users/sunyahu/opt/anaconda3/envs/test_pyo3")
.unwrap()
.into_raw() as *mut i32;
ffi::Py_SetPythonHome(pyhome);
}
let gil = Python::acquire_gil();
let py = gil.python();
PyModule::import(py, "torch").unwrap();
}

The error comes out at last line

[davidhewitt]

Can you try compiling with these rustflags:

RUSTFLAGS="-C link-arg=-Wl,-rpath,/Users/sunyahu/opt/anaconda3/envs/test_pyo3/lib"

(From a quick test on my laptop, I need to set the rpath or the wrong libpython is loaded)

[sunya1989]

The funny thing I find out is that this error only appears when i debug:

sunyahu@MacBook-Pro test_pyo3 % cargo run
   Compiling pyo3-build-config v0.18.2
   Compiling pyo3-ffi v0.18.2
   Compiling pyo3 v0.18.2
   Compiling test_pyo3 v0.1.0 (/Users/sunyahu/test/test_pyo3)
warning: use of deprecated associated function `pyo3::Python::<'py>::acquire_gil`: prefer Python::with_gil
  --> src/main.rs:10:23
   |
10 |     let gil = Python::acquire_gil();
   |                       ^^^^^^^^^^^
   |
   = note: `#[warn(deprecated)]` on by default

warning: `test_pyo3` (bin "test_pyo3") generated 1 warning
    Finished dev [unoptimized + debuginfo] target(s) in 3.25s
     Running `target/debug/test_pyo3`
success!

(i add println!("success!") at last line)

but when i run lldb:

rust-lldb /Users/sunyahu/test/test_pyo3/target/debug/test_pyo3

(lldb) r

the error appears

Process 4376 launched: '/Users/sunyahu/test/test_pyo3/target/debug/test_pyo3' (arm64)
Process 4376 stopped
* thread #1, name = 'main', queue = 'com.apple.main-thread', stop reason = EXC_BAD_INSTRUCTION (code=1, subcode=0x4a03000)
    frame #0: 0x0000000120b4a108 libcrypto.3.dylib`_armv8_sve_probe
libcrypto.3.dylib`:
->  0x120b4a108 <+0>: eor    z0.d, z0.d, z0.d
    0x120b4a10c <+4>: ret    

libcrypto.3.dylib`:
    0x120b4a110 <+0>: xar    z0.d, z0.d, z0.d, #0x20
    0x120b4a114 <+4>: ret    
Target 0: (test_pyo3) stopped.

[davidhewitt]

Sounds like an environment issue. Can you figure out where this libcrypto is coming from, is it outside the Conda environment?

[sunya1989]

i have it under /Users/sunyahu/opt/anaconda3/envs/test_pyo3/ directory

find /Users/sunyahu/opt/anaconda3/envs/test_pyo3/ -name libcrypto.3.dylib
/Users/sunyahu/opt/anaconda3/envs/test_pyo3//lib/libcrypto.3.dylib

[davidhewitt]

Right, but is that the one loaded when the debugger blows up?

[sunya1989]

It works after i move libcrypto.3.dylib and libcrypto.dylib out of /Users/sunyahu/opt/anaconda3/envs/test_pyo3/ directory

But I don't konw why and It is not a good solution, (it may reference other libcrypto files in other diretory after i remove this one and it works....)

[sunya1989]

Removing libcrypto.3.dylib and libcrypto.dylib will also cause following problem when install ray:

pip install ray

then we got error messages:

WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Requirement already satisfied: ray in ./python3.9/site-packages (2.0.1)
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/click/

[sunya1989]

it seems like an openssl issue:

https://stackoverflow.com/questions/74059978/why-is-lldb-generating-exc-bad-instruction-with-user-compiled-library-on-macos