using pyo3-ffi when dynamically loading libpython

[konstin]

Is it possible to use pyo3-ffi in a binary while dynamically loading libpython? Specifically, I'd like to decide at runtime which libpython to load (with abi3 and libloading). Is it still possible to use pyo3-ffi and its extern blocks, or is there a fundamental limitation why this isn't possible?

[davidhewitt]

I think that the problem with extern blocks is that they'll need to have well-defined symbols at binary load time. I think if you needed to achieve this you'll probably want to have a wrapper binary which just sets the library load path and then forwards to a normally-linked binary?

[konstin]

splitting into a binary and a shared library or another binary would work, but i'd really like to ship a single dependency-free binary (and download the correct libpython at runtime). i'm currently using libloading, but there i have to load each symbol individually and write all the c glue code manually

[davidhewitt]

Is it viable for you to use PyOxidizer and just ship libpython inside the binary too? Otherwise, maybe there's some arcane linker flag which will do what you want, but I'm not aware of it.

[makkus]

Hey @konstin , I'm having the same problem currently, did you figure out a good way to do this in the meantime? One thought I had was for the binary to call itself if no libpython could be found (with LD_LIBRARY_PATH set), after downloading and otherwise setting up libpython. To get around the 'having 2 binaries' problem, but haven't tried that yet.

[luketpeterson]

I'm doing exactly this myself now. @makkus and/or @konstin What did you ultimately end up doing?

Thanks.