Rust -> Python -> Rust

[Kinrany]

This seems like a reasonably common case: a Rust app includes scripts written in Python that in turn rely on modules provided by the app and also written in Rust.

I would expect this to be mentioned in Calling Python from Rust.

This turns out to be a rather hard topic to search for because search engines see queries as a salad of "Python, Rust, module", and basically every PyO3 page fits.

There is PyModule::from_code that creates and loads a module, but there doesn't seem to be a way to load a module created with PyModule::new. I'm confused whether the pymodule macro is at all needed for modules that are not being compiled as a separate dynamically linked library.

Help?

[Kinrany]

One thing that works is calling PyModule::from_code with an empty string in place of source code and then adding items to that module with PyModule::add and similar methods.

So unless I'm confused about this, PyModule::from_code seems to bundle creating a module and running Python code in that module's context. Not sure if I'm missing a more orthogonal API here.

Edit: nope, doesn't work.
Edit 2: it does.

[Kinrany]

Thanks!

What's the difference in scope between PyO3 and PyOxidizer's pyembed? I had the impression that PyO3 covers Rust-Python interop pretty comprehensively.

[davidhewitt]

PyO3 wraps the Python C/API to provide low-level FFI bindings as well as a high-level API to produce native Python modules. pyembed is built upon PyO3 to provide an abstraction specifically to control a Python interpreter embedded in a Rust binary (often with one or more additional "built-in" modules defined using PyO3).

[Kinrany]

Would append_to_inittab work for an arbitrary depth of rust->python->rust-python->rust calls? In the general case a Rust module will have to import Python modules, and that wouldn't work without initializing the interpreter first, correct?

[davidhewitt]

append_to_inittab should be run before you start the Python interpreter; it just creates a list of modules to initialize and doesn't actually run any import code until the Python interpreter is started, so I believe that any combination you want should work ok.

[Kinrany]

Oh, that would be useful indeed. My code currently starts with an import-sorted list of modules declared with from_code, doing basically the same thing but with lots of boilerplate. Thanks!

[alexpyattaev]

I have a related issue but I'd like to use pyclass defined in "inner" crate to be accessible to rust code from outer crate. Linker appears to dislike this setup... It appears the only way to access rust pyclasses from rust "outer" crate is to link everything in one go, and thus piling everything into one crate (which is not best for compile times but gets the job done).