Skip to content

PyInit for cython extension not located when compiled with pyodide, but is in pyodide auditwheel output #1153

@AndrewAnnex

Description

@AndrewAnnex

I am transitioning my library to use scikit-build-core to enable me to provide Pyodide wheels for my python project that provides both a ctypes based interface to a pure c library and a cython extension that links to the same shared library (for important reasons).

My project is structured like so:

src/
      spiceypy/ 
             __init__.py (imports functions from spiceypy.py and imports cyice to be a sub-namespace spiceypy.cyice)
             spiceypy.py (contains all the ctypes wrapper functions)
             utils/
                       __init__.py 
                       libcspice.wasm (shared library I wrap with ctypes and also link to the cython extension
              cyice/
                       __init__.py 
                       cyice.so (the pyx and pxd files are also in here as is the generated cython c code)
     

I want to keep this layout to minify any api breaking changes, and so far as I can tell should work.

Because of the complicated layout I built this prototype python package to verify I could get things working without too many extra tests or complications, and it does work: https://github.com/AndrewAnnex/spiceypy-cyice-scikit-build-core-test

However in the main project my tests fail to import this cython extension, claiming that the PyInit_cyice function is not exported

2025-09-29T22:28:13.1982434Z __________________ ERROR collecting benchmarks/test_cyice.py ___________________
2025-09-29T22:28:13.1988102Z ImportError while importing test module '/tmp/cibw-run-9djpqn83/cp312-pyodide_wasm32/venv-test/lib/python3.12/site-packages/spiceypy/benchmarks/test_cyice.py'.
2025-09-29T22:28:13.1989985Z Hint: make sure your test modules/packages have valid Python names.
2025-09-29T22:28:13.1991029Z Traceback:
2025-09-29T22:28:13.1991636Z /lib/python312.zip/importlib/__init__.py:90: in import_module
2025-09-29T22:28:13.1992589Z     return _bootstrap._gcd_import(name[level:], package, level)
2025-09-29T22:28:13.1993216Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-09-29T22:28:13.1993897Z ../venv-test/lib/python3.12/site-packages/spiceypy/__init__.py:28: in <module>
2025-09-29T22:28:13.1994457Z     from .cyice import cyice
2025-09-29T22:28:13.1994887Z ../venv-test/lib/python3.12/site-packages/spiceypy/cyice/__init__.py:29: in <module>
2025-09-29T22:28:13.1995337Z     from .cyice import *
2025-09-29T22:28:13.1995737Z E   ImportError: dynamic module does not define module export function 

However, running pyodide auditwheel shows it as expected:

spiceypy/cyice/cyice.so:
      FUNC	__wasm_call_ctors
      FUNC	__wasm_apply_data_relocs
      FUNC	PyInit_cyice
    GLOBAL	__pyx_module_is_main_spiceypy__cyice__cyice

I can also directly load the libcspice.wasm with ctypes within pyodide (via jupyterlite) and see it works as expected.

So there seems to something subtly wrong with the cython extension compilation I'm not spotting. Or maybe the relative import is wrong in some way thats causing the export to not work as expected? Directly loading the same cyice.so with ctypes shows that the library seems to replicate the error also, but in my cmake I am as far as I can tell compiling a shared library and I am even now explicitly exporting the pyinit function (https://github.com/AndrewAnnex/SpiceyPy/blob/f295c02d3ca8d6e812f20a694668af7352668848/CMakeLists.txt#L136-L141), but I didn't have to make any hacks like I did in the test repo to have this work as expected.

I also get a weird error from pyodide in the tests claiming "Error: Didn't expect to load any more file_packager files!" which isn't clear, but I get the same error when attempting to install the wheel inside jupyterlite with piplite, but if I run it a 2nd time it goes away, which isn't confidence building.. But I am not sure if that's actually an issue caused by the compilation or something else entirely.

Link to branch: https://github.com/AndrewAnnex/SpiceyPy/tree/add_pyodide_build
Link to PR: AndrewAnnex/SpiceyPy#516
Link to my CMakeLists.txt: https://github.com/AndrewAnnex/SpiceyPy/blob/add_pyodide_build/CMakeLists.txt
Link to my pyproject.toml: https://github.com/AndrewAnnex/SpiceyPy/blob/add_pyodide_build/pyproject.toml
Build log from github actions:
build_log_github.txt

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions