Skip to content

SIDE_MODULE_CXXFLAGS are empty (issues building with C++ files with exceptions handling) #251

@Andrej730

Description

@Andrej730

Not sure if this issue belong to this repo or https://github.com/pyodide/pyodide, but shouldn't be a big of deal.

Just came across the issue compiling pyodide Python extension and meeting error Error: Dynamic linking error: cannot resolve symbol invoke_iii when trying to import it in wasm-python.

Investigating in depth, I've found that one of the libraries I was using was referring to this invoke_iii symbol.
Narrowing it down, I've found that the issue was that this library had some exception handling in its code and its .o file was compiled with -fexceptions, but without -fwasm-exceptions - I don't really know how it works on low level, but then emscripten ends up using invoke_iii symbols when handling exceptions, which pyodide doesn't use.

The reason why there's no -fwasm-exceptions - it was .cpp file and therefore it was using CMAKE_CXX_FLAGS, which is initialized based on SIDE_MODULE_CXXFLAGS:

set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} $ENV{SIDE_MODULE_CXXFLAGS}")

Which is initialized from Makefile.envs's CXXFLAGS_BASE which is empty.

https://github.com/pyodide/pyodide/blob/df94f6edcf3937d1d692c6e740e1be27c850af38/Makefile.envs#L153-L154

-fwasm-exceptions only provided for CFLAGS_BASE -> SIDE_MODULE_CFLAGS -> CMAKE_C_FLAGS

https://github.com/pyodide/pyodide/blob/df94f6edcf3937d1d692c6e740e1be27c850af38/Makefile.envs#L134-L140

Should CXXFLAGS_BASE reuse flags from CFLAGS_BASE to avoid issues when project is using c++ files with exceptions handling?


Note for others meeting similar issues:

A workarounds:

  1. Initialize SIDE_MODULE_CXXFLAGS from cflags globally before running pyodide build-recipes-no-deps or other pyodide command you're using.
export SIDE_MODULE_CXXFLAGS=`pyodide config get cflags`
  1. Or inside your meta.yaml recipe or wherever before your script runs emcmake cmake, initalize CXXFLAGS from CFLAGS env variable:
export SIDE_MODULE_CXXFLAGS=$SIDE_MODULE_CFLAGS

How to locate the exact failing library:
Symbol doesn't have to be invoke_iii exactly, it can be invoke_ii, invoke_vi, depending on the kind of exception.
If you want to find where this symbol is coming from, emnm won't help you with this - invoke_iii is not present in any of it's options.
What you can do, is to do grep -a invoke_iii library.so, which will give you output like below and maybe there will be clues (for me opencascade suggested that symbol is coming to opencascade library, so I was able investigate it this library in particular and find the exact file that introduces it).

invoke_iii ��env�_ZN15OSD_EnvironmentD2Ev ��env�pthread_detach ��envpthread_join ��envpthread_self ��env/_ZN11opencascade6handleI16Standard_FailureED2Ev ��env�__wasm_setjmp_test ��env�setTempRet0  �env

If someone wants to reproduce this qucikly:

// hello.cpp
#include <Python.h>

static PyObject* add(PyObject* self, PyObject* args) {
    int a, b;
    if (!PyArg_ParseTuple(args, "ii", &a, &b)) {
        return NULL;
    }
    try {
        return PyLong_FromLong(a + b);
    } catch (...) {
        return NULL;
    }
}

static PyMethodDef methods[] = {
    {"add", add, METH_VARARGS, "Add two numbers"},
    {NULL, NULL, 0, NULL}
};

static struct PyModuleDef module = {
    PyModuleDef_HEAD_INIT,
    "hello",
    NULL,
    -1,
    methods
};

PyMODINIT_FUNC PyInit_hello(void) {
    return PyModule_Create(&module);
}
# CMakeLists.txt
cmake_minimum_required(VERSION 3.15)
project(hello)

set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fexceptions")
include_directories($ENV{Python_INCLUDE_DIRS})

add_library(hello MODULE hello.cpp)

set_target_properties(hello PROPERTIES PREFIX "")
export CMAKE_TOOLCHAIN_FILE="$(pyodide config get cmake_toolchain_file)"
export SIDE_MODULE_CFLAGS="$(pyodide config get cflags)"
export SIDE_MODULE_LDFLAGS="$(pyodide config get ldflags)"
export SIDE_MODULE_CXXFLAGS="$(pyodide config get cxxflags)"
export Python_INCLUDE_DIRS="$(pyodide config get python_include_dir)"
mkdir build
cd build
rm -rf ./* && emcmake cmake .. -G Ninja
ninja -v
wasm-python -c "import hello; print(hello.add(5,2))"

# Pyodide has suffered a fatal error. Please report this to the Pyodide maintainers.
# The cause of the fatal error was:
# Error: Dynamic linking error: cannot resolve symbol invoke_ii

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions