Error when xeus-cpp is installed with xeus-python modules in Jupyterlite

[davy39]

I just noticed that xeus-cpp can't be used when xeus-python modules are installed in the same wasm env (matplotlib, ipycanvas, numpy, pandas) with Jupyterlite.

Maybe it is related to previous bug, about shared libraries management.

Here is the error I get :

Could not load dynamic lib: /tmp/incr_module_2.wasm
Error: bad export type for 'PyExc_ValueError': undefined
Failed to execute via ::process:Failed to load incremental module
Error: Compilation error! Could not load dynamic lib: /tmp/incr_module_2.wasm
Error: bad export type for 'PyExc_ValueError': undefined
Failed to execute via ::process:Failed to load incremental module

See a Jupyterlite deployment with this bug here.

[anutosh491]

Thanks for the report.

I think this is a known bug.

cc @DerThorsten should be able to educate us if we found a way through this ?

[davy39]

Actually I did not read carefully enough the previous answer from @martinRenou. I'm gonna try using multiple environments with Jupyterlite.

[DerThorsten]

I agree with the previous answer from @martinRenou which suggests to use multiple environments.
The problem is with emscripten<4 we need to "preload"/"pre-open" all shared libraries found in the environment before using the kernel. Since we cannot realy distinguish to which kernel a .so belongs, we will preload python libs for the xeus-cpp kernel and vice verca.
Now some shared libaries for python assume the existance of some symbols which are only found in the "host-executable", ie xeus-python itself.
So when we start the xeus-cpp kernel, we pre-load some python libraries and these libraries will look for symbols like "PyExc_ValueError".

With emscripten>=4 (and the respective emscripten-forge branch / channel) one can skip the preloading of shared libraries and only open the shared libraries when needed. This should eliminate issues like this one.

[martinRenou]

With emscripten>=4

Can't wait for this!

[davy39]

Using multiple environments fixed my problems 👍

Thanks