"ERROR: LoadError: No appropriate factory for type v" on for clang compiler

[Gobutah]

On my system this bash script

cat << EOF > meson.build
project('CxxWrap', 'cpp',
version : '0.1',
default_options : ['warning_level=3', 'cpp_std=c++17'])

compiler = meson.get_compiler('cpp')
##############################
# CxxWrap dirs
##############################
r = run_command('julia','--project', '-e', 'using CxxWrap; print(CxxWrap.prefix_path())')
assert(r.returncode()==0, 'Cannot get CxxWrap path')
cxxwrap_libdir = r.stdout()+'/lib'
cxxwrap_incdir = r.stdout()+'/include'
lib_cxxwrap = compiler.find_library('cxxwrap_julia', dirs : [cxxwrap_libdir])

##############################
# julia dirs
##############################
r = run_command('julia', '-e', 'print(Sys.BINDIR[1:end-4])')
assert(r.returncode()==0, 'Cannot get Julia lib path')
julia_libdir = r.stdout()+'/lib'
julia_incdir = r.stdout()+'/include/julia'
lib_julia = compiler.find_library('julia',
               dirs : [julia_libdir])

lib = shared_library( 'demo', ['libdemo.cpp'],
  include_directories: [ cxxwrap_incdir, julia_incdir ],
  dependencies: [lib_cxxwrap,lib_julia],
  cpp_args : '-DJULIA_ENABLE_THREADING')
EOF

cat << EOF > libdemo.cpp
#include "jlcxx/array.hpp"
#include "jlcxx/module.hpp"

struct Demo {
  Demo(){};
  jlcxx::Array<int> test() const {
    jlcxx::Array<int> jres;
    jres.push_back(0);
    return jres;
  }
};

JLCXX_MODULE define_julia_module(jlcxx::Module &mod) {
  mod.add_type<Demo>("Demo").method("test", &Demo::test);
}
EOF

cat << EOF > demo.jl
module M
using CxxWrap
@wrapmodule(() -> joinpath("build/libdemo.so"),:define_julia_module)
export Demo
export test
end
using Main.M
d=Demo()
M.test(d)
EOF


CXX=clang++ meson setup build
cd build
ninja
cd ..
julia demo.jl

produces the following output:

The Meson build system
Version: 0.61.2
Source dir: /home/user/tmp
Build dir: /home/user/tmp/build
Build type: native build
Project name: CxxWrap
Project version: 0.1
C++ compiler for the host machine: clang++ (clang 14.0.0-1ubuntu1 "Ubuntu clang version 14.0.0-1ubuntu1.1")
C++ linker for the host machine: clang++ ld.bfd 2.38
Host machine cpu family: x86_64
Host machine cpu: x86_64
WARNING: You should add the boolean check kwarg to the run_command call.
         It currently defaults to false,
         but it will default to true in future releases of meson.
         See also: https://github.com/mesonbuild/meson/issues/9300
Library cxxwrap_julia found: YES
Library julia found: YES
Build targets in project: 1

Found ninja-1.10.1 at /usr/bin/ninja
[1/2] Compiling C++ object libdemo.so.p/libdemo.cpp.o
In file included from ../libdemo.cpp:2:
/home/user/.julia/artifacts/7b2099c0a944e2b2d7f7ca3254c9681a9bf579a0/include/jlcxx/module.hpp:1205:16: warning: unused variable 'is_abstract' [-Wunused-variable]
    const bool is_abstract = jl_is_abstracttype(m_dt);
               ^
/home/user/.julia/artifacts/7b2099c0a944e2b2d7f7ca3254c9681a9bf579a0/include/jlcxx/module.hpp:753:23: warning: lambda capture 'this' is not used [-Wunused-lambda-capture]
      method("copy", [this](const T& other)
                      ^
/home/user/.julia/artifacts/7b2099c0a944e2b2d7f7ca3254c9681a9bf579a0/include/jlcxx/module.hpp:1324:5: note: in instantiation of function template specialization 'jlcxx::Module::add_copy_constructor<Demo>' requested here
    add_copy_constructor<T>(base_dt);
    ^
/home/user/.julia/artifacts/7b2099c0a944e2b2d7f7ca3254c9681a9bf579a0/include/jlcxx/module.hpp:1341:10: note: in instantiation of function template specialization 'jlcxx::Module::add_type_internal<Demo, jlcxx::ParameterList<>, _jl_datatype_t>' requested here
  return add_type_internal<T, SuperParametersT>(name, super);
         ^
../libdemo.cpp:14:7: note: in instantiation of function template specialization 'jlcxx::Module::add_type<Demo, jlcxx::ParameterList<>, _jl_datatype_t>' requested here
  mod.add_type<Demo>("Demo").method("test", &Demo::test);
      ^
2 warnings generated.
[2/2] Linking target libdemo.so
C++ exception while wrapping module M: No appropriate factory for type v
ERROR: LoadError: No appropriate factory for type v
Stacktrace:
 [1] register_julia_module
   @ ~/.julia/packages/CxxWrap/I0P2E/src/CxxWrap.jl:432 [inlined]
 [2] readmodule(so_path_cb::Main.M.var"#1#2", funcname::Symbol, m::Module, flags::Nothing)
   @ CxxWrap.CxxWrapCore ~/.julia/packages/CxxWrap/I0P2E/src/CxxWrap.jl:825
 [3] wrapmodule(so_path_cb::Function, funcname::Symbol, m::Module, flags::Nothing)
   @ CxxWrap.CxxWrapCore ~/.julia/packages/CxxWrap/I0P2E/src/CxxWrap.jl:835
in expression starting at /home/user/tmp/demo.jl:1

C++ compiler for the host machine: clang++ (clang 14.0.0-1ubuntu1 "Ubuntu clang version 14.0.0-1ubuntu1.1")

[Gobutah]

Other clang (AOCC 5.0) same issue:
If I replace the line

CXX=clang++ meson setup build

by

CXX=/opt/AMD/aocc-compiler-5.0.0/bin/clang++ CC=/opt/AMD/aocc-compiler-5.0.0/bin/clang meson setup build

I get the same issue:

The Meson build system
Version: 0.61.2
Source dir: /home/user/tmp
Build dir: /home/user/tmp/build
Build type: native build
Project name: CxxWrap
Project version: 0.1
C++ compiler for the host machine: /opt/AMD/aocc-compiler-5.0.0/bin/clang++ (clang 17.0.6 "AMD clang version 17.0.6 (CLANG: AOCC_5.0.0-Build#1377 2024_09_24)")
C++ linker for the host machine: /opt/AMD/aocc-compiler-5.0.0/bin/clang++ ld.lld 17.0.6
Host machine cpu family: x86_64
Host machine cpu: x86_64
WARNING: You should add the boolean check kwarg to the run_command call.
         It currently defaults to false,
         but it will default to true in future releases of meson.
         See also: https://github.com/mesonbuild/meson/issues/9300
Library cxxwrap_julia found: YES
Library julia found: YES
Build targets in project: 1

Found ninja-1.10.1 at /usr/bin/ninja
[1/2] Compiling C++ object libdemo.so.p/libdemo.cpp.o
In file included from ../libdemo.cpp:2:
/home/user/.julia/artifacts/7b2099c0a944e2b2d7f7ca3254c9681a9bf579a0/include/jlcxx/module.hpp:1205:16: warning: unused variable 'is_abstract' [-Wunused-variable]
 1205 |     const bool is_abstract = jl_is_abstracttype(m_dt);
      |                ^~~~~~~~~~~
/home/user/.julia/artifacts/7b2099c0a944e2b2d7f7ca3254c9681a9bf579a0/include/jlcxx/module.hpp:753:23: warning: lambda capture 'this' is not used [-Wunused-lambda-capture]
  753 |       method("copy", [this](const T& other)
      |                       ^
/home/user/.julia/artifacts/7b2099c0a944e2b2d7f7ca3254c9681a9bf579a0/include/jlcxx/module.hpp:753:22: note: while substituting into a lambda expression here
  753 |       method("copy", [this](const T& other)
      |                      ^
/home/user/.julia/artifacts/7b2099c0a944e2b2d7f7ca3254c9681a9bf579a0/include/jlcxx/module.hpp:1324:5: note: in instantiation of function template specialization 'jlcxx::Module::add_copy_constructor<Demo>' requested here
 1324 |     add_copy_constructor<T>(base_dt);
      |     ^
/home/user/.julia/artifacts/7b2099c0a944e2b2d7f7ca3254c9681a9bf579a0/include/jlcxx/module.hpp:1341:10: note: in instantiation of function template specialization 'jlcxx::Module::add_type_internal<Demo, jlcxx::ParameterList<>, _jl_datatype_t>' requested here
 1341 |   return add_type_internal<T, SuperParametersT>(name, super);
      |          ^
../libdemo.cpp:14:7: note: in instantiation of function template specialization 'jlcxx::Module::add_type<Demo, jlcxx::ParameterList<>, _jl_datatype_t>' requested here
   14 |   mod.add_type<Demo>("Demo").method("test", &Demo::test);
      |       ^
2 warnings generated.
[2/2] Linking target libdemo.so
C++ exception while wrapping module M: No appropriate factory for type v
ERROR: LoadError: No appropriate factory for type v
Stacktrace:
 [1] register_julia_module
   @ ~/.julia/packages/CxxWrap/I0P2E/src/CxxWrap.jl:432 [inlined]
 [2] readmodule(so_path_cb::Main.M.var"#1#2", funcname::Symbol, m::Module, flags::Nothing)
   @ CxxWrap.CxxWrapCore ~/.julia/packages/CxxWrap/I0P2E/src/CxxWrap.jl:825
 [3] wrapmodule(so_path_cb::Function, funcname::Symbol, m::Module, flags::Nothing)
   @ CxxWrap.CxxWrapCore ~/.julia/packages/CxxWrap/I0P2E/src/CxxWrap.jl:835
in expression starting at /home/user/tmp/demo.jl:1

C++ compiler for the host machine: /opt/AMD/aocc-compiler-5.0.0/bin/clang++ (clang 17.0.6 "AMD clang version 17.0.6 (CLANG: AOCC_5.0.0-Build#1377 2024_09_24)")

[Gobutah]

I don't get this issue on GCC.

[barche]

Are you building against a Julia compiled with the same compiler? This kind of error typically appears when there are subtle incompatibilities between the compiler used to build Julia itself and libcxxwrap-julia.

[Gobutah]

No. I was using precompiled julia. But I don't understand why it is necessary since Julia is not using C++ itself, so only C ABI would be needed to be equal. https://stackoverflow.com/a/11274684/6212870 says that it should be compatible, especially since only shared objects are mixed between compilers here and not compiler-specific object files.

Can you explain in this context why using the same compiler is necessary? I thought Julia (written in C) is linking against a shared library but only hooks into C entry points (extern "C") so it should be no problem to use a different compiler?

I will try and report what happens when I compile julia itself with the same compiler (clang).

[Gobutah]

I compiled Julia with the same compiler /opt/AMD/aocc-compiler-5.0.0/bin/clang++ (clang 17.0.6 "AMD clang version 17.0.6 (CLANG: AOCC_5.0.0-Build#1377 2024_09_24)") and the issue persists.
I will now delete the .julia directory and try again.

[Gobutah]

It still does not work:

$ julia demo.jl 
C++ exception while wrapping module M: No appropriate factory for type v
ERROR: LoadError: No appropriate factory for type v
Stacktrace:
 [1] register_julia_module
   @ ~/.julia/packages/CxxWrap/I0P2E/src/CxxWrap.jl:432 [inlined]
 [2] readmodule(so_path_cb::Main.M.var"#1#2", funcname::Symbol, m::Module, flags::Nothing)
   @ CxxWrap.CxxWrapCore ~/.julia/packages/CxxWrap/I0P2E/src/CxxWrap.jl:825
 [3] wrapmodule(so_path_cb::Function, funcname::Symbol, m::Module, flags::Nothing)
   @ CxxWrap.CxxWrapCore ~/.julia/packages/CxxWrap/I0P2E/src/CxxWrap.jl:835
in expression starting at /home/peter/tmp/demo.jl:1

With both, julia and also the above example building with /opt/AMD/aocc-compiler-5.0.0/bin/clang++ (clang 17.0.6 "AMD clang version 17.0.6 (CLANG: AOCC_5.0.0-Build#1377 2024_09_24)"). Even after deleting .julia directory and reinstalling CxxWrap, still it does not work. I double checked that I am indeed using the self-compiled julia (I compiled v1.11.2 and before I used precompiled v.1.11.4 so I am sure that it I now used the self-compiled julia).

[Gobutah]

It seems CxxWrap is not compatible with clang compiler.

[barche]

What happens if you compile libcxxwrap-julia with JLCXX_USE_TYPE_MAP defined?

[Gobutah]

I modified the demo bash file for reproducibility:

cat << EOF > meson.build
project('CxxWrap', 'cpp',
version : '0.1',
default_options : ['warning_level=3', 'cpp_std=c++17'])

compiler = meson.get_compiler('cpp')
##############################
# CxxWrap dirs
##############################
r = run_command('julia','--project', '-e', 'using CxxWrap; print(CxxWrap.prefix_path())')
assert(r.returncode()==0, 'Cannot get CxxWrap path')
cxxwrap_libdir = r.stdout()+'/lib'
cxxwrap_incdir = r.stdout()+'/include'
lib_cxxwrap = compiler.find_library('cxxwrap_julia', dirs : [cxxwrap_libdir])

##############################
# julia dirs
##############################
r = run_command('julia', '-e', 'print(Sys.BINDIR[1:end-4])')
assert(r.returncode()==0, 'Cannot get Julia lib path')
julia_libdir = r.stdout()+'/lib'
julia_incdir = r.stdout()+'/include/julia'
lib_julia = compiler.find_library('julia',
               dirs : [julia_libdir])

lib = shared_library( 'demo', ['libdemo.cpp'],
  include_directories: [ cxxwrap_incdir, julia_incdir ],
  dependencies: [lib_cxxwrap,lib_julia],
  cpp_args : ['-DJULIA_ENABLE_THREADING', '-DJLCXX_USE_TYPE_MAP'],
  c_args : ['-DJLCXX_USE_TYPE_MAP'])
EOF

cat << EOF > libdemo.cpp
#include "jlcxx/array.hpp"
#include "jlcxx/module.hpp"

struct Demo {
  Demo(){};
  jlcxx::Array<int> test() const {
    jlcxx::Array<int> jres;
    jres.push_back(0);
    return jres;
  }
};

JLCXX_MODULE define_julia_module(jlcxx::Module &mod) {
  mod.add_type<Demo>("Demo").method("test", &Demo::test);
}
EOF

cat << EOF > demo.jl
module M
using CxxWrap
@wrapmodule(() -> joinpath("build/libdemo.so"),:define_julia_module)
export Demo
export test
end
using Main.M
d=Demo()
M.test(d)
EOF


CXX=/opt/AMD/aocc-compiler-5.0.0/bin/clang++ CC=/opt/AMD/aocc-compiler-5.0.0/bin/clang meson setup build
cd build
ninja
cd ..
julia demo.jl

It does not link with -DJLCXX_USE_TYPE_MAP:

[1/1] Linking target libdemo.so
FAILED: libdemo.so 
/opt/AMD/aocc-compiler-5.0.0/bin/clang++  -o libdemo.so libdemo.so.p/libdemo.cpp.o -Wl,--as-needed -Wl,--no-undefined -shared -fPIC -Wl,--start-group -Wl,-soname,libdemo.so -Wl,-rpath,/home/user/.julia/artifacts/7b2099c0a944e2b2d7f7ca3254c9681a9bf579a0/lib:/opt/julia-1.11.2/lib -Wl,-rpath-link,/home/user/.julia/artifacts/7b2099c0a944e2b2d7f7ca3254c9681a9bf579a0/lib -Wl,-rpath-link,/opt/julia-1.11.2/lib /home/user/.julia/artifacts/7b2099c0a944e2b2d7f7ca3254c9681a9bf579a0/lib/libcxxwrap_julia.so /opt/julia-1.11.2/lib/libjulia.so -Wl,--end-group
ld.lld: error: undefined symbol: jlcxx::jlcxx_type(std::type_index)
>>> referenced by type_conversion.hpp:363 (/home/user/.julia/artifacts/7b2099c0a944e2b2d7f7ca3254c9681a9bf579a0/include/jlcxx/type_conversion.hpp:363)
>>>               libdemo.so.p/libdemo.cpp.o:(jlcxx::HashedCache<int>::value())
>>> referenced by type_conversion.hpp:363 (/home/user/.julia/artifacts/7b2099c0a944e2b2d7f7ca3254c9681a9bf579a0/include/jlcxx/type_conversion.hpp:363)
>>>               libdemo.so.p/libdemo.cpp.o:(jlcxx::HashedCache<Demo>::value())
>>> referenced by type_conversion.hpp:363 (/home/user/.julia/artifacts/7b2099c0a944e2b2d7f7ca3254c9681a9bf579a0/include/jlcxx/type_conversion.hpp:363)
>>>               libdemo.so.p/libdemo.cpp.o:(jlcxx::HashedCache<jlcxx::BoxedValue<Demo>>::value())
>>> referenced 4 more times

ld.lld: error: undefined symbol: jlcxx::jlcxx_constreftype(std::type_index)
>>> referenced by type_conversion.hpp:381 (/home/user/.julia/artifacts/7b2099c0a944e2b2d7f7ca3254c9681a9bf579a0/include/jlcxx/type_conversion.hpp:381)
>>>               libdemo.so.p/libdemo.cpp.o:(jlcxx::HashedCache<Demo const&>::value())
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.

[Gobutah]

I think I only compiled my cpp file with that flags. In the director /home/user/.julia/artifacts/7b2099c0a944e2b2d7f7ca3254c9681a9bf579a0/lib I have these libs:

cmake/                       libfunctions.so*
libbasic_types.so*           libhello.so*
libcxxwrap_julia.so@         libinheritance.so*
libcxxwrap_julia.so.0@       libjlcxx_containers.so*
libcxxwrap_julia.so.0.13.4*  libparametric.so*
libcxxwrap_julia_stl.so*     libpointer_modification.so*
libexcept.so*                libtypes.so*
libextended.so*

How do I recompile them also with that flag? Maybe indeed I have not even used the same compiler with these before. There are 3 not 2 places to select a compiler:

1. compiling Julia
2. compiling these libs
3. compiling my own demo library

Before I confirmed to have used clang and AOCC-clang for 1. and 3. but I am not sure it was used for 2. So can you give me a short instruction to select a compiler for these? When are they compiled? Probably when I add CxxWrap in Julia for the first time right? So I should delete the .julia directory and add the package again. But how do I select C++ compiler and flags for thiem?

[barche]

I just tried using clang on Arch linux, and I see the same error using the default settings. To get around it, you need to compile libcxxwrap-julia and your own library with JLCXX_USE_TYPE_MAP defined, e.g. by setting CMAKE_CXX_FLAGS to -DJLCXX_USE_TYPE_MAP. The reason this happen sis that there is no universal way in C++ to identify a type reliably and uniquely across a program consisting of multiple shared libraries, see the comment and the links here: https://github.com/JuliaInterop/libcxxwrap-julia/blob/9b83f9f1b7eeb7a4d84fbfb9acf406dbe25fa3ee/src/jlcxx.cpp#L340