ZenDNN Shared Library undefined symbol ggml_get_type_traits_cpu

[DaAwesomeP]

After building with ZenDNN I am seeing this error when trying to load the backend:

load_backend: failed to load libggml-zendnn.so: libggml-zendnn.so: undefined symbol: ggml_get_type_traits_cpu

I assume it is referring to these lines:

ggml/src/ggml-zendnn/ggml-zendnn.cpp

Lines 125 to 126 in ac0c8be

	ggml_type const vec_dot_type = ggml_get_type_traits_cpu(src0->type)->vec_dot_type;
	ggml_from_float_t const from_float = ggml_get_type_traits_cpu(vec_dot_type)->from_float;

Maybe this is related to being in a separate shared lib from ggml-cpu? The file does include gghml-cpu.h but isn't linked to any of the ggml-cpu shared libs (and can't because they're modules). I noticed that ggml-blas does not make use of the ggml_get_type_traits_cpu function, so it doesn't have the same issue.

Not sure what the best solution is here. The easy thing would be to move ggml_get_type_traits_cpu to ggml-base where any backend can access it.

My GGML options:

GGML_LTO ON
BUILD_SHARED_LIBS ON
GGML_BACKEND_DL ON
GGML_NATIVE OFF
GGML_CPU ON
GGML_CPU_ALL_VARIANTS ON
GGML_ZENDNN ON

cc @z-vishal

[Djip007]

did it work if you remove BUILD_SHARED_LIBS ON ?

[z-vishal]

Hi @DaAwesomeP thanks for creating the issue

I tested with BUILD_SHARED_LIBS=ON(this is on by default for Linux with non-MinGW compilers), but GGML_BACKEND_DL=OFF and ZenDNN loaded fine, so shared libraries themselves aren't the problem.

The issue is specifically with GGML_BACKEND_DL:

• When GGML_BACKEND_DL=OFF, the ggml_add_backend_library function creates regular libs and links them publicly into the main ggml library
  https://github.com/ggml-org/llama.cpp/blob/ced765be44ce173c374f295b3c6f4175f8fd109b/ggml/src/CMakeLists.txt#L258C5-L262C12

• But when GGML_BACKEND_DL=ON, backends are built as MODULE libs
  https://github.com/ggml-org/llama.cpp/blob/ced765be44ce173c374f295b3c6f4175f8fd109b/ggml/src/CMakeLists.txt#L247
  then they cannot access symbols from other MODULE libs and each backend only gets symbols from ggml-base

As you suggested, we can move ggml_get_type_traits_cpu from ggml-cpu to ggml-base so other backends can access it, for those who need it with dynamic loading.

For now, you can build without GGML_BACKEND_DL=ON
will work on a PR to fix this properly

[z-vishal]

Hi @DaAwesomeP this issue has been addressed in this PR: ggml-org/llama.cpp#19159

[z-vishal]

Hi @ggerganov could you please close this issue? It has been addressed in the PR linked above and has also been synced in the GGML repo

[DaAwesomeP]

Thank you very much @z-vishal! I will try this out next week.