[CANN] Compile bug: no matching function for call to 'CastIntrinsicsImpl'

[Qiliang]

Git commit

9f91251

Operating systems

Linux

GGML backends

CPU

Problem description & steps to reproduce

ggml CANN backend

Ascend NPU: 910A or 910B. not 910b1, 910b2, 910b3

Ascend-hdk-910-npu-driver_23.0.0_linux-aarch64.run --quiet --docker
Ascend-cann-toolkit_8.0.RC2_linux-aarch64.run --force --quiet --install-for-all --full
Ascend-cann-kernels-910_8.0.RC2_linux.run --install --install-for-all --quiet

cmake -B build -DGGML_CANN=on -DCMAKE_BUILD_TYPE=release -DSOC_TYPE=ascend910a
or
cmake -B build -DGGML_CANN=on -DCMAKE_BUILD_TYPE=release -DSOC_TYPE=ascend910b

cmake --build build --config release --target llama-cli

[ 41%] Performing build step for 'ascendc_kernels_device'
[ 12%] Building CXX object CMakeFiles/device_obj.dir/home/ma-user/llama.cpp/build/auto_gen/ascendc_kernels/auto_gen_dup.cpp.o
[ 25%] Building CXX object CMakeFiles/device_obj.dir/home/ma-user/llama.cpp/build/auto_gen/ascendc_kernels/auto_gen_get_row_f16.cpp.o
[ 37%] Building CXX object CMakeFiles/device_obj.dir/home/ma-user/llama.cpp/build/auto_gen/ascendc_kernels/auto_gen_get_row_f32.cpp.o
[ 50%] Building CXX object CMakeFiles/device_obj.dir/home/ma-user/llama.cpp/build/auto_gen/ascendc_kernels/auto_gen_get_row_q4_0.cpp.o
In file included from /home/ma-user/llama.cpp/build/auto_gen/ascendc_kernels/auto_gen_get_row_q4_0.cpp:7:
In file included from /home/ma-user/llama.cpp/ggml/src/ggml-cann/kernels/get_row_q4_0.cpp:1:
In file included from /usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/kernel_operator.h:27:
In file included from /usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/interface/kernel_operator_intf.h:48:
In file included from /usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/interface/kernel_operator_vec_vconv_intf.h:26:
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:450:5: error: no matching function for call to 'CastIntrinsicsImpl'
    CastIntrinsicsImpl(dst, src, roundMode, repeatTimes, repeatParams);
    ^~~~~~~~~~~~~~~~~~
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:402:9: note: in instantiation of function template specialization 'AscendC::CastImpl<half, AscendC::IntegerSubType<4, true>, true>' requested here
        CastImpl((__ubuf__ U*)(dst + dstOffset), (__ubuf__ T*)(src + srcOffset), roundMode, fullMask,
        ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/inner_interface/inner_kernel_operator_vec_vconv_intf.cppm:130:5: note: in instantiation of function template specialization 'AscendC::CastImpl<half, AscendC::IntegerSubType<4, true>>' requested here
    CastImpl((__ubuf__ T1*)dstLocal.GetPhyAddr(), (__ubuf__ T2*)srcLocal.GetPhyAddr(), round_mode, calCount);
    ^
/home/ma-user/llama.cpp/ggml/src/ggml-cann/kernels/get_row_q4_0.cpp:116:9: note: in instantiation of function template specialization 'AscendC::Cast<half, AscendC::IntegerSubType<4, true>>' requested here
        Cast(cast_local, input_local, RoundMode::CAST_NONE, QK4_0);
        ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:26:24: note: candidate function not viable: no known conversion from '__ubuf__ AscendC::IntegerSubType<4, true> *' to '__ubuf__ int32_t *' (aka '__ubuf__ int *') for 2nd argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ half* dst, __ubuf__ int32_t* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:33:24: note: candidate function not viable: no known conversion from '__ubuf__ AscendC::IntegerSubType<4, true> *' to '__ubuf__ int8_t *' (aka '__ubuf__ signed char *') for 2nd argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ half* dst, __ubuf__ int8_t* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:47:24: note: candidate function not viable: no known conversion from '__ubuf__ AscendC::IntegerSubType<4, true> *' to '__ubuf__ uint8_t *' (aka '__ubuf__ unsigned char *') for 2nd argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ half* dst, __ubuf__ uint8_t* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:215:24: note: candidate function not viable: no known conversion from '__ubuf__ AscendC::IntegerSubType<4, true> *' to '__ubuf__ float *' for 2nd argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ half* dst, __ubuf__ float* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:323:24: note: candidate function not viable: no known conversion from '__ubuf__ AscendC::IntegerSubType<4, true> *' to '__ubuf__ int16_t *' (aka '__ubuf__ short *') for 2nd argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ half* dst, __ubuf__ int16_t* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:61:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ float *' for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ float* dst, __ubuf__ int32_t* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:75:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ float *' for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ float* dst, __ubuf__ half* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:89:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ int32_t *' (aka '__ubuf__ int *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ int32_t* dst, __ubuf__ half* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:131:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ int8_t *' (aka '__ubuf__ signed char *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ int8_t* dst, __ubuf__ half* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:173:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ uint8_t *' (aka '__ubuf__ unsigned char *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ uint8_t* dst, __ubuf__ half* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:263:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ int32_t *' (aka '__ubuf__ int *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ int32_t* dst, __ubuf__ float* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:305:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ int16_t *' (aka '__ubuf__ short *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ int16_t* dst, __ubuf__ half* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:311:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ uint8_t *' (aka '__ubuf__ unsigned char *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ uint8_t* dst, __ubuf__ int16_t* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:317:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ int8_t *' (aka '__ubuf__ signed char *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ int8_t* dst, __ubuf__ int16_t* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:329:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ float *' for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ float* dst, __ubuf__ float* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:335:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ int64_t *' (aka '__ubuf__ long *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ int64_t* dst, __ubuf__ float* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:341:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ int16_t *' (aka '__ubuf__ short *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ int16_t* dst, __ubuf__ float* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:347:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ float *' for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ float* dst, __ubuf__ int16_t* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:353:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ int16_t *' (aka '__ubuf__ short *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ int16_t* dst, __ubuf__ int32_t* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:359:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ int64_t *' (aka '__ubuf__ long *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ int64_t* dst, __ubuf__ int32_t* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:365:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ float *' for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ float* dst, __ubuf__ int64_t* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:371:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ int32_t *' (aka '__ubuf__ int *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ int32_t* dst, __ubuf__ int64_t* src, const RoundMode& roundMode,
                       ^
1 error generated.
gmake[6]: *** [CMakeFiles/device_obj.dir/build.make:118: CMakeFiles/device_obj.dir/home/ma-user/llama.cpp/build/auto_gen/ascendc_kernels/auto_gen_get_row_q4_0.cpp.o] Error 1
gmake[5]: *** [CMakeFiles/Makefile2:85: CMakeFiles/device_obj.dir/all] Error 2
gmake[4]: *** [Makefile:91: all] Error 2
gmake[3]: *** [ggml/src/ggml-cann/kernels/CMakeFiles/ascendc_kernels_device.dir/build.make:86: ggml/src/ggml-cann/kernels/ascendc_kernels_device-prefix/src/ascendc_kernels_device-stamp/ascendc_kernels_device-build] Error 2
gmake[2]: *** [CMakeFiles/Makefile2:1867: ggml/src/ggml-cann/kernels/CMakeFiles/ascendc_kernels_device.dir/all] Error 2
gmake[1]: *** [CMakeFiles/Makefile2:3323: examples/main/CMakeFiles/llama-cli.dir/rule] Error 2
gmake: *** [Makefile:1232: llama-cli] Error 2

I see file : /usr/local/Ascend/ascend-toolkit/latest/compiler/tikcpp/ascendc_kernel_cmake/host_config.cmake

set(ascend910b_list ascend910b1 ascend910b2 ascend910b2c ascend910b3 ascend910b4 ascend910b4-1 ascend910c1 ascend910c2 ascend910c3 ascend910c4 ascend910c4-1)
set(ascend910_list  ascend910a ascend910proa ascend910b ascend910prob ascend910premiuma)
set(ascend310p_list ascend310p1 ascend310p3 ascend310p3vir01 ascend310p3vir02 ascend310p3vir04 ascend310p3vir08)
set(ascend310b_list ascend310b1 ascend310b2 ascend310b3 ascend310b4)
set(all_product ${ascend910b_list} ${ascend910_list} ${ascend310p_list})

if(NOT DEFINED SOC_VERSION)
    message(FATAL_ERROR "SOC_VERSION value not set.")
endif()

string(TOLOWER "${SOC_VERSION}" _LOWER_SOC_VERSION)

if(_LOWER_SOC_VERSION IN_LIST ascend910b_list)
    set(DYNAMIC_MODE ON)
    set(BUILD_MODE   aiv)
elseif(_LOWER_SOC_VERSION IN_LIST ascend910_list)
    set(BUILD_MODE   c100)
elseif(_LOWER_SOC_VERSION IN_LIST ascend310p_list)
    set(BUILD_MODE   m200)
elseif(_LOWER_SOC_VERSION IN_LIST ascend310b_list)
    set(BUILD_MODE   m300)
else()
    message(FATAL_ERROR "SOC_VERSION ${SOC_VERSION} does not support, the support list is ${all_product}")
endif()

if(NOT DEFINED RUN_MODE)
    set(RUN_MODE "npu")
endif()

if(NOT DEFINED ASCEND_KERNEL_LAUNCH_ONLY)
    set(ASCEND_KERNEL_LAUNCH_ONLY OFF)
endif()

.....

ascend910_list contains ascend910a and ascend910b

First Bad Commit

No response

Relevant log output

[ 41%] Performing build step for 'ascendc_kernels_device'
[ 12%] Building CXX object CMakeFiles/device_obj.dir/home/ma-user/llama.cpp/build/auto_gen/ascendc_kernels/auto_gen_dup.cpp.o
[ 25%] Building CXX object CMakeFiles/device_obj.dir/home/ma-user/llama.cpp/build/auto_gen/ascendc_kernels/auto_gen_get_row_f16.cpp.o
[ 37%] Building CXX object CMakeFiles/device_obj.dir/home/ma-user/llama.cpp/build/auto_gen/ascendc_kernels/auto_gen_get_row_f32.cpp.o
[ 50%] Building CXX object CMakeFiles/device_obj.dir/home/ma-user/llama.cpp/build/auto_gen/ascendc_kernels/auto_gen_get_row_q4_0.cpp.o
In file included from /home/ma-user/llama.cpp/build/auto_gen/ascendc_kernels/auto_gen_get_row_q4_0.cpp:7:
In file included from /home/ma-user/llama.cpp/ggml/src/ggml-cann/kernels/get_row_q4_0.cpp:1:
In file included from /usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/kernel_operator.h:27:
In file included from /usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/interface/kernel_operator_intf.h:48:
In file included from /usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/interface/kernel_operator_vec_vconv_intf.h:26:
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:450:5: error: no matching function for call to 'CastIntrinsicsImpl'
    CastIntrinsicsImpl(dst, src, roundMode, repeatTimes, repeatParams);
    ^~~~~~~~~~~~~~~~~~
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:402:9: note: in instantiation of function template specialization 'AscendC::CastImpl<half, AscendC::IntegerSubType<4, true>, true>' requested here
        CastImpl((__ubuf__ U*)(dst + dstOffset), (__ubuf__ T*)(src + srcOffset), roundMode, fullMask,
        ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/inner_interface/inner_kernel_operator_vec_vconv_intf.cppm:130:5: note: in instantiation of function template specialization 'AscendC::CastImpl<half, AscendC::IntegerSubType<4, true>>' requested here
    CastImpl((__ubuf__ T1*)dstLocal.GetPhyAddr(), (__ubuf__ T2*)srcLocal.GetPhyAddr(), round_mode, calCount);
    ^
/home/ma-user/llama.cpp/ggml/src/ggml-cann/kernels/get_row_q4_0.cpp:116:9: note: in instantiation of function template specialization 'AscendC::Cast<half, AscendC::IntegerSubType<4, true>>' requested here
        Cast(cast_local, input_local, RoundMode::CAST_NONE, QK4_0);
        ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:26:24: note: candidate function not viable: no known conversion from '__ubuf__ AscendC::IntegerSubType<4, true> *' to '__ubuf__ int32_t *' (aka '__ubuf__ int *') for 2nd argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ half* dst, __ubuf__ int32_t* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:33:24: note: candidate function not viable: no known conversion from '__ubuf__ AscendC::IntegerSubType<4, true> *' to '__ubuf__ int8_t *' (aka '__ubuf__ signed char *') for 2nd argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ half* dst, __ubuf__ int8_t* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:47:24: note: candidate function not viable: no known conversion from '__ubuf__ AscendC::IntegerSubType<4, true> *' to '__ubuf__ uint8_t *' (aka '__ubuf__ unsigned char *') for 2nd argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ half* dst, __ubuf__ uint8_t* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:215:24: note: candidate function not viable: no known conversion from '__ubuf__ AscendC::IntegerSubType<4, true> *' to '__ubuf__ float *' for 2nd argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ half* dst, __ubuf__ float* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:323:24: note: candidate function not viable: no known conversion from '__ubuf__ AscendC::IntegerSubType<4, true> *' to '__ubuf__ int16_t *' (aka '__ubuf__ short *') for 2nd argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ half* dst, __ubuf__ int16_t* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:61:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ float *' for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ float* dst, __ubuf__ int32_t* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:75:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ float *' for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ float* dst, __ubuf__ half* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:89:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ int32_t *' (aka '__ubuf__ int *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ int32_t* dst, __ubuf__ half* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:131:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ int8_t *' (aka '__ubuf__ signed char *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ int8_t* dst, __ubuf__ half* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:173:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ uint8_t *' (aka '__ubuf__ unsigned char *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ uint8_t* dst, __ubuf__ half* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:263:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ int32_t *' (aka '__ubuf__ int *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ int32_t* dst, __ubuf__ float* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:305:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ int16_t *' (aka '__ubuf__ short *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ int16_t* dst, __ubuf__ half* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:311:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ uint8_t *' (aka '__ubuf__ unsigned char *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ uint8_t* dst, __ubuf__ int16_t* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:317:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ int8_t *' (aka '__ubuf__ signed char *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ int8_t* dst, __ubuf__ int16_t* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:329:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ float *' for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ float* dst, __ubuf__ float* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:335:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ int64_t *' (aka '__ubuf__ long *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ int64_t* dst, __ubuf__ float* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:341:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ int16_t *' (aka '__ubuf__ short *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ int16_t* dst, __ubuf__ float* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:347:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ float *' for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ float* dst, __ubuf__ int16_t* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:353:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ int16_t *' (aka '__ubuf__ short *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ int16_t* dst, __ubuf__ int32_t* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:359:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ int64_t *' (aka '__ubuf__ long *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ int64_t* dst, __ubuf__ int32_t* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:365:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ float *' for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ float* dst, __ubuf__ int64_t* src, const RoundMode& roundMode,
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:371:24: note: candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ int32_t *' (aka '__ubuf__ int *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ int32_t* dst, __ubuf__ int64_t* src, const RoundMode& roundMode,
                       ^
1 error generated.
gmake[6]: *** [CMakeFiles/device_obj.dir/build.make:118: CMakeFiles/device_obj.dir/home/ma-user/llama.cpp/build/auto_gen/ascendc_kernels/auto_gen_get_row_q4_0.cpp.o] Error 1
gmake[5]: *** [CMakeFiles/Makefile2:85: CMakeFiles/device_obj.dir/all] Error 2
gmake[4]: *** [Makefile:91: all] Error 2
gmake[3]: *** [ggml/src/ggml-cann/kernels/CMakeFiles/ascendc_kernels_device.dir/build.make:86: ggml/src/ggml-cann/kernels/ascendc_kernels_device-prefix/src/ascendc_kernels_device-stamp/ascendc_kernels_device-build] Error 2
gmake[2]: *** [CMakeFiles/Makefile2:1867: ggml/src/ggml-cann/kernels/CMakeFiles/ascendc_kernels_device.dir/all] Error 2
gmake[1]: *** [CMakeFiles/Makefile2:3323: examples/main/CMakeFiles/llama-cli.dir/rule] Error 2
gmake: *** [Makefile:1232: llama-cli] Error 2

[leo-pony]

Manually specify SOC_TYPE currently has problem, plz see this issue #10519 .
You can build without specify SOC_TYPE, let auto-detect.
Notice:
Ascend910A and Ascend910B not support auto-detected now.

[hipudding]

We have verified with 910B3/910B4，but we have no 910A device currently. Can you paste all output after running npu-smi info ?

[Qiliang]

I try build it in huaweicloud - ModelArts - notebook

use my custom image:

 Ascend-hdk-910-npu-driver_23.0.0_linux-aarch64.run --quiet --docker &&  \
 Ascend-cann-toolkit_8.0.RC3.alpha003_linux-aarch64.run --force --quiet --install-for-all --full &&  \
 Ascend-cann-kernels-910_8.0.RC3.alpha003_linux-aarch64.run --install --install-for-all --quiet 

(MindSpore) [ma-user llama.cpp]$npu-smi info
+------------------------------------------------------------------------------------------------+
| npu-smi 23.0.rc3                 Version: 23.0.rc3                                             |
+---------------------------+---------------+----------------------------------------------------+
| NPU   Name                | Health        | Power(W)    Temp(C)           Hugepages-Usage(page)|
| Chip                      | Bus-Id        | AICore(%)   Memory-Usage(MB)  HBM-Usage(MB)        |
+===========================+===============+====================================================+
| 2     910A                | Warning       | 68.0        38                0    / 0             |
| 0                         | 0000:41:00.0  | 0           2041 / 15137      0    / 32768         |
+===========================+===============+====================================================+
+---------------------------+---------------+----------------------------------------------------+
| NPU     Chip              | Process id    | Process name             | Process memory(MB)      |
+===========================+===============+====================================================+
| No running processes found in NPU 2                                                            |
+===========================+===============+====================================================+

the same error:

-- The C compiler identification is GNU 12.2.0
-- The CXX compiler identification is GNU 12.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: /home/ma-user/work/llama.cpp/build/ggml/src/ggml-cann/kernels/ascendc_kernels_device-prefix/src/ascendc_kernels_device-build
[ 41%] Performing build step for 'ascendc_kernels_device'
[ 12%] Building CXX object CMakeFiles/device_obj.dir/home/ma-user/work/llama.cpp/build/auto_gen/ascendc_kernels/auto_gen_dup.cpp.o
[ 25%] Building CXX object CMakeFiles/device_obj.dir/home/ma-user/work/llama.cpp/build/auto_gen/ascendc_kernels/auto_gen_get_row_f16.cpp.o
[ 37%] Building CXX object CMakeFiles/device_obj.dir/home/ma-user/work/llama.cpp/build/auto_gen/ascendc_kernels/auto_gen_get_row_f32.cpp.o
[ 50%] Building CXX object CMakeFiles/device_obj.dir/home/ma-user/work/llama.cpp/build/auto_gen/ascendc_kernels/auto_gen_get_row_q4_0.cpp.o
In file included from /home/ma-user/work/llama.cpp/build/auto_gen/ascendc_kernels/auto_gen_get_row_q4_0.cpp:7:
In file included from /home/ma-user/work/llama.cpp/ggml/src/ggml-cann/kernels/get_row_q4_0.cpp:1:
In file included from /usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/kernel_operator.h:27:
In file included from /usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/interface/kernel_operator_intf.h:48:
In file included from /usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/interface/kernel_operator_vec_vconv_intf.h:26:
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:450:5: error: 
      no matching function for call to 'CastIntrinsicsImpl'
    CastIntrinsicsImpl(dst, src, roundMode, repeatTimes, repeatParams);
    ^~~~~~~~~~~~~~~~~~
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:402:9: note: 
      in instantiation of function template specialization 'AscendC::CastImpl<half,
      AscendC::IntegerSubType<4, true>, true>' requested here
        CastImpl((__ubuf__ U*)(dst + dstOffset), (__ubuf__ T*)(src + srcOffset), roundMode, fullMask,
        ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/inner_interface/inner_kernel_operator_vec_vconv_intf.cppm:131:5: note: 
      in instantiation of function template specialization 'AscendC::CastImpl<half,
      AscendC::IntegerSubType<4, true>>' requested here
    CastImpl((__ubuf__ T1*)dstLocal.GetPhyAddr(), (__ubuf__ T2*)srcLocal.GetPhyAddr(), round_mode, c...
    ^
/home/ma-user/work/llama.cpp/ggml/src/ggml-cann/kernels/get_row_q4_0.cpp:122:9: note: 
      in instantiation of function template specialization 'AscendC::Cast<half, AscendC::IntegerSubType<4,
      true>>' requested here
        Cast(cast_local, input_local, RoundMode::CAST_NONE, QK4_0);
        ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:26:24: note: 
      candidate function not viable: no known conversion from '__ubuf__ AscendC::IntegerSubType<4, true> *'
      to '__ubuf__ int32_t *' (aka '__ubuf__ int *') for 2nd argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ half* dst, __ubuf__ int32_t* src, const RoundMode...
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:33:24: note: 
      candidate function not viable: no known conversion from '__ubuf__ AscendC::IntegerSubType<4, true> *'
      to '__ubuf__ int8_t *' (aka '__ubuf__ signed char *') for 2nd argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ half* dst, __ubuf__ int8_t* src, const RoundMode...
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:47:24: note: 
      candidate function not viable: no known conversion from '__ubuf__ AscendC::IntegerSubType<4, true> *'
      to '__ubuf__ uint8_t *' (aka '__ubuf__ unsigned char *') for 2nd argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ half* dst, __ubuf__ uint8_t* src, const RoundMode...
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:215:24: note: 
      candidate function not viable: no known conversion from '__ubuf__ AscendC::IntegerSubType<4, true> *'
      to '__ubuf__ float *' for 2nd argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ half* dst, __ubuf__ float* src, const RoundMode& ...
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:323:24: note: 
      candidate function not viable: no known conversion from '__ubuf__ AscendC::IntegerSubType<4, true> *'
      to '__ubuf__ int16_t *' (aka '__ubuf__ short *') for 2nd argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ half* dst, __ubuf__ int16_t* src, const RoundMode...
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:61:24: note: 
      candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ float *' for
      1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ float* dst, __ubuf__ int32_t* src, const RoundMod...
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:75:24: note: 
      candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ float *' for
      1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ float* dst, __ubuf__ half* src, const RoundMode& ...
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:89:24: note: 
      candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ int32_t *'
      (aka '__ubuf__ int *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ int32_t* dst, __ubuf__ half* src, const RoundMode...
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:131:24: note: 
      candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ int8_t *'
      (aka '__ubuf__ signed char *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ int8_t* dst, __ubuf__ half* src, const RoundMode...
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:173:24: note: 
      candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ uint8_t *'
      (aka '__ubuf__ unsigned char *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ uint8_t* dst, __ubuf__ half* src, const RoundMode...
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:263:24: note: 
      candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ int32_t *'
      (aka '__ubuf__ int *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ int32_t* dst, __ubuf__ float* src, const RoundMod...
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:305:24: note: 
      candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ int16_t *'
      (aka '__ubuf__ short *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ int16_t* dst, __ubuf__ half* src, const RoundMode...
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:311:24: note: 
      candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ uint8_t *'
      (aka '__ubuf__ unsigned char *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ uint8_t* dst, __ubuf__ int16_t* src, const RoundM...
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:317:24: note: 
      candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ int8_t *'
      (aka '__ubuf__ signed char *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ int8_t* dst, __ubuf__ int16_t* src, const RoundMo...
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:329:24: note: 
      candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ float *' for
      1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ float* dst, __ubuf__ float* src, const RoundMode...
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:335:24: note: 
      candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ int64_t *'
      (aka '__ubuf__ long *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ int64_t* dst, __ubuf__ float* src, const RoundMod...
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:341:24: note: 
      candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ int16_t *'
      (aka '__ubuf__ short *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ int16_t* dst, __ubuf__ float* src, const RoundMod...
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:347:24: note: 
      candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ float *' for
      1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ float* dst, __ubuf__ int16_t* src, const RoundMod...
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:353:24: note: 
      candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ int16_t *'
      (aka '__ubuf__ short *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ int16_t* dst, __ubuf__ int32_t* src, const RoundM...
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:359:24: note: 
      candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ int64_t *'
      (aka '__ubuf__ long *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ int64_t* dst, __ubuf__ int32_t* src, const RoundM...
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:365:24: note: 
      candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ float *' for
      1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ float* dst, __ubuf__ int64_t* src, const RoundMod...
                       ^
/usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:371:24: note: 
      candidate function not viable: no known conversion from '__ubuf__ half *' to '__ubuf__ int32_t *'
      (aka '__ubuf__ int *') for 1st argument
__aicore__ inline void CastIntrinsicsImpl(__ubuf__ int32_t* dst, __ubuf__ int64_t* src, const RoundM...
                       ^
1 error generated.
gmake[6]: *** [CMakeFiles/device_obj.dir/build.make:118: CMakeFiles/device_obj.dir/home/ma-user/work/llama.cpp/build/auto_gen/ascendc_kernels/auto_gen_get_row_q4_0.cpp.o] Error 1
gmake[5]: *** [CMakeFiles/Makefile2:85: CMakeFiles/device_obj.dir/all] Error 2
gmake[4]: *** [Makefile:91: all] Error 2
gmake[3]: *** [ggml/src/ggml-cann/kernels/CMakeFiles/ascendc_kernels_device.dir/build.make:86: ggml/src/ggml-cann/kernels/ascendc_kernels_device-prefix/src/ascendc_kernels_device-stamp/ascendc_kernels_device-build] Error 2
gmake[2]: *** [CMakeFiles/Makefile2:1867: ggml/src/ggml-cann/kernels/CMakeFiles/ascendc_kernels_device.dir/all] Error 2
gmake[1]: *** [CMakeFiles/Makefile2:3323: examples/main/CMakeFiles/llama-cli.dir/rule] Error 2

[Qiliang]

Manually specify SOC_TYPE currently has problem, plz see this issue #10519 . You can build without specify SOC_TYPE, let auto-detect. Notice: Ascend910A and Ascend910B not support auto-detected now.

How can i build llama.cpp in Ascend910A ? auto-detect or manually specify ?
can you give me same examples?

[Qiliang]

We have verified with 910B3/910B4，but we have no 910A device currently. Can you paste all output after running npu-smi info ?

see my reply . up.

[leo-pony]

How can i build llama.cpp in Ascend910A ? auto-detect or manually specify ? can you give me same examples?

The PR fix cann't specify SOC_TYPE manually has merged. Now you can update code, and specify the SOC_TYPE.
for example:

cmake -B build -DGGML_CANN=on -DCMAKE_BUILD_TYPE=debug -DSOC_TYPE=Ascend910A
cmake --build build --config debug

[Qiliang]

How can i build llama.cpp in Ascend910A ? auto-detect or manually specify ? can you give me same examples?

The PR fix cann't specify SOC_TYPE manually has merged. Now you can update code, and specify the SOC_TYPE. for example:

cmake -B build -DGGML_CANN=on -DCMAKE_BUILD_TYPE=debug -DSOC_TYPE=Ascend910A cmake --build build --config debug

thanks ,

I update code, but the same error

git clone https://github.com/ggerganov/llama.cpp.git
cmake -B build -DGGML_CANN=on -DCMAKE_BUILD_TYPE=debug -DSOC_TYPE=Ascend910A
cmake --build build --config debug

usr/local/Ascend/ascend-toolkit/latest/tools/tikcpp/tikcfw/impl/dav_c100/kernel_operator_vec_vconv_impl.h:450:5: error: 
      no matching function for call to 'CastIntrinsicsImpl'

[leo-pony]

@Qiliang
I temporarily modify code that adapt to Ascend910A, and compiled successfully.
But I don't have Ascend910A device, could you on your environment? Notice: Only support no quantization mode, your can test on f16 model.
Code link: https://github.com/leo-pony/llama.cpp/tree/ascend910AAdapt

If failed inference, please paste the detail error information.

[leo-pony]

Notice: Only support no quantization mode, your can test on f16 model.

@Qiliang Only support no quantization mode, your can test on f16 model.