How to setup the hexagon sdk build environment

[chraac]

Here we can have discussions regarding how to setup the hexagon sdk build environment.

Steps

1. Create qualcomm account
2. Following the official instuction here to acquire the hexagon sdk

[cm4ker]

Hi, @chraac!

I faced with issue then start build.

Here is a folder with sdk

I glad to provide additional info if need.

[chraac]

Interesting, from the sdk log, looks we should call setup_sdk_env.cmd first, trying to build with command line, but found amother error log, gonna figure out after finish my current work.......

[finneyyan]

I try to setup it on Windows, and have same question.
setup_sdk_env.cmd OK:

> C:\Qualcomm\Hexagon_SDK\5.5.5.0\setup_sdk_env.cmd
Setting up the Hexagon SDK environment locally...


Checking if all the SDK dependencies are available...


Setting up the QAIC binary for the platform
make: Entering directory `C:/Qualcomm/Hexagon_SDK/5.5.5.0/ipc/fastrpc/qaic'
make: Leaving directory `C:/Qualcomm/Hexagon_SDK/5.5.5.0/ipc/fastrpc/qaic'
make: Entering directory `C:/Qualcomm/Hexagon_SDK/5.5.5.0/ipc/fastrpc/qaic'
Qaic's Another IDL Compiler: version 01.00.47.
Copyright (c) 2012-2023 QUALCOMM Inc. All rights reserved.
make: Leaving directory `C:/Qualcomm/Hexagon_SDK/5.5.5.0/ipc/fastrpc/qaic'

Setup Done

compile Failed:

> cmake --preset  arm64-windows-llvm-debug -DLLAMA_CURL=OFF -DGGML_OPENMP=OFF -DBUILD_SHARED_LIBS=OFF -DGGML_QNN=ON -DGGML_QNN_SDK_PATH="C:/Program Files/v2.35.0.250530/qairt/2.35.0.250530" -DGGML_QNN_ENABLE_CPU_BACKEND=OFF -DGGML_HEXAGON_NPU_ONLY=ON -DHEXAGON_SDK_ROOT="C:/Qualcomm/Hexagon_SDK/5.5.5.0"
Preset CMake variables:

  CMAKE_BUILD_TYPE="Debug"
  CMAKE_EXPORT_COMPILE_COMMANDS="ON"
  CMAKE_INSTALL_RPATH="$ORIGIN;$ORIGIN/.."
  CMAKE_TOOLCHAIN_FILE="C:/Users/user1/llama.cpp-qnn/cmake/arm64-windows-llvm.cmake"

-- The C compiler identification is Clang 20.1.0 with GNU-like command-line
-- The CXX compiler identification is Clang 20.1.0 with GNU-like command-line
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/LLVM/bin/clang.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/LLVM/bin/clang++.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: C:/Program Files/Git/cmd/git.exe (found version "2.47.0.windows.1")
-- Setting GGML_NATIVE_DEFAULT to OFF
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - no
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - not found
-- Found Threads: TRUE
-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: arm64
-- GGML_SYSTEM_ARCH: ARM
-- Including CPU backend
-- ARM detected
-- Performing Test GGML_COMPILER_SUPPORTS_FP16_FORMAT_I3E
-- Performing Test GGML_COMPILER_SUPPORTS_FP16_FORMAT_I3E - Failed
-- ARM feature FMA enabled
-- Adding CPU backend variant ggml-cpu:
-- Using QNN backend
Building for Linux or Windows
CMAKE_BUILD_TYPE: Debug
CMAKE_CXX_FLAGS_DEBUG: -O0 -g -Xclang -gcodeview -D_DEBUG -D_DLL -D_MT -Xclang --dependent-lib=msvcrtd
CMAKE_CXX_FLAGS_RELEASE: -O3 -DNDEBUG -D_DLL -D_MT -Xclang --dependent-lib=msvcrt
QNN_SDK_PATH: C:/Program Files/v2.35.0.250530/qairt/2.35.0.250530
GGML_QNN: ON
GGML_QNN_ENABLE_HEXAGON_BACKEND: ON
GGML_HEXAGON_NPU_ONLY: ON
GGML_HEXAGON_ENABLE_QUANTIZED_TENSORS: OFF
GGML_HEXAGON_ENABLE_PERFORMANCE_TRACKING: OFF
HEXAGON_SDK_ROOT: C:/Qualcomm/Hexagon_SDK/5.5.5.0
GGML_HEXAGON_NPU_ONLY is enabled
GGML_QNN_ENABLE_HEXAGON_BACKEND is enabled
-- The ASM compiler identification is Clang with GNU-like command-line
-- Found assembler: C:/Program Files/LLVM/bin/clang.exe
CMake Deprecation Warning at ggml/src/ggml-qnn/npu/CMakeLists.txt:2 (cmake_policy):
  The OLD behavior for policy CMP0115 will be removed from a future version
  of CMake.

  The cmake-policies(7) manual explains that the OLD behaviors of all
  policies are deprecated and that a policy should be set to OLD only under
  specific short-term circumstances.  Projects should be ported to the NEW
  behavior and not rely on setting a policy to OLD.


HEXAGON_SDK_ROOT: C:/Qualcomm/Hexagon_SDK/5.5.5.0
CMake Error at C:/Qualcomm/Hexagon_SDK/5.5.5.0/build/cmake/hexagon_fun.cmake:18 (include):
  include could not find requested file:

    C:/Qualcomm/Hexagon_SDK/5.5.5.0/addons/wos/build/cmake/windows_fun.cmake
Call Stack (most recent call first):
  ggml/src/ggml-qnn/npu/CMakeLists.txt:14 (include)


GGML_HEXAGON_ENABLE_PERFORMANCE_TRACKING is disabled
CMake Error at C:/Qualcomm/Hexagon_SDK/5.5.5.0/build/cmake/hexagon_fun_internal.cmake:374 (link_custom_library_wos):
  Unknown CMake command "link_custom_library_wos".
Call Stack (most recent call first):
  C:/Qualcomm/Hexagon_SDK/5.5.5.0/build/cmake/hexagon_fun.cmake:457 (link_custom_library_source)
  ggml/src/ggml-qnn/npu/CMakeLists.txt:151 (link_custom_library)


-- Configuring incomplete, errors occurred!

I'm willing to try more if necessary.

[chraac]

nice nice, I've also experienced the same error when trying, to compile in windows
regarding the error here

Unknown CMake command "link_custom_library_wos".

didn't have time to look into it so far, but if we luckly enough, it'll be fixed in the newer hexagon-sdk......

[finneyyan]

@chraac I try to setup it in Linux.
After acquire hexagon_sdk, I do something:

1. acquire android-ndk-r25c and cmake-3.28.3-linux-x86_64 and put them in /Hexagon_SDK/tools; acquire ninja and put it in /Hexagon_SDK/tools/cmake-3.28.3-linux-x86_64/bin
2. runbash setup_sdk_env.source & source setup_sdk_env.source (would be errors if not do step1)
3. cd llama.cpp and run compile command (enable GGML_QNN_ENABLE_HEXAGON_BACKEND), many errors; then do following try to resolve:
4. run sudo ln -s /usr/lib/x86_64-linux-gnu/libtinfo.so.6 /usr/lib/x86_64-linux-gnu/libtinfo.so.5 (require libtinfo.so.5 but I cannot get it by apt install, but I have libtinfo.so.6)
5. run export QNN_SDK_ROOT=/root/qairt/2.35.0.250530
   However, I still cannot compile it success.

Current error:
/root/llama.cpp-qnn/ggml/src/ggml-qnn/npu/device/vtcm_mem.hpp:18:21: error: use of undeclared identifier 'HAP_request_VTCM'
   18 |         _vtcm_mem = HAP_request_VTCM((unsigned int) size, single_page ? 1 : 0);
      |                     ^
/root/llama.cpp-qnn/ggml/src/ggml-qnn/npu/device/vtcm_mem.hpp:29:21: error: use of undeclared identifier 'HAP_request_async_VTCM'
   29 |         _vtcm_mem = HAP_request_async_VTCM((unsigned int) size, single_page ? 1 : 0, (unsigned int) timeout_us);
      |                     ^
/root/llama.cpp-qnn/ggml/src/ggml-qnn/npu/device/vtcm_mem.hpp:41:24: error: use of undeclared identifier 'HAP_release_VTCM'
   41 |             auto ret = HAP_release_VTCM(_vtcm_mem);
      |                        ^
In file included from /root/llama.cpp-qnn/ggml/src/ggml-qnn/npu/device/op_flash_attn.cpp:2:
In file included from /root/llama.cpp-qnn/ggml/src/ggml-qnn/npu/device/op_flash_attn.hpp:3:
In file included from /root/llama.cpp-qnn/ggml/src/ggml-qnn/npu/device/op_types.hpp:12:
/root/llama.cpp-qnn/ggml/src/ggml-qnn/npu/device/thread_pool.hpp:241:53: error: implicit instantiation of undefined template 'std::array<std::unique_ptr<hexagon::qurt_thread<32768>>, 3>'
  241 |     std::array<qurt_thread_ptr, kMaxSubThreadCount> _threads;
      |                                                     ^
/root/llama.cpp-qnn/ggml/src/ggml-qnn/npu/device/op_types.hpp:42:5: note: in instantiation of template class 'hexagon::thread_pool<4>' requested here
   42 |     default_thread_pool::thread_params * const thread_params;
      |     ^
/root/tools/Hexagon-SDK/tools/HEXAGON_Tools/8.8.06/Tools/bin/../target/hexagon/include/c++/v1/__fwd/array.h:22:29: note: template is declared here
   22 | struct _LIBCPP_TEMPLATE_VIS array;
      |                             ^
4 errors generated.

What should I do then?

[chraac]

Looks you're running into the same problem as this issue #41, please use non-root adb to run the backend

[finneyyan]

I'm sorry I didn't check if there are any similar questions before asking. I will check it. Thanks for your reply!

[chraac]

I'm sorry I didn't check if there are any similar questions before asking. I will check it. Thanks for your reply!

No worries, honestly, I'm happy to help someone that willing to test this new backend, but kinda busy on the daytime, so you may expect a delay in response tho....

[finneyyan]

I found the reason. What made me speechless was that I moved llama-cli to another location and did not put it together with other libraries. When I put them in the same path, it worked properly.
Thanks again.

[chraac]

add a section to the wiki build instructions here:
https://github.com/chraac/llama.cpp/wiki/How-to-Build