Build on NVIDIA Grace Hopper

[sraskar]

Hello,
I am trying to build on Nvidia Grace Hopper system. I am using GCC 11.2.0 and cuda 12.3.0

> cmake -B build -DGGML_CUDA=ON
-- OpenMP found
-- Using llamafile
-- CUDA found
-- Using CUDA architectures: 52;61;70;75
-- CUDA host compiler is GNU 11.2.0

-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: aarch64
-- ARM detected
-- Configuring done (1.5s)
-- Generating done (7.1s)
-- Build files have been written to: ~/llama.cpp/build

I get following errors:

> cmake --build build --config Release
~/llama.cpp/ggml/src/ggml-quants.c:10449:37: note: in expansion of macro ‘ggml_vld1q_u8_x2’
     const ggml_uint8x16x2_t mask1 = ggml_vld1q_u8_x2(k_mask1);
                                     ^~~~~~~~~~~~~~~~
~/llama.cpp/ggml/src/ggml-quants.c:10478:17: error: incompatible types when assigning to type ‘int8x16x4_t {aka struct int8x16x4_t}’ from type ‘int’
             q8b = ggml_vld1q_s8_x4(q8); q8 += 64;
                 ^
~/llama.cpp/ggml/src/ggml-quants.c: In function ‘ggml_vec_dot_iq1_s_q8_K’:
~/llama.cpp/ggml/src/ggml-quants.c:11014:17: error: incompatible types when assigning to type ‘int8x16x4_t {aka struct int8x16x4_t}’ from type ‘int’
             q8b = ggml_vld1q_s8_x4(q8); q8 += 64;
                 ^
~/llama.cpp/ggml/src/ggml-quants.c: In function ‘ggml_vec_dot_iq1_m_q8_K’:
~/llama.cpp/ggml/src/ggml-quants.c:11362:17: error: incompatible types when assigning to type ‘int8x16x4_t {aka struct int8x16x4_t}’ from type ‘int’
             q8b = ggml_vld1q_s8_x4(q8); q8 += 64;
                 ^
~/llama.cpp/ggml/src/ggml-quants.c: In function ‘ggml_vec_dot_iq4_xs_q8_K’:
~/llama.cpp/ggml/src/ggml-quants.c:11831:20: error: incompatible types when assigning to type ‘uint8x16x2_t {aka struct uint8x16x2_t}’ from type ‘int’
             q4bits = ggml_vld1q_u8_x2(q4); q4 += 32;
                    ^
/~/llama.cpp/ggml/src/ggml-quants.c:11832:20: error: incompatible types when assigning to type ‘int8x16x4_t {aka struct int8x16x4_t}’ from type ‘int’
             q8b    = ggml_vld1q_s8_x4(q8); q8 += 64;
                    ^

Any recommendations on getting this to work on grace hopper system? Thanks.

[eimanaln]

Hello!
I'm facing the same issue too.
Did you manage to resolve it? If so, how?
I would appreciate the help.
Thank you!

[eimanaln]

I managed to resolve the issue by upgrading to GCC 13.3 (support for arm neon intrinsics starts from GCC 9+).