[CI][C++] test-conda-cpp-valgrind fails building vendored/xxhash.h

[raulcd]

Describe the bug, including details regarding any error messages, version, and platform.

The test-conda-cpp-valgrind job has been failing for 3 weeks with the following error:

[7/824] Building CXX object src/arrow/CMakeFiles/arrow_array.dir/array/array_dict.cc.o
FAILED: [code=1] src/arrow/CMakeFiles/arrow_array.dir/array/array_dict.cc.o 
/usr/local/bin/sccache /opt/conda/envs/arrow/bin/x86_64-conda-linux-gnu-c++ -DARROW_HAVE_RUNTIME_AVX2 -DARROW_HAVE_RUNTIME_BMI2 -DARROW_HAVE_RUNTIME_SSE4_2 -DARROW_HAVE_SSE4_2 -DARROW_VALGRIND -DARROW_WITH_TIMING_TESTS -I/build/cpp/src -I/arrow/cpp/src -I/arrow/cpp/src/generated -Wno-noexcept-type -Wno-self-move -fvisibility-inlines-hidden -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /opt/conda/envs/arrow/include -fdiagnostics-color=always  -Wall -fno-semantic-interposition -msse4.2  -g -O0 -ggdb -g1 -Og -std=c++17 -fPIC -MD -MT src/arrow/CMakeFiles/arrow_array.dir/array/array_dict.cc.o -MF src/arrow/CMakeFiles/arrow_array.dir/array/array_dict.cc.o.d -o src/arrow/CMakeFiles/arrow_array.dir/array/array_dict.cc.o -c /arrow/cpp/src/arrow/array/array_dict.cc
In file included from /arrow/cpp/src/arrow/vendored/xxhash.h:18,
                 from /arrow/cpp/src/arrow/util/hashing.h:50,
                 from /arrow/cpp/src/arrow/array/dict_internal.h:37,
                 from /arrow/cpp/src/arrow/array/array_dict.cc:29:
In function 'void XXH3_hashLong_internal_loop(xxh_u64*, const xxh_u8*, size_t, const xxh_u8*, size_t, XXH3_f_accumulate, XXH3_f_scrambleAcc)',
    inlined from 'XXH64_hash_t XXH3_hashLong_64b_internal(const void*, size_t, const void*, size_t, XXH3_f_accumulate, XXH3_f_scrambleAcc)' at /arrow/cpp/src/arrow/vendored/xxhash/xxhash.h:5607:32,
    inlined from 'XXH64_hash_t XXH3_hashLong_64b_withSecret(const void*, size_t, XXH64_hash_t, const xxh_u8*, size_t)' at /arrow/cpp/src/arrow/vendored/xxhash/xxhash.h:5629:38:
/arrow/cpp/src/arrow/vendored/xxhash/xxhash.h:4822:1: error: inlining failed in call to 'always_inline' 'void XXH3_scrambleAcc_sse2(void*, const void*)': function not considered for inlining
 4822 | XXH3_scrambleAcc_sse2(void* XXH_RESTRICT acc, const void* XXH_RESTRICT secret)
      | ^~~~~~~~~~~~~~~~~~~~~
/arrow/cpp/src/arrow/vendored/xxhash/xxhash.h:5545:19: note: called from here
 5545 |         f_scramble(acc, secret + secretSize - XXH_STRIPE_LEN);
      |         ~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/arrow/cpp/src/arrow/vendored/xxhash/xxhash.h:4508:1: error: inlining failed in call to 'always_inline' 'void XXH3_accumulate_sse2(xxh_u64*, const xxh_u8*, const xxh_u8*, size_t)': function not considered for inlining
 4508 | XXH3_accumulate_##name(xxh_u64* XXH_RESTRICT acc,           \
      | ^~~~~~~~~~~~~~~~
/arrow/cpp/src/arrow/vendored/xxhash/xxhash.h:4819:34: note: in expansion of macro 'XXH3_ACCUMULATE_TEMPLATE'
 4819 | XXH_FORCE_INLINE XXH_TARGET_SSE2 XXH3_ACCUMULATE_TEMPLATE(sse2)
      |                                  ^~~~~~~~~~~~~~~~~~~~~~~~
/arrow/cpp/src/arrow/vendored/xxhash/xxhash.h:5544:14: note: called from here
 5544 |         f_acc(acc, input + n*block_len, secret, nbStripesPerBlock);
      |         ~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/arrow/cpp/src/arrow/vendored/xxhash/xxhash.h:4508:1: error: inlining failed in call to 'always_inline' 'void XXH3_accumulate_sse2(xxh_u64*, const xxh_u8*, const xxh_u8*, size_t)': function not considered for inlining
 4508 | XXH3_accumulate_##name(xxh_u64* XXH_RESTRICT acc,           \
      | ^~~~~~~~~~~~~~~~
/arrow/cpp/src/arrow/vendored/xxhash/xxhash.h:4819:34: note: in expansion of macro 'XXH3_ACCUMULATE_TEMPLATE'
 4819 | XXH_FORCE_INLINE XXH_TARGET_SSE2 XXH3_ACCUMULATE_TEMPLATE(sse2)
      |                                  ^~~~~~~~~~~~~~~~~~~~~~~~
/arrow/cpp/src/arrow/vendored/xxhash/xxhash.h:5552:14: note: called from here
 5552 |         f_acc(acc, input + nb_blocks*block_len, secret, nbStripes);
      |         ~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[8/824] Building CXX object src/arrow/CMakeFiles/arrow_array.dir/array/array_run_end.cc.o

The first failure comes after the following changes but I can't seem what could cause the failure:
db1a8a9...0ae71f8

The last success was:
https://github.com/ursacomputing/crossbow/actions/runs/16895764772/job/47865118610

Component(s)

Continuous Integration, C++

[pitrou]

Probably a compiler or compiler option change? We should first bump the xxhash version, that might be sufficient to fix the issue.

[raulcd]

There seems to be a 0.8.3 version available:
https://github.com/Cyan4973/xxHash/releases/tag/v0.8.3

We are using 0.8.2.

I can take a look at updating the version

[raulcd]

As discussed on the xxhash PR the bump did not solve the issue, the failure with the bump is the same one:

AILED: [code=1] src/arrow/CMakeFiles/arrow_array.dir/array/builder_dict.cc.o 
/usr/local/bin/sccache /opt/conda/envs/arrow/bin/x86_64-conda-linux-gnu-c++ -DARROW_HAVE_RUNTIME_AVX2 -DARROW_HAVE_RUNTIME_BMI2 -DARROW_HAVE_RUNTIME_SSE4_2 -DARROW_HAVE_SSE4_2 -DARROW_VALGRIND -DARROW_WITH_TIMING_TESTS -I/build/cpp/src -I/arrow/cpp/src -I/arrow/cpp/src/generated -Wno-noexcept-type -Wno-self-move -fvisibility-inlines-hidden -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /opt/conda/envs/arrow/include -fdiagnostics-color=always  -Wall -fno-semantic-interposition -msse4.2  -g -O0 -ggdb -g1 -Og -std=c++17 -fPIC -MD -MT src/arrow/CMakeFiles/arrow_array.dir/array/builder_dict.cc.o -MF src/arrow/CMakeFiles/arrow_array.dir/array/builder_dict.cc.o.d -o src/arrow/CMakeFiles/arrow_array.dir/array/builder_dict.cc.o -c /arrow/cpp/src/arrow/array/builder_dict.cc
In file included from /arrow/cpp/src/arrow/vendored/xxhash.h:18,
                 from /arrow/cpp/src/arrow/util/hashing.h:50,
                 from /arrow/cpp/src/arrow/array/dict_internal.h:37,
                 from /arrow/cpp/src/arrow/array/builder_dict.cc:23:
In function 'void XXH3_hashLong_internal_loop(xxh_u64*, const xxh_u8*, size_t, const xxh_u8*, size_t, XXH3_f_accumulate, XXH3_f_scrambleAcc)',
    inlined from 'XXH64_hash_t XXH3_hashLong_64b_internal(const void*, size_t, const void*, size_t, XXH3_f_accumulate, XXH3_f_scrambleAcc)' at /arrow/cpp/src/arrow/vendored/xxhash/xxhash.h:6315:32,
    inlined from 'XXH64_hash_t XXH3_hashLong_64b_withSecret(const void*, size_t, XXH64_hash_t, const xxh_u8*, size_t)' at /arrow/cpp/src/arrow/vendored/xxhash/xxhash.h:6335:38:
/arrow/cpp/src/arrow/vendored/xxhash/xxhash.h:5241:1: error: inlining failed in call to 'always_inline' 'void XXH3_scrambleAcc_sse2(void*, const void*)': function not considered for inlining
 5241 | XXH3_scrambleAcc_sse2(void* XXH_RESTRICT acc, const void* XXH_RESTRICT secret)
      | ^~~~~~~~~~~~~~~~~~~~~
/arrow/cpp/src/arrow/vendored/xxhash/xxhash.h:6244:19: note: called from here
 6244 |         f_scramble(acc, secret + secretSize - XXH_STRIPE_LEN);
      |         ~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/arrow/cpp/src/arrow/vendored/xxhash/xxhash.h:4927:1: error: inlining failed in call to 'always_inline' 'void XXH3_accumulate_sse2(xxh_u64*, const xxh_u8*, const xxh_u8*, size_t)': function not considered for inlining
 4927 | XXH3_accumulate_##name(xxh_u64* XXH_RESTRICT acc,           \
      | ^~~~~~~~~~~~~~~~
/arrow/cpp/src/arrow/vendored/xxhash/xxhash.h:5238:34: note: in expansion of macro 'XXH3_ACCUMULATE_TEMPLATE'
 5238 | XXH_FORCE_INLINE XXH_TARGET_SSE2 XXH3_ACCUMULATE_TEMPLATE(sse2)
      |                                  ^~~~~~~~~~~~~~~~~~~~~~~~
/arrow/cpp/src/arrow/vendored/xxhash/xxhash.h:6243:14: note: called from here
 6243 |         f_acc(acc, input + n*block_len, secret, nbStripesPerBlock);
      |         ~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/arrow/cpp/src/arrow/vendored/xxhash/xxhash.h:4927:1: error: inlining failed in call to 'always_inline' 'void XXH3_accumulate_sse2(xxh_u64*, const xxh_u8*, const xxh_u8*, size_t)': function not considered for inlining
 4927 | XXH3_accumulate_##name(xxh_u64* XXH_RESTRICT acc,           \
      | ^~~~~~~~~~~~~~~~
/arrow/cpp/src/arrow/vendored/xxhash/xxhash.h:5238:34: note: in expansion of macro 'XXH3_ACCUMULATE_TEMPLATE'
 5238 | XXH_FORCE_INLINE XXH_TARGET_SSE2 XXH3_ACCUMULATE_TEMPLATE(sse2)
      |                                  ^~~~~~~~~~~~~~~~~~~~~~~~
/arrow/cpp/src/arrow/vendored/xxhash/xxhash.h:6251:14: note: called from here
 6251 |         f_acc(acc, input + nb_blocks*block_len, secret, nbStripes);
      |         ~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[19/824] Building CXX object src/arrow/CMakeFiles/arrow_array.dir/array/array_dict.cc.o
FAILED: [code=1] src/arrow/CMakeFiles/arrow_array.dir/array/array_dict.cc.o 
/usr/local/bin/sccache /opt/conda/envs/arrow/bin/x86_64-conda-linux-gnu-c++ -DARROW_HAVE_RUNTIME_AVX2 -DARROW_HAVE_RUNTIME_BMI2 -DARROW_HAVE_RUNTIME_SSE4_2 -DARROW_HAVE_SSE4_2 -DARROW_VALGRIND -DARROW_WITH_TIMING_TESTS -I/build/cpp/src -I/arrow/cpp/src -I/arrow/cpp/src/generated -Wno-noexcept-type -Wno-self-move -fvisibility-inlines-hidden -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /opt/conda/envs/arrow/include -fdiagnostics-color=always  -Wall -fno-semantic-interposition -msse4.2  -g -O0 -ggdb -g1 -Og -std=c++17 -fPIC -MD -MT src/arrow/CMakeFiles/arrow_array.dir/array/array_dict.cc.o -MF src/arrow/CMakeFiles/arrow_array.dir/array/array_dict.cc.o.d -o src/arrow/CMakeFiles/arrow_array.dir/array/array_dict.cc.o -c /arrow/cpp/src/arrow/array/array_dict.cc
In file included from /arrow/cpp/src/arrow/vendored/xxhash.h:18,
                 from /arrow/cpp/src/arrow/util/hashing.h:50,
                 from /arrow/cpp/src/arrow/array/dict_internal.h:37,
                 from /arrow/cpp/src/arrow/array/array_dict.cc:29:
In function 'void XXH3_hashLong_internal_loop(xxh_u64*, const xxh_u8*, size_t, const xxh_u8*, size_t, XXH3_f_accumulate, XXH3_f_scrambleAcc)',
    inlined from 'XXH64_hash_t XXH3_hashLong_64b_internal(const void*, size_t, const void*, size_t, XXH3_f_accumulate, XXH3_f_scrambleAcc)' at /arrow/cpp/src/arrow/vendored/xxhash/xxhash.h:6315:32,
    inlined from 'XXH64_hash_t XXH3_hashLong_64b_withSecret(const void*, size_t, XXH64_hash_t, const xxh_u8*, size_t)' at /arrow/cpp/src/arrow/vendored/xxhash/xxhash.h:6335:38:
/arrow/cpp/src/arrow/vendored/xxhash/xxhash.h:5241:1: error: inlining failed in call to 'always_inline' 'void XXH3_scrambleAcc_sse2(void*, const void*)': function not considered for inlining
 5241 | XXH3_scrambleAcc_sse2(void* XXH_RESTRICT acc, const void* XXH_RESTRICT secret)
      | ^~~~~~~~~~~~~~~~~~~~~
/arrow/cpp/src/arrow/vendored/xxhash/xxhash.h:6244:19: note: called from here
 6244 |         f_scramble(acc, secret + secretSize - XXH_STRIPE_LEN);
      |         ~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/arrow/cpp/src/arrow/vendored/xxhash/xxhash.h:4927:1: error: inlining failed in call to 'always_inline' 'void XXH3_accumulate_sse2(xxh_u64*, const xxh_u8*, const xxh_u8*, size_t)': function not considered for inlining
 4927 | XXH3_accumulate_##name(xxh_u64* XXH_RESTRICT acc,           \
      | ^~~~~~~~~~~~~~~~
/arrow/cpp/src/arrow/vendored/xxhash/xxhash.h:5238:34: note: in expansion of macro 'XXH3_ACCUMULATE_TEMPLATE'
 5238 | XXH_FORCE_INLINE XXH_TARGET_SSE2 XXH3_ACCUMULATE_TEMPLATE(sse2)
      |                                  ^~~~~~~~~~~~~~~~~~~~~~~~
/arrow/cpp/src/arrow/vendored/xxhash/xxhash.h:6243:14: note: called from here
 6243 |         f_acc(acc, input + n*block_len, secret, nbStripesPerBlock);
      |         ~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/arrow/cpp/src/arrow/vendored/xxhash/xxhash.h:4927:1: error: inlining failed in call to 'always_inline' 'void XXH3_accumulate_sse2(xxh_u64*, const xxh_u8*, const xxh_u8*, size_t)': function not considered for inlining
 4927 | XXH3_accumulate_##name(xxh_u64* XXH_RESTRICT acc,           \
      | ^~~~~~~~~~~~~~~~
/arrow/cpp/src/arrow/vendored/xxhash/xxhash.h:5238:34: note: in expansion of macro 'XXH3_ACCUMULATE_TEMPLATE'
 5238 | XXH_FORCE_INLINE XXH_TARGET_SSE2 XXH3_ACCUMULATE_TEMPLATE(sse2)
      |                                  ^~~~~~~~~~~~~~~~~~~~~~~~
/arrow/cpp/src/arrow/vendored/xxhash/xxhash.h:6251:14: note: called from here
 6251 |         f_acc(acc, input + nb_blocks*block_len, secret, nbStripes);
      |         ~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[20/824] Building CXX object src/arrow/CMakeFiles/arrow_array.dir/array/concatenate.cc.o
[21/824] Building CXX object src/arrow/CMakeFiles/arrow_array.dir/array/data.cc.o
[22/824] Building CXX object src/arrow/CMakeFiles/arrow_array.dir/array/diff.cc.o
ninja: build stopped: subcommand failed.

[raulcd]

This was fixed as part of:

• GH-47483: [C++] Bump vendored xxhash to 0.8.3 #47476