cmake hang

[Tangruilin]

command cmake -S . -B build -G Ninja -DCMAKE_INSTALL_PREFIX=./install -DICEBERG_BUILD_STATIC=ON -DICEBERG_BUILD_SHARED=O -DENABLE_ROARING_MICROBENCHMARKS=ON -DHAVE_PTHREAD_AFFINITY=OFF may hang

my envirment is Mac m2 pro

[wgtmac]

You might need to set up http(s) proxy. It may hang when trying to download vendored dependencies.

[Tangruilin]

You might need to set up http(s) proxy. It may hang when trying to download vendored dependencies.

the google-benchmark version that iceberg-cpp dependecy not support PTHREAD_AFFINITY, maybe it need to be marked in README.txt

Additionally, I'd like to ask if there are plans for Iceberg to support development on Mac.

[Tangruilin]

You might need to set up http(s) proxy. It may hang when trying to download vendored dependencies.

the `google-benchmark` version that iceberg-cpp dependecy not support PTHREAD_AFFINITY, maybe it need to be marked in `README.txt`

Additionally, I'd like to ask if there are plans for Iceberg to support development on Mac.

use flag -DCXXFEATURECHECK_DEBUG=on can look the error

[zhjwpku]

You might need to set up http(s) proxy. It may hang when trying to download vendored dependencies.

the `google-benchmark` version that iceberg-cpp dependecy not support PTHREAD_AFFINITY, maybe it need to be marked in `README.txt`

Additionally, I'd like to ask if there are plans for Iceberg to support development on Mac.

I think most of the contributors use Mac to develop iceberg-cpp. Could you provide a bit more detail about the issue you're encountering?

[zhjwpku]

I download CRoaring and build with the following commands:

cmake -S . -B build -DENABLE_ROARING_MICROBENCHMARKS=ON -DHAVE_PTHREAD_AFFINITY=OFF
cmake --build build

I got the same error:

/Users/zjw/zjw/CRoaring/build/_deps/benchmark-src/src/sysinfo.cc:618:3: error: unknown type name 'cpu_set_t'
  618 |   cpu_set_t previous_affinity;
      |   ^
/Users/zjw/zjw/CRoaring/build/_deps/benchmark-src/src/sysinfo.cc:589:5: error: unknown type name 'cpu_set_t'
  589 |     cpu_set_t affinity;
      |     ^
/Users/zjw/zjw/CRoaring/build/_deps/benchmark-src/src/sysinfo.cc:594:25: error: use of undeclared identifier 'CPU_SETSIZE'
  594 |     for (int i = 0; i < CPU_SETSIZE; ++i)
      |                         ^
/Users/zjw/zjw/CRoaring/build/_deps/benchmark-src/src/sysinfo.cc:618:13: warning: private field 'previous_affinity' is not used [-Wunused-private-field]
  618 |   cpu_set_t previous_affinity;
      |             ^

It seems pointless to run these tests and benchmarks within iceberg-cpp. I disable them in #241, can you give it a try?

[Tangruilin]

I download CRoaring and build with the following commands:

cmake -S . -B build -DENABLE_ROARING_MICROBENCHMARKS=ON -DHAVE_PTHREAD_AFFINITY=OFF
cmake --build build

I got the same error:

/Users/zjw/zjw/CRoaring/build/_deps/benchmark-src/src/sysinfo.cc:618:3: error: unknown type name 'cpu_set_t'
  618 |   cpu_set_t previous_affinity;
      |   ^
/Users/zjw/zjw/CRoaring/build/_deps/benchmark-src/src/sysinfo.cc:589:5: error: unknown type name 'cpu_set_t'
  589 |     cpu_set_t affinity;
      |     ^
/Users/zjw/zjw/CRoaring/build/_deps/benchmark-src/src/sysinfo.cc:594:25: error: use of undeclared identifier 'CPU_SETSIZE'
  594 |     for (int i = 0; i < CPU_SETSIZE; ++i)
      |                         ^
/Users/zjw/zjw/CRoaring/build/_deps/benchmark-src/src/sysinfo.cc:618:13: warning: private field 'previous_affinity' is not used [-Wunused-private-field]
  618 |   cpu_set_t previous_affinity;
      |             ^

It seems pointless to run these tests and benchmarks within iceberg-cpp. I disable them in #241, can you give it a try?

cpu_set_t is not support on MacOS, it is only supported on Linux.
Then I found the flag HAVE_PTHREAD_AFFINITY=off is not useful, you can manually comment out the CPU affinity settings to solve this problem.
like below:

# pthread affinity, if available
if(HAVE_PTHREAD_AFFINITY)
  #target_compile_definitions(benchmark PRIVATE -DBENCHMARK_HAS_PTHREAD_AFFINITY)
endif()

[zhjwpku]

It seems pointless to run these tests and benchmarks within iceberg-cpp. I disable them in #241, can you give it a try?

cpu_set_t is not support on MacOS, it is only supported on Linux. Then I found the flag HAVE_PTHREAD_AFFINITY=off is not useful, you can manually comment out the CPU affinity settings to solve this problem. like below:

pthread affinity, if available

if(HAVE_PTHREAD_AFFINITY)
#target_compile_definitions(benchmark PRIVATE -DBENCHMARK_HAS_PTHREAD_AFFINITY)
endif()

I digged into this a little bit, when you set HAVE_PTHREAD_AFFINITY either to on or off, google benchmark will define HAVE_PTHREAD_AFFINITY, see [1]. Then I tried cmake -S . -B build -DENABLE_ROARING_MICROBENCHMARKS=ON to build CRoaring, it passed. You can observe some message like:

-- Compiling and running to test HAVE_PTHREAD_AFFINITY
-- Performing Test HAVE_PTHREAD_AFFINITY -- failed to compile

I think HAVE_PTHREAD_AFFINITY is supposed to be automatically detected and set by the google benchmark building system, may I ask why you specify it manually?

[1] https://github.com/google/benchmark/blob/main/cmake/CXXFeatureCheck.cmake#L26-L29

[Tangruilin]

It seems pointless to run these tests and benchmarks within iceberg-cpp. I disable them in #241, can you give it a try?

cpu_set_t is not support on MacOS, it is only supported on Linux. Then I found the flag HAVE_PTHREAD_AFFINITY=off is not useful, you can manually comment out the CPU affinity settings to solve this problem. like below:

pthread affinity, if available

if(HAVE_PTHREAD_AFFINITY)
#target_compile_definitions(benchmark PRIVATE -DBENCHMARK_HAS_PTHREAD_AFFINITY)
endif()

I digged into this a little bit, when you set HAVE_PTHREAD_AFFINITY either to on or off, google benchmark will define HAVE_PTHREAD_AFFINITY, see [1]. Then I tried cmake -S . -B build -DENABLE_ROARING_MICROBENCHMARKS=ON to build CRoaring, it passed. You can observe some message like:

-- Compiling and running to test HAVE_PTHREAD_AFFINITY
-- Performing Test HAVE_PTHREAD_AFFINITY -- failed to compile

I think HAVE_PTHREAD_AFFINITY is supposed to be automatically detected and set by the google benchmark building system, may I ask why you specify it manually?

[1] https://github.com/google/benchmark/blob/main/cmake/CXXFeatureCheck.cmake#L26-L29

..., MacOS do not have cpu_set_t , so benchmark's PTHREAD_AFFINITY can not open on Mac

[zhjwpku]

It seems pointless to run these tests and benchmarks within iceberg-cpp. I disable them in #241, can you give it a try?

cpu_set_t is not support on MacOS, it is only supported on Linux. Then I found the flag HAVE_PTHREAD_AFFINITY=off is not useful, you can manually comment out the CPU affinity settings to solve this problem. like below:

pthread affinity, if available

if(HAVE_PTHREAD_AFFINITY)
#target_compile_definitions(benchmark PRIVATE -DBENCHMARK_HAS_PTHREAD_AFFINITY)
endif()

I digged into this a little bit, when you set HAVE_PTHREAD_AFFINITY either to on or off, google benchmark will define HAVE_PTHREAD_AFFINITY, see [1]. Then I tried cmake -S . -B build -DENABLE_ROARING_MICROBENCHMARKS=ON to build CRoaring, it passed. You can observe some message like:

-- Compiling and running to test HAVE_PTHREAD_AFFINITY
-- Performing Test HAVE_PTHREAD_AFFINITY -- failed to compile

I think HAVE_PTHREAD_AFFINITY is supposed to be automatically detected and set by the google benchmark building system, may I ask why you specify it manually?
[1] https://github.com/google/benchmark/blob/main/cmake/CXXFeatureCheck.cmake#L26-L29

..., MacOS do not have cpu_set_t , so benchmark's PTHREAD_AFFINITY can not open on Mac

After checking your reply again, I confirm that the error triggered by -DCXXFEATURECHECK_DEBUG=on is not an actual issue, it's intentionally used by the build system to probe OS capabilities during feature detection, I'll close this issue if no further comments.

[Tangruilin]

It seems pointless to run these tests and benchmarks within iceberg-cpp. I disable them in #241, can you give it a try?

cpu_set_t is not support on MacOS, it is only supported on Linux. Then I found the flag HAVE_PTHREAD_AFFINITY=off is not useful, you can manually comment out the CPU affinity settings to solve this problem. like below:

pthread affinity, if available

if(HAVE_PTHREAD_AFFINITY)
#target_compile_definitions(benchmark PRIVATE -DBENCHMARK_HAS_PTHREAD_AFFINITY)
endif()

I digged into this a little bit, when you set HAVE_PTHREAD_AFFINITY either to on or off, google benchmark will define HAVE_PTHREAD_AFFINITY, see [1]. Then I tried cmake -S . -B build -DENABLE_ROARING_MICROBENCHMARKS=ON to build CRoaring, it passed. You can observe some message like:

-- Compiling and running to test HAVE_PTHREAD_AFFINITY
-- Performing Test HAVE_PTHREAD_AFFINITY -- failed to compile

I think HAVE_PTHREAD_AFFINITY is supposed to be automatically detected and set by the google benchmark building system, may I ask why you specify it manually?
[1] https://github.com/google/benchmark/blob/main/cmake/CXXFeatureCheck.cmake#L26-L29

..., MacOS do not have cpu_set_t , so benchmark's PTHREAD_AFFINITY can not open on Mac

After checking your reply again, I confirm that the error triggered by -DCXXFEATURECHECK_DEBUG=on is not an actual issue, it's intentionally used by the build system to probe OS capabilities during feature detection, I'll close this issue if no further comments.

I think this is a Google Benchmark's bug where CPU affinity is enabled, and it doesn't seem to be compatible with macOS. However, since Iceberg depends on Google Benchmark, it should at least inform macOS developers how to handle this error, rather than casually assuming that CPU affinity detection is expected behavior and therefore not treating it as a bug

[Tangruilin]

It seems pointless to run these tests and benchmarks within iceberg-cpp. I disable them in #241, can you give it a try?

cpu_set_t is not support on MacOS, it is only supported on Linux. Then I found the flag HAVE_PTHREAD_AFFINITY=off is not useful, you can manually comment out the CPU affinity settings to solve this problem. like below:

pthread affinity, if available

if(HAVE_PTHREAD_AFFINITY)
#target_compile_definitions(benchmark PRIVATE -DBENCHMARK_HAS_PTHREAD_AFFINITY)
endif()

I digged into this a little bit, when you set HAVE_PTHREAD_AFFINITY either to on or off, google benchmark will define HAVE_PTHREAD_AFFINITY, see [1]. Then I tried cmake -S . -B build -DENABLE_ROARING_MICROBENCHMARKS=ON to build CRoaring, it passed. You can observe some message like:

-- Compiling and running to test HAVE_PTHREAD_AFFINITY
-- Performing Test HAVE_PTHREAD_AFFINITY -- failed to compile

I think HAVE_PTHREAD_AFFINITY is supposed to be automatically detected and set by the google benchmark building system, may I ask why you specify it manually?
[1] https://github.com/google/benchmark/blob/main/cmake/CXXFeatureCheck.cmake#L26-L29

..., MacOS do not have cpu_set_t , so benchmark's PTHREAD_AFFINITY can not open on Mac

After checking your reply again, I confirm that the error triggered by -DCXXFEATURECHECK_DEBUG=on is not an actual issue, it's intentionally used by the build system to probe OS capabilities during feature detection, I'll close this issue if no further comments.

I think this is a Google Benchmark's bug where CPU affinity is enabled, and it doesn't seem to be compatible with macOS. However, since Iceberg depends on Google Benchmark, it should at least inform macOS developers how to handle this error, rather than casually assuming that CPU affinity detection is expected behavior and therefore not treating it as a bug

it seems #241 can slove this, if set -DENABLE_ROARING_MICROBENCHMARKS=OFF this can be ok. ths!

[zhjwpku]

It seems pointless to run these tests and benchmarks within iceberg-cpp. I disable them in #241, can you give it a try?

cpu_set_t is not support on MacOS, it is only supported on Linux. Then I found the flag HAVE_PTHREAD_AFFINITY=off is not useful, you can manually comment out the CPU affinity settings to solve this problem. like below:

pthread affinity, if available

if(HAVE_PTHREAD_AFFINITY)
#target_compile_definitions(benchmark PRIVATE -DBENCHMARK_HAS_PTHREAD_AFFINITY)
endif()

I digged into this a little bit, when you set HAVE_PTHREAD_AFFINITY either to on or off, google benchmark will define HAVE_PTHREAD_AFFINITY, see [1]. Then I tried cmake -S . -B build -DENABLE_ROARING_MICROBENCHMARKS=ON to build CRoaring, it passed. You can observe some message like:

-- Compiling and running to test HAVE_PTHREAD_AFFINITY
-- Performing Test HAVE_PTHREAD_AFFINITY -- failed to compile

I think HAVE_PTHREAD_AFFINITY is supposed to be automatically detected and set by the google benchmark building system, may I ask why you specify it manually?
[1] https://github.com/google/benchmark/blob/main/cmake/CXXFeatureCheck.cmake#L26-L29

..., MacOS do not have cpu_set_t , so benchmark's PTHREAD_AFFINITY can not open on Mac

After checking your reply again, I confirm that the error triggered by -DCXXFEATURECHECK_DEBUG=on is not an actual issue, it's intentionally used by the build system to probe OS capabilities during feature detection, I'll close this issue if no further comments.

I think this is a Google Benchmark's bug where CPU affinity is enabled, and it doesn't seem to be compatible with macOS. However, since Iceberg depends on Google Benchmark, it should at least inform macOS developers how to handle this error, rather than casually assuming that CPU affinity detection is expected behavior and therefore not treating it as a bug

Iceberg-cpp doesn't directly depend on Google Benchmark, we have CRoaring as a third party, and ENABLE_ROARING_MICROBENCHMARKS is an option from CRoaring, and I have created a PR to disable that. I also don't think this is a Google Benchmark's bug, as I already stated, the error triggered by -DCXXFEATURECHECK_DEBUG=on is just a check, from which Benchmark can know the os capabilities.

[zhjwpku]

Closed as #241 merged, thanks for the report.