Skip to content

build(deps): Migrate Boost and msgpack-cxx dependencies to task-based installation (fixes #1243); Drop required Boost version back to 1.81; Add missing ZStd CMake target links. #1254

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Aug 24, 2025
+83 −179
Merged

build(deps): Migrate Boost and msgpack-cxx dependencies to task-based installation (fixes #1243); Drop required Boost version back to 1.81; Add missing ZStd CMake target links. #1254

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
0189799
modernize boost and msgpack-cxx
Bill-hbrhbr Aug 22, 2025
f0987f8
Merge branch 'main' into modernize-boost-msgpack-cxx-install
Bill-hbrhbr Aug 22, 2025
0f9dcf3
Add missing zstd target
Bill-hbrhbr Aug 22, 2025
249e1b3
Fix zstd_TARGEt
Bill-hbrhbr Aug 23, 2025
6de558a
Complete msgpack-cxx cmake gen args
Bill-hbrhbr Aug 23, 2025
542ed09
Remove unnecessary cmake gen arg setting
Bill-hbrhbr Aug 23, 2025
eec7e05
Minor improvements
Bill-hbrhbr Aug 23, 2025
ddaac0c
Remove cxx11 setting to OFF (unnecessary)
Bill-hbrhbr Aug 23, 2025
b6d5164
Remove unused scripts
Bill-hbrhbr Aug 23, 2025
4576159
Update docs version
Bill-hbrhbr Aug 23, 2025
87a0be4
Apply suggestions from code review
Bill-hbrhbr Aug 24, 2025
aa8e7c3
Address review comments
Bill-hbrhbr Aug 24, 2025
a33ac1c
Add back missing target
Bill-hbrhbr Aug 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 20 additions & 13 deletions components/core/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -157,12 +157,18 @@ if(CLP_NEED_BOOST)
if(CLP_USE_STATIC_LIBS)
set(Boost_USE_STATIC_LIBS ON)
endif()
find_package(Boost 1.89 REQUIRED iostreams program_options filesystem regex url)
if(Boost_FOUND)
message(STATUS "Found Boost ${Boost_VERSION}")
else()
message(FATAL_ERROR "Could not find ${CLP_LIBS_STRING} libraries for Boost")
endif()
find_package(
Boost
1.81...1.88
REQUIRED
filesystem
iostreams
program_options
regex
system
url
)
message(STATUS "Found Boost ${Boost_VERSION}")
endif()

if(CLP_NEED_CATCH2)
Expand Down Expand Up @@ -271,14 +277,10 @@ if(CLP_NEED_MONGOCXX)
endif()
endif()

# Find and setup msgpack
# Find and setup msgpack-cxx
if(CLP_NEED_MSGPACKCXX)
find_package(msgpack-cxx 7.0.0 REQUIRED)
if(msgpack-cxx_FOUND)
message(STATUS "Found msgpack-cxx ${msgpack-cxx_VERSION}")
else()
message(FATAL_ERROR "Could not find msgpack-cxx")
endif()
message(STATUS "Found msgpack-cxx ${msgpack-cxx_VERSION}")
endif()

find_package(Threads REQUIRED)
Expand Down Expand Up @@ -731,7 +733,11 @@ if(CLP_BUILD_TESTING)
target_link_libraries(unitTest
PRIVATE
absl::flat_hash_map
Boost::filesystem Boost::iostreams Boost::program_options Boost::regex Boost::url
Boost::filesystem
Boost::iostreams
Boost::program_options
Boost::regex
Boost::url
Catch2::Catch2WithMain
${CURL_LIBRARIES}
clp_s::search::ast
Expand All @@ -745,6 +751,7 @@ if(CLP_BUILD_TESTING)
LibLZMA::LibLZMA
MariaDBClient::MariaDBClient
${MONGOCXX_TARGET}
msgpack-cxx
nlohmann_json::nlohmann_json
simdjson::simdjson
spdlog::spdlog
Expand Down
1 change: 1 addition & 0 deletions components/core/cmake/Options/options.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -284,6 +284,7 @@ function(set_clp_s_json_constructor_dependencies)
CLP_NEED_FMT
CLP_NEED_MONGOCXX
CLP_NEED_SPDLOG
CLP_NEED_ZSTD
)
endfunction()

Expand Down
1 change: 1 addition & 0 deletions components/core/src/clp/clg/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,7 @@ if(CLP_BUILD_EXECUTABLES)
fmt::fmt
log_surgeon::log_surgeon
MariaDBClient::MariaDBClient
msgpack-cxx
nlohmann_json::nlohmann_json
spdlog::spdlog
${sqlite_LIBRARY_DEPENDENCIES}
Expand Down
1 change: 1 addition & 0 deletions components/core/src/clp/clp/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,7 @@ if(CLP_BUILD_EXECUTABLES)
${sqlite_LIBRARY_DEPENDENCIES}
LibArchive::LibArchive
MariaDBClient::MariaDBClient
msgpack-cxx
nlohmann_json::nlohmann_json
${STD_FS_LIBS}
clp::string_utils
Expand Down
9 changes: 5 additions & 4 deletions components/core/src/clp_s/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ set(
../clp/cli_utils.hpp
../clp/Defs.h
../clp/EncodedVariableInterpreter.cpp
../clp/EncodedVariableInterpreter.hpp
../clp/EncodedVariableInterpreter.hpp
../clp/ErrorCode.hpp
../clp/ffi/encoding_methods.cpp
../clp/ffi/encoding_methods.hpp
Expand Down Expand Up @@ -69,15 +69,15 @@ set(
../clp/ir/parsing.inc
../clp/ir/types.hpp
../clp/LogSurgeonReader.cpp
../clp/LogSurgeonReader.hpp
../clp/LogSurgeonReader.hpp
../clp/NetworkReader.cpp
../clp/NetworkReader.hpp
../clp/networking/socket_utils.cpp
../clp/networking/socket_utils.hpp
../clp/Query.cpp
../clp/Query.hpp
../clp/QueryToken.cpp
../clp/QueryToken.hpp
../clp/QueryToken.hpp
../clp/ReaderInterface.cpp
../clp/ReaderInterface.hpp
../clp/ReadOnlyMemoryMappedFile.cpp
Expand Down Expand Up @@ -115,6 +115,7 @@ if(CLP_BUILD_CLP_S_CLP_DEPENDENCIES)
clp::string_utils
log_surgeon::log_surgeon
ystdlib::containers
${zstd_TARGET}
PRIVATE
Boost::regex
${CURL_LIBRARIES}
Expand All @@ -124,7 +125,6 @@ if(CLP_BUILD_CLP_S_CLP_DEPENDENCIES)
OpenSSL::Crypto
spdlog::spdlog
ystdlib::error_handling
${zstd_TARGET}
)
endif()

Expand Down Expand Up @@ -354,6 +354,7 @@ if(CLP_BUILD_CLP_S_JSONCONSTRUCTOR)
clp_s_json_constructor
PUBLIC
clp_s::archive_reader
${zstd_TARGET}
Comment on lines 354 to +357
Copy link
Contributor

@coderabbitai coderabbitai bot Aug 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Adding ${zstd_TARGET} as PUBLIC on clp_s::json_constructor: OK, but ensure the build option reflects this.

Linking zstd as PUBLIC here is acceptable if any public header (directly or indirectly) requires zstd.h. Regardless, because zstd is now a required dependency for this target, verify that CLP_NEED_ZSTD has been added to set_clp_s_json_constructor_dependencies in options.cmake as planned in this migration.

Use this quick check:

🏁 Script executed:

#!/bin/bash
# Locate the dependency setter and verify CLP_NEED_ZSTD is included
rg -nP 'function\s*\(\s*set_clp_s_json_constructor_dependencies' -C3
rg -n 'CLP_NEED_ZSTD' -C2 --glob '*.cmake' --glob '**/CMakeLists.txt'

Length of output: 2941

Ensure CLP_NEED_ZSTD is added to set_clp_s_json_constructor_dependencies

The set_clp_s_json_constructor_dependencies function in components/core/cmake/Options/options.cmake currently only sets:

function(set_clp_s_json_constructor_dependencies)
    set_clp_need_flags(
        CLP_NEED_FMT
        CLP_NEED_MONGOCXX
    )
endfunction()

Since ${zstd_TARGET} has been added as a PUBLIC dependency of clp_s_json_constructor, you must also expose it via the CLP_NEED_ZSTD option flag. Please update the function as follows:

--- a/components/core/cmake/Options/options.cmake
+++ b/components/core/cmake/Options/options.cmake
@@ -282,6 +282,7 @@ function(set_clp_s_json_constructor_dependencies)
     set_clp_need_flags(
         CLP_NEED_FMT
         CLP_NEED_MONGOCXX
+        CLP_NEED_ZSTD
     )
 endfunction()

• File: components/core/cmake/Options/options.cmake
• Function: set_clp_s_json_constructor_dependencies (around line 282)

Once this is in place, the public zstd dependency and corresponding build option will be properly wired.

🤖 Prompt for AI Agents 
In components/core/cmake/Options/options.cmake around line 282, the function
set_clp_s_json_constructor_dependencies defines needed flags but omits
CLP_NEED_ZSTD; update the function to call set_clp_need_flags with CLP_NEED_FMT,
CLP_NEED_MONGOCXX and also CLP_NEED_ZSTD so the public ${zstd_TARGET} dependency
is exposed via the build option for clp_s_json_constructor.

PRIVATE
fmt::fmt
${MONGOCXX_TARGET}
Expand Down
3 changes: 2 additions & 1 deletion components/core/src/clp_s/indexer/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ set(
../../clp/database_utils.hpp
../../clp/Defs.h
../../clp/EncodedVariableInterpreter.cpp
../../clp/EncodedVariableInterpreter.hpp
../../clp/EncodedVariableInterpreter.hpp
../../clp/ErrorCode.hpp
../../clp/ffi/encoding_methods.cpp
../../clp/ffi/encoding_methods.hpp
Expand Down Expand Up @@ -118,6 +118,7 @@ if(CLP_BUILD_EXECUTABLES)
clp::string_utils
date::date
MariaDBClient::MariaDBClient
msgpack-cxx
nlohmann_json::nlohmann_json
OpenSSL::Crypto
simdjson::simdjson
Expand Down
1 change: 1 addition & 0 deletions components/core/src/reducer/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ if(CLP_BUILD_EXECUTABLES)
target_link_libraries(reducer-server
PRIVATE
Boost::program_options
Boost::system
clp::string_utils
fmt::fmt
${MONGOCXX_TARGET}
Expand Down
4 changes: 0 additions & 4 deletions components/core/tools/scripts/lib_install/centos-stream-9/install-packages-from-source.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,4 @@ set -u
script_dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
lib_install_scripts_dir="${script_dir}/.."

# NOTE: The remaining installation scripts depend on boost, so we install it beforehand.
"${lib_install_scripts_dir}/install-boost.sh" 1.89.0

"${lib_install_scripts_dir}/liblzma.sh" 5.8.1
"${lib_install_scripts_dir}/msgpack.sh" 7.0.0
48 changes: 0 additions & 48 deletions components/core/tools/scripts/lib_install/install-boost.sh
View file
Open in desktop

This file was deleted.

2 changes: 0 additions & 2 deletions components/core/tools/scripts/lib_install/macos/install-all.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,15 +27,13 @@ if ! command -v uv ; then
fi

brew install \
boost \
coreutils \
gcc \
java11 \
libarchive \
llvm@16 \
lz4 \
mariadb-connector-c \
msgpack-cxx \
xz \
zstd

Expand Down
5 changes: 0 additions & 5 deletions components/core/tools/scripts/lib_install/manylinux_2_28/install-packages-from-source.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,6 @@ set -o pipefail
script_dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
lib_install_scripts_dir="${script_dir}/.."

# NOTE: The remaining installation scripts depend on boost, so we install it beforehand.
"${lib_install_scripts_dir}/install-boost.sh" 1.89.0

# NOTE:
# 1. libarchive may statically link with LZMA, LZ4, and Zstandard, so we install them beforehand.
# 2. The versions of libarchive, LZMA, LZ4, and Zstandard available in manylinux_2_28's package
Expand All @@ -18,5 +15,3 @@ lib_install_scripts_dir="${script_dir}/.."
"${lib_install_scripts_dir}/lz4.sh" 1.10.0
"${lib_install_scripts_dir}/zstandard.sh" 1.5.7
"${lib_install_scripts_dir}/libarchive.sh" 3.8.0

"${lib_install_scripts_dir}/msgpack.sh" 7.0.0
89 changes: 0 additions & 89 deletions components/core/tools/scripts/lib_install/msgpack.sh
View file
Open in desktop

This file was deleted.

5 changes: 0 additions & 5 deletions components/core/tools/scripts/lib_install/musllinux_1_2/install-packages-from-source.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,6 @@ set -o pipefail
script_dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
lib_install_scripts_dir="${script_dir}/.."

# NOTE: The remaining installation scripts depend on boost, so we install it beforehand.
"${lib_install_scripts_dir}/install-boost.sh" 1.89.0

# NOTE:
# 1. libarchive may statically link with LZMA, LZ4, and Zstandard, so we install them beforehand.
# 2. The versions of libarchive, LZMA, LZ4, and Zstandard available in musllinux_1_2's package
Expand All @@ -18,5 +15,3 @@ lib_install_scripts_dir="${script_dir}/.."
"${lib_install_scripts_dir}/lz4.sh" 1.10.0
"${lib_install_scripts_dir}/zstandard.sh" 1.5.7
"${lib_install_scripts_dir}/libarchive.sh" 3.8.0

"${lib_install_scripts_dir}/msgpack.sh" 7.0.0
4 changes: 0 additions & 4 deletions components/core/tools/scripts/lib_install/ubuntu-jammy/install-packages-from-source.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,7 @@ set -u
script_dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
lib_install_scripts_dir=$script_dir/..

# NOTE: boost must be installed first since the remaining packages depend on it
"$lib_install_scripts_dir"/install-boost.sh 1.89.0

"$lib_install_scripts_dir"/libarchive.sh 3.5.1
"$lib_install_scripts_dir"/liblzma.sh 5.8.1
"$lib_install_scripts_dir"/lz4.sh 1.10.0
"$lib_install_scripts_dir"/msgpack.sh 7.0.0
"$lib_install_scripts_dir"/zstandard.sh 1.5.7
2 changes: 2 additions & 0 deletions docs/src/dev-guide/components-core/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ The task will download, build, and install (within the build directory) the foll
|-----------------------------------------------------------------------|----------------|
| [abseil-cpp](https://github.com/abseil/abseil-cpp) | 20250512.0 |
| [ANTLR](https://www.antlr.org) | v4.13.2 |
| [Boost](https://github.com/boostorg/boost) | v1.87.0 |
| [Catch2](https://github.com/catchorg/Catch2.git) | v3.8.0 |
Comment on lines +40 to 41
Copy link
Contributor

@coderabbitai coderabbitai bot Aug 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick (assertive)

Clarify accepted Boost version range in docs to match CMake constraints

The table shows the task-installed Boost at v1.87.0, but CMake accepts 1.81–1.88. Adding a brief note prevents confusion when users bring their own Boost.

Apply either of the following:

Option A — inline parenthetical in the table cell:

-| [Boost](https://github.com/boostorg/boost)                            | v1.87.0        |
+| [Boost](https://github.com/boostorg/boost)                            | v1.87.0 (CMake accepts 1.81–1.88) |

Option B — short note immediately below the table:

 | [zstd](https://github.com/facebook/zstd)                              | v1.5.7         |
 
+Note: The Task-installed Boost default is v1.87.0. The CMake build accepts Boost versions in the range 1.81–1.88.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
| [Boost](https://github.com/boostorg/boost) | v1.87.0 |
| [Catch2](https://github.com/catchorg/Catch2.git) | v3.8.0 |
| [Boost](https://github.com/boostorg/boost) | v1.87.0 (CMake accepts 1.81–1.88) |
| [Catch2](https://github.com/catchorg/Catch2.git) | v3.8.0 |
Suggested change
| [Boost](https://github.com/boostorg/boost) | v1.87.0 |
| [Catch2](https://github.com/catchorg/Catch2.git) | v3.8.0 |
| [zstd](https://github.com/facebook/zstd) | v1.5.7 |
Note: The Task-installed Boost default is v1.87.0. The CMake build accepts Boost versions in the range 1.81–1.88.

| [date](https://github.com/HowardHinnant/date.git) | v3.0.1 |
| [fmt](https://github.com/fmtlib/fmt) | v10.2.1 |
Expand All @@ -45,6 +46,7 @@ The task will download, build, and install (within the build directory) the foll
| [liblzma](https://github.com/tukaani-project/xz) | v5.8.1 |
| [lz4](https://github.com/lz4/lz4) | v1.10.0 |
| [mongo-cxx-driver](https://github.com/mongodb/mongo-cxx-driver) | r3.10.2 |
| [msgpack-cxx](https://github.com/msgpack/msgpack-c/tree/cpp_master) | v7.0.0 |
| [simdjson](https://github.com/simdjson/simdjson) | v3.13.0 |
| [spdlog](https://github.com/gabime/spdlog) | v1.14.1 |
| [SQLite3](https://www.sqlite.org/download.html) | v3.36.0 |
Expand Down
Loading
Loading