Skip to content

build(deps): Add task-based installations for LZ4 and Zstandard dependencies; Use latest versions for source-based installations. #1156

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Aug 11, 2025
Merged

build(deps): Add task-based installations for LZ4 and Zstandard dependencies; Use latest versions for source-based installations. #1156

Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
9b862bb
Remove lz4 installations
Bill-hbrhbr Aug 4, 2025
2df11f3
Install LZ4 via task and update docs and find module scripts
Bill-hbrhbr Aug 4, 2025
982ff31
Use LZ4 built-in cmake find package moodule
Bill-hbrhbr Aug 4, 2025
5835dbc
Temporarily test lz4
Bill-hbrhbr Aug 4, 2025
ac3292d
Revert "Temporarily test lz4"
Bill-hbrhbr Aug 4, 2025
fc2b711
Migrate zstandard to task
Bill-hbrhbr Aug 4, 2025
48fd88d
misc fix
Bill-hbrhbr Aug 4, 2025
47a2b18
Fix macos package selection with find package hints. Add missing link…
Bill-hbrhbr Aug 4, 2025
4ec9c70
Use variable instead of library target for multiplexing lz4 and zstd …
Bill-hbrhbr Aug 6, 2025
cbd83cd
Merge branch 'main' into modernize-lz4-zstd-install
Bill-hbrhbr Aug 6, 2025
7dc87e9
Put back zstd guard and remove superfluous zstd linkage
Bill-hbrhbr Aug 8, 2025
ea50989
Put back lz4 and zstd installations for now since libarchive cannot s…
Bill-hbrhbr Aug 8, 2025
434177b
Merge branch 'main' into modernize-lz4-zstd-install
Bill-hbrhbr Aug 8, 2025
60b2770
Merge branch 'main' into modernize-lz4-zstd-install
Bill-hbrhbr Aug 9, 2025
85e2e53
Apply suggestions from code review
Bill-hbrhbr Aug 10, 2025
a7c2b63
Address review concern
Bill-hbrhbr Aug 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 5 additions & 8 deletions components/core/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -297,16 +297,13 @@ if(CLP_NEED_YSTDLIB)
add_subdirectory("${CLP_YSTDLIB_SOURCE_DIRECTORY}" "${CMAKE_BINARY_DIR}/ystdlib" EXCLUDE_FROM_ALL)
endif()

# Find and setup ZStd Library
if(CLP_NEED_ZSTD)
find_package(zstd 1.5.7 REQUIRED HINTS "${zstd_ROOT}")
message(STATUS "Found zstd ${zstd_VERSION}")
if(CLP_USE_STATIC_LIBS)
set(ZStd_USE_STATIC_LIBS ON)
endif()
find_package(ZStd 1.4.4 REQUIRED)
if(ZStd_FOUND)
message(STATUS "Found ZStd ${ZStd_VERSION}")
set(zstd_TARGET zstd::libzstd_static)
else()
message(FATAL_ERROR "Could not find ${CLP_LIBS_STRING} libraries for ZStd")
set(zstd_TARGET zstd::libzstd_shared)
endif()
endif()

Expand Down Expand Up @@ -772,7 +769,7 @@ if(CLP_BUILD_TESTING)
ystdlib::containers
ystdlib::error_handling
${LIBLZMA_LIBRARIES}
ZStd::ZStd
${zstd_TARGET}
)
target_compile_features(unitTest
PRIVATE cxx_std_20
Expand Down
103 changes: 0 additions & 103 deletions components/core/cmake/Modules/FindLZ4.cmake
View file
Open in desktop

This file was deleted.

106 changes: 0 additions & 106 deletions components/core/cmake/Modules/FindZStd.cmake
View file
Open in desktop

This file was deleted.

2 changes: 1 addition & 1 deletion components/core/src/clp/clg/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,7 @@ if(CLP_BUILD_EXECUTABLES)
yaml-cpp
ystdlib::containers
ystdlib::error_handling
ZStd::ZStd
${zstd_TARGET}
)
# Put the built executable at the root of the build directory
set_target_properties(
Expand Down
2 changes: 1 addition & 1 deletion components/core/src/clp/clo/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,7 @@ if(CLP_BUILD_EXECUTABLES)
clp::string_utils
ystdlib::containers
ystdlib::error_handling
ZStd::ZStd
${zstd_TARGET}
)
# Put the built executable at the root of the build directory
set_target_properties(
Expand Down
2 changes: 1 addition & 1 deletion components/core/src/clp/clp/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -189,7 +189,7 @@ if(CLP_BUILD_EXECUTABLES)
yaml-cpp
ystdlib::containers
ystdlib::error_handling
ZStd::ZStd
${zstd_TARGET}
)
# Put the built executable at the root of the build directory
set_target_properties(
Expand Down
2 changes: 1 addition & 1 deletion components/core/src/clp/make_dictionaries_readable/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ if(CLP_BUILD_EXECUTABLES)
spdlog::spdlog
clp::string_utils
ystdlib::containers
ZStd::ZStd
${zstd_TARGET}
)
# Put the built executable at the root of the build directory
set_target_properties(
Expand Down
4 changes: 2 additions & 2 deletions components/core/src/clp_s/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,7 @@ if(CLP_BUILD_CLP_S_CLP_DEPENDENCIES)
OpenSSL::Crypto
spdlog::spdlog
ystdlib::error_handling
ZStd::ZStd
${zstd_TARGET}
)
endif()

Expand Down Expand Up @@ -204,7 +204,7 @@ if(CLP_BUILD_CLP_S_IO)
clp_s::clp_dependencies
fmt::fmt
spdlog::spdlog
ZStd::ZStd
${zstd_TARGET}
)
endif()

Expand Down
2 changes: 1 addition & 1 deletion components/core/src/clp_s/indexer/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,7 @@ if(CLP_BUILD_EXECUTABLES)
spdlog::spdlog
yaml-cpp
ystdlib::containers
ZStd::ZStd
${zstd_TARGET}
)
# Put the built executable at the root of the build directory
set_target_properties(
Expand Down
2 changes: 1 addition & 1 deletion components/core/src/glt/glt/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -192,7 +192,7 @@ if(CLP_BUILD_EXECUTABLES)
clp::string_utils
yaml-cpp
ystdlib::error_handling
ZStd::ZStd
${zstd_TARGET}
)
# Put the built executable at the root of the build directory
set_target_properties(
Expand Down
1 change: 1 addition & 0 deletions components/core/tools/scripts/lib_install/centos-stream-9/install-prebuilt-packages.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ dnf install -y \
libarchive-devel \
libcurl-devel \
libzstd-devel \
lz4-devel \
make \
mariadb-connector-c-devel \
openssl-devel \
Expand Down
4 changes: 2 additions & 2 deletions components/core/tools/scripts/lib_install/ubuntu-jammy/install-packages-from-source.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,6 @@ lib_install_scripts_dir=$script_dir/..

"$lib_install_scripts_dir"/libarchive.sh 3.5.1
"$lib_install_scripts_dir"/liblzma.sh 5.8.1
"$lib_install_scripts_dir"/lz4.sh 1.8.2
"$lib_install_scripts_dir"/lz4.sh 1.10.0
"$lib_install_scripts_dir"/msgpack.sh 7.0.0
"$lib_install_scripts_dir"/zstandard.sh 1.4.9
"$lib_install_scripts_dir"/zstandard.sh 1.5.7
2 changes: 2 additions & 0 deletions docs/src/dev-guide/components-core/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ The task will download, build, and install (within the build directory) the foll
| [fmt](https://github.com/fmtlib/fmt) | v10.2.1 |
| [json](https://github.com/nlohmann/json.git) | v3.11.3 |
| [log-surgeon](https://github.com/y-scope/log-surgeon) | f801a3f |
| [lz4](https://github.com/lz4/lz4) | v1.10.0 |
| [mongo-cxx-driver](https://github.com/mongodb/mongo-cxx-driver) | r3.10.2 |
| [simdjson](https://github.com/simdjson/simdjson) | v3.13.0 |
| [spdlog](https://github.com/gabime/spdlog) | v1.14.1 |
Expand All @@ -50,6 +51,7 @@ The task will download, build, and install (within the build directory) the foll
| [yaml-cpp](https://github.com/jbeder/yaml-cpp.git) | v0.7.0 |
| [yscope-log-viewer](https://github.com/y-scope/yscope-log-viewer.git) | 969ff35 |
| [ystdlib-cpp](https://github.com/y-scope/ystdlib-cpp.git) | d80cf86 |
| [zstd](https://github.com/facebook/zstd) | v1.5.7 |

### Environment

Expand Down
Loading
Loading