Skip to content

Commit 18f2298

Browse files
preshanthsanbw
authored andcommitted
Updated cmake to robustly find dependencies using modules in cmake
I have edited our build system to robustly find dependencies when they are in non system paths such as in a spack environment for example. This lead to explicitly specifying some of the dependency root paths but that is a more robust way to build anyways. This build has been tested on vista in tacc by both @sbhatnag and I. He has also checked that the cpu only build builds fine on stampede on tacc. It has been built on a bunch of RHEL8 machine in NRAO and my ubuntu 24.04 laptop and it does fine. So I think its ready to merge. Squashed commits: * Fixing build script to mimic the vista one * stampede build * Adding cxx setting for kokkos * Adding an explicit include for casacore headers * Updating the FindCasacore.cmake path to be with other modules. We are also explicitly hinting for the findcasacore to look inside our local path * Fix CI pipeline issues with ccache and Spack detection Resolved three CI configuration issues: 1. Removed explicit CMAKE_CXX_COMPILER_LAUNCHER settings that caused Kokkos build failures. Kokkos rejects compiler launchers when the base compiler cannot compile CUDA. Since ccache is already in PATH at /usr/lib64/ccache/, CMake uses it automatically without explicit launcher configuration. 2. Isolated SPACK_ROOT environment variables to spack-test job only. Previously defined as global variables, these leaked into build-job causing Find modules to prefer Spack installations over system packages. Updated FindGSL.cmake and FindWCSLIB.cmake to respect LIBRA_USE_SPACK flag before searching Spack directories. 3. Set allow_failure: true for all manual CI jobs (manual-trigger, manual-cpu-only, manual-spack) to prevent untriggered manual jobs from blocking pipeline success on merge requests. These changes ensure cmake-refactoring branch builds successfully in CI while maintaining ccache functionality and proper dependency resolution. Or if you prefer something shorter: Fix CI: Remove explicit ccache launcher and isolate Spack variables - Remove CMAKE_CXX_COMPILER_LAUNCHER to fix Kokkos CUDA build with ccache - Move SPACK_ROOT variables from global to spack-test job scope - Update Find modules to respect LIBRA_USE_SPACK flag - Set allow_failure: true for manual jobs to unblock MR pipelines * Adding the script to setup env variables for TACC/vista * Updating actions workflow to include a submodule sync and update * Updating to fix namespace clash * Updating to remove kokkos and only keeper header * remove spurious find * Updating the kokkos target linking name * Trying to debug the linking from kokkos wrappers * Updating for wcslib finding * Updating with proper cmake targets to link against * Updated cfitsio spack finding * Update wcslib finding with spack * Adding proper include dirs * Adding the find modules * Adding findcasacore cmake * Updated to remove pkgconfig and introduce finding based on cmake * Updating the FFTW resume * Updating the FFTW resume * Updating the FFTW resume * Updating the resume features * Updating to make python optional and let pybind11 do the finding. * Components portion of the refactor * Simplified main cmake that works * Refactoring existing CMakeLists into functions using external projectadd * Refactor Cmake to be more modular. Step1 we have not altered syntax yet Closes: Merge request: https://gitlab.nrao.edu/ardg/libra/-/merge_requests/139
1 parent e1ca0ae commit 18f2298

39 files changed

+1846
-106
lines changed

.gitignore

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
11
*~
22
/build/
3-
/install/
3+
/install/
4+
src/synthesis/TransformMachines/test/tBeamCalc.cc.bak

.gitlab-ci.yml

Lines changed: 11 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -17,10 +17,6 @@ variables:
1717
CCACHE_BASEDIR: $CI_PROJECT_DIR
1818
CCACHE_DIR: "${CI_BUILDS_DIR}/ccache"
1919
GIT_CLEAN_FLAGS: none
20-
SPACK_ROOT: "${CI_BUILDS_DIR}/spack-installs/spack-persistent"
21-
SPACK_TEST_BASE_ENV: "spack-test-base"
22-
LIBRA_TEST_ENV: "libra-test-${CI_PIPELINE_ID}"
23-
SPACK_CACHE_DIR: "${CI_BUILDS_DIR}/spack-cache"
2420

2521
cache:
2622
- key: ccache-global
@@ -75,9 +71,6 @@ build-job:
7571
- export CCACHE_BASEDIR="$PWD"
7672
- export CCACHE_COMPILERCHECK=content
7773
- export CCACHE_MAXSIZE=5G
78-
# Ensure ccache is used by cmake
79-
- export CMAKE_CXX_COMPILER_LAUNCHER=ccache
80-
- export CMAKE_C_COMPILER_LAUNCHER=ccache
8174
- mkdir -p $CCACHE_DIR
8275
- if [ -d "$CCACHE_DIR" ] && [ "$(ls -A $CCACHE_DIR)" ]; then echo "Using existing ccache with $(ccache --show-stats | grep 'files in cache')"; else echo "Starting with empty ccache"; fi
8376
- ccache --show-stats || true
@@ -100,7 +93,7 @@ build-job:
10093
- which nvcc
10194
- mkdir -p build
10295
- cd build
103-
- cmake -DApps_BUILD_TESTS=ON -DNCORES=8 -DKokkos_CUDA_ARCH_NAME=Kokkos_ARCH_VOLTA70 -DCMAKE_CXX_COMPILER_LAUNCHER=ccache -DCMAKE_C_COMPILER_LAUNCHER=ccache ..
96+
- cmake -DApps_BUILD_TESTS=ON -DNCORES=8 -DKokkos_CUDA_ARCH_NAME=Kokkos_ARCH_VOLTA70 ..
10497
- make
10598
after_script:
10699
- echo "=== ccache statistics ==="
@@ -191,6 +184,11 @@ test-job:
191184
spack-test:
192185
needs: []
193186
stage: spack-test
187+
variables:
188+
SPACK_ROOT: "${CI_BUILDS_DIR}/spack-installs/spack-persistent"
189+
SPACK_TEST_BASE_ENV: "spack-test-base"
190+
LIBRA_TEST_ENV: "libra-test-${CI_PIPELINE_ID}"
191+
SPACK_CACHE_DIR: "${CI_BUILDS_DIR}/spack-cache"
194192
rules:
195193
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
196194
changes:
@@ -361,37 +359,32 @@ manual-trigger:
361359
- export CCACHE_BASEDIR="$PWD"
362360
- export CCACHE_COMPILERCHECK=content
363361
- export CCACHE_MAXSIZE=2G
364-
- export CMAKE_CXX_COMPILER_LAUNCHER=ccache
365-
- export CMAKE_C_COMPILER_LAUNCHER=ccache
366362
- if [ -d "$CCACHE_DIR" ] && [ "$(ls -A $CCACHE_DIR)" ]; then echo "Using existing ccache"; else echo "No active ccache found"; fi
367363
- mkdir -p $CCACHE_DIR
368364
- ccache --show-stats || true
369365
- git clean -ffdx
370366
- mkdir -p build
371367
- cd build
372-
- cmake -DApps_BUILD_TESTS=ON -DKokkos_CUDA_ARCH_NAME=Kokkos_ARCH_VOLTA70 -DCMAKE_CXX_COMPILER_LAUNCHER=ccache -DCMAKE_C_COMPILER_LAUNCHER=ccache ..
368+
- cmake -DApps_BUILD_TESTS=ON -DKokkos_CUDA_ARCH_NAME=Kokkos_ARCH_VOLTA70 ..
373369
- make -j$(nproc)
374370
- ccache --show-stats || true
375371
when: manual
376372
only:
377373
- branches
378-
allow_failure: false
374+
allow_failure: true
379375

380376
manual-cpu-only:
381377
stage: build
382378
rules:
383379
- when: manual
384-
allow_failure: false
380+
allow_failure: true
385381
before_script:
386382
# Ensure submodules are in sync with the index
387383
- git submodule sync
388384
- git submodule update --init --recursive
389385
- export CCACHE_BASEDIR="$PWD"
390386
- export CCACHE_COMPILERCHECK=content
391387
- export CCACHE_MAXSIZE=5G
392-
# Ensure ccache is used by cmake
393-
- export CMAKE_CXX_COMPILER_LAUNCHER=ccache
394-
- export CMAKE_C_COMPILER_LAUNCHER=ccache
395388
- mkdir -p $CCACHE_DIR
396389
- if [ -d "$CCACHE_DIR" ] && [ "$(ls -A $CCACHE_DIR)" ]; then echo "Using existing ccache with $(ccache --show-stats | grep 'files in cache')"; else echo "Starting with empty ccache"; fi
397390
- ccache --show-stats || true
@@ -404,7 +397,7 @@ manual-cpu-only:
404397
- echo "=== Building without CUDA ==="
405398
- mkdir -p build
406399
- cd build
407-
- cmake -DApps_BUILD_TESTS=ON -DNCORES=8 -DLIBRA_DISABLE_CUDA=ON -DCMAKE_CXX_COMPILER_LAUNCHER=ccache -DCMAKE_C_COMPILER_LAUNCHER=ccache ..
400+
- cmake -DApps_BUILD_TESTS=ON -DNCORES=8 -DLIBRA_DISABLE_CUDA=ON ..
408401
- make -j$(nproc)
409402
after_script:
410403
- echo "=== ccache statistics ==="
@@ -416,5 +409,5 @@ manual-spack:
416409
when: manual
417410
rules:
418411
- when: manual
419-
allow_failure: false
412+
allow_failure: true
420413

CMakeLists.txt

Lines changed: 14 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -1,37 +1,25 @@
1-
#-*- cmake -*-
2-
# Copyright (C) 2025
3-
# Associated Universities, Inc. Washington DC, USA.
4-
#
5-
# This library is free software; you can redistribute it and/or modify it
6-
# under the terms of the GNU Library General Public License as published by
7-
# the Free Software Foundation; either version 2 of the License, or (at your
8-
# option) any later version.
9-
#
10-
# This library is distributed in the hope that it will be useful, but WITHOUT
11-
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
12-
# FITNESS FOR A PARTICULAR PURPOSE. See the GNU Library General Public
13-
# License for more details.is
14-
#
15-
# You should have received a copy of the GNU Library General Public License
16-
# along with this library; if not, write to the Free Software Foundation,
17-
# Inc., 675 Massachusetts Ave, Cambridge, MA 02139, USA.
18-
#
19-
# Correspondence concerning this should be addressed as follows:
20-
# Postal address: National Radio Astronomy Observatory
21-
# 1003 Lopezville Road,
22-
# Socorro, NM - 87801, USA
23-
#
1+
# LibRA Main CMakeLists.txt - Clean modular approach
2+
# Copyright 2025 Associated Universities, Inc. Washington DC, USA.
243

254
cmake_minimum_required(VERSION 3.18)
265
project("LibRA Apps"
27-
DESCRIPTION "LibRA CL, C/C++, Py Interfaces"
28-
HOMEPAGE_URL https://github.com/ARDG-NRAO/LibRA
29-
LANGUAGES CXX Fortran C)
6+
DESCRIPTION "LibRA CL, C/C++, Py Interfaces"
7+
HOMEPAGE_URL https://github.com/ARDG-NRAO/LibRA
8+
LANGUAGES CXX Fortran C)
309

3110

3211

3312
# Path and directory setup
3413
include(${CMAKE_CURRENT_SOURCE_DIR}/cmake/paths.cmake)
14+
message("================ Top-level CMAKE_MODULE_PATH after paths.cmake ================")
15+
message("CMAKE_MODULE_PATH: ${CMAKE_MODULE_PATH}")
16+
message("Testing if FindWCSLIB.cmake can be found...")
17+
find_file(WCSLIB_FIND_MODULE FindWCSLIB.cmake PATHS ${CMAKE_MODULE_PATH})
18+
message("FindWCSLIB.cmake found at: ${WCSLIB_FIND_MODULE}")
19+
message("Testing find_package(WCSLIB) at top level...")
20+
find_package(WCSLIB QUIET)
21+
message("Top-level WCSLIB_FOUND: ${WCSLIB_FOUND}")
22+
message("================ End MODULE_PATH debug ================")
3523

3624
# Load project configuration
3725
include(${CMAKE_CURRENT_SOURCE_DIR}/cmake/libraconfig.cmake)
@@ -45,7 +33,6 @@ endif()
4533
include(${CMAKE_CURRENT_SOURCE_DIR}/cmake/PrintLibraSummary.cmake)
4634
print_libra_summary()
4735

48-
4936
# Include dependencies
5037
include(${CMAKE_CURRENT_SOURCE_DIR}/cmake/dependencies/Kokkos.cmake)
5138
include(${CMAKE_CURRENT_SOURCE_DIR}/cmake/dependencies/FFTW.cmake)

0 commit comments

Comments
 (0)