Skip to content

Maintaining Spack Buildcache

Jump to bottom
Matthew Thompson edited this page Apr 24, 2025 · 18 revisions

This page details the efforts to create a buildcache for use in Spack CI tests in MAPL.

Useful Links

Building Packages on AWS

Preliminaries

Padded Length

Testing showed we apparently need to increase the length of the padding in the install tree. This is what I ran:

spack config add config:install_tree:padded_length:256

packages.yaml

My current packages.yaml is set as:

packages:
  all:
    require: target=x86_64_v3
    providers:
      mpi: [openmpi, intel-oneapi-mpi]
      blas: [openblas, intel-oneapi-mkl]
      lapack: [openblas, intel-oneapi-mkl]

  gcc:
    buildable: false
  openmpi:
    require:
    - any_of: ['%gcc']
      message: Only use Open MPI with GCC

  intel-oneapi-mpi:
    require:
    - any_of: ['%oneapi']
      message: Only use Intel MPI with Intel

  hdf5:
    variants: +fortran +szip +hl +threadsafe +mpi
    # Note that cdo requires threadsafe, but hdf5 doesn't
    # seem to want that with parallel. Hmm.
  netcdf-c:
    variants: +dap
    #variants: +hdf4 +dap
  esmf:
    #variants: ~pnetcdf ~xerces ~external-parallelio
    variants: ~pnetcdf ~xerces
  cdo:
    variants: ~proj ~fftw3
    # cdo wanted a lot of extra stuff for proj and fftw3. Turn off for now
  pflogger:
    variants: +mpi
  pfunit:
    variants: +mpi +fhamcrest
  fms:
    variants: ~gfs_phys +pic +yaml constants=GEOS precision=32,64 +deprecated_io
  mapl:
    variants: +extdata2g +fargparse +pflogger +pfunit ~pnetcdf

Building Packages

We want all the dependencies of MAPL, so we need to do:

spack install --only dependencies mapl
spack install mepo

This (should) get us everything that MAPL needs to build.

NOTE: Make sure you are using ESMF 8.8.0. This is needed because of the sprintf issue Atanas found. It's triggered in ubuntu 24

Additional Packages for GEOSgcm

We might want to reuse this cache for GEOSgcm as well. As such we need extra packages:

spack install openblas
spack install py-ruamel-yaml

Building the Buildcache

Create Spack Environment

It turns out, you can't build or push a buildcache unless you are in a Spack environment. So we need to create one:

spack env create geos-buildcache
spack env activate -p geos-buildcache

Add Packages to Environment

spack install --add esmf gftl gftl-shared fargparse pflogger pfunit yafyaml ecbuild udunits mepo openblas
spack concretize -f

Add mirror for buildcache

spack mirror add --oci-username mathomp4 --oci-password-variable MAPL_BUILDCACHE_TOKEN --unsigned geos-buildcache oci://ghcr.io/GEOS-ESM/geos-buildcache

Pushing the Buildcache

spack buildcache push --update-index oci://ghcr.io/GEOS-ESM/geos-buildcache

Update because of Spack

packages.yaml

Note: On 2025-03-27, I did a git pull on spack and uninstalled all and cleaned all and when I did that, my packages.yaml was updated with:

  gcc:
    externals:
    - spec: gcc@13.3.0 languages='c,c++,fortran'
      prefix: /usr
      extra_attributes:
        compilers:
          c: /usr/bin/gcc
          cxx: /usr/bin/g++
          fortran: /usr/bin/gfortran
        flags: {}
        environment: {}
        extra_rpaths: []
    - spec: gcc@14.2.0 languages='c,c++,fortran'
      prefix: /usr
      extra_attributes:
        compilers:
          c: /usr/bin/gcc-14
          cxx: /usr/bin/g++-14
          fortran: /usr/bin/gfortran-14
        flags: {}
        environment: {}
        extra_rpaths: []
    buildable: false

spack find

Also, when I run spack find I now see:

$ spack find
-- linux-ubuntu24.04-x86_64_v3 / gcc@14.2.0 ---------------------
automake@1.16.5      findutils@4.10.0    libedit@3.1-20240808  libxml2@2.13.5        openssh@9.9p1     readline@8.2
berkeley-db@18.1.40  gdbm@1.23           libevent@2.1.12       libyaml@0.2.5         openssl@3.4.1     snappy@1.2.1
bison@3.8.2          gettext@0.23.1      libffi@3.4.6          lz4@1.10.0            parallelio@2.6.3  sqlite@3.46.0
bzip2@1.0.8          git@2.48.1          libiconv@1.17         m4@1.4.19             pcre2@10.44       tar@1.35
c-blosc@1.21.5       gmake@4.4.1         libidn2@2.3.7         ncurses@6.5           perl@5.40.0       udunits@2.2.28
cmake@3.31.6         hdf5@1.14.5         libmd@1.1.0           netcdf-c@4.9.2        pfunit@4.11.1     util-linux-uuid@2.40.4
curl@8.11.1          hwloc@2.11.1        libpciaccess@0.17     netcdf-fortran@4.6.1  pigz@2.8          xz@5.6.3
diffutils@3.10       intel-tbb@2022.0.0  libsigsegv@2.14       nghttp2@1.65.0        pkgconf@2.3.0     zlib-ng@2.2.3
ecbuild@3.7.2        krb5@1.21.3         libtool@2.4.7         numactl@2.0.18        pmix@5.0.5        zstd@1.5.6
esmf@8.8.0           libaec@1.0.6        libunistring@1.2      openblas@0.3.29       py-cython@3.0.11
expat@2.7.0          libbsd@0.12.2       libxcrypt@4.4.38      openmpi@5.0.6         python@3.13.2

-- linux-ubuntu24.04-x86_64_v3 / no compiler --------------------
autoconf@2.72                       gftl@1.15.2          py-colorama@0.4.6    py-pip@24.3.1            py-trove-classifiers@2023.8.7
ca-certificates-mozilla@2025-02-25  gftl-shared@1.10.0   py-editables@0.5     py-pluggy@1.5.0          py-wheel@0.45.1
compiler-wrapper@1.0                glibc@2.39           py-flit-core@3.10.1  py-pyyaml@6.0.2          python-venv@1.0
fargparse@1.9.0                     mepo@2.3.0           py-hatchling@1.25.0  py-ruamel-yaml@0.17.32   util-macros@1.20.1
gcc@14.2.0                          pflogger@1.16.1      py-packaging@24.2    py-setuptools@76.0.0     yafyaml@1.5.1
gcc-runtime@14.2.0                  py-calver@2022.6.26  py-pathspec@0.11.1   py-setuptools-scm@8.2.0
==> 93 installed packages

I'm not sure why some are in "no compiler". I'll ask on Spack Slack.

Clone this wiki locally