Skip to content

Latest commit

 

History

History
81 lines (60 loc) · 8.04 KB

create_release_spack.md

File metadata and controls

81 lines (60 loc) · 8.04 KB

Making a new stable DAQ release

Overview

Making a new stable DAQ release consists of:

  1. Preparations before performing each candidate/stable release build, detailed below
  2. Build/deploy candidate releases during the testing period of a release cycle
  3. One last build of the release at the end of the testing period

A patch release is built in the same way as a non-patch release, except it doesn't involve candidate releases.

As of this writing (Feb-18-2026) the only software environment we release is the classic fddaq environment; for this reason it's the one that's referred to here, though the workflow will be essentially the same for any other environment (nddaq, etc.). Note these instructions will make more sense if you read up on our software environment model.

Prepare for the new release

Create release configurations

The release configuration package versions are defined by configs/coredaq/fddaq-coredaq-vX.Y.Z/release.yaml (core release, fddaq variant) and configs/fddaq/fddaq-vX.Y.Z/release.yaml (fddaq specific packages). To prepare the release configuration for a new release, it is best to start from (copy) the configuration of the develop release (configs/coredaq/fddaq-develop/release.yaml and configs/fddaq/fddaq-develop/release.yaml.

The release YAML file contains sections meant to define the versions of packages. In general, the externals, devtools, systems and pymodules sections will already have versions defined since you copied the YAML files from the develop release. While the develop release builds DUNE DAQ packages from the head of their develop branches, it uses versioned non-DUNE DAQ packages (e.g., Boost). However, you'll want to add the correct versions for the DUNE DAQ packages, e.g. edit

        - name: fddetdataformats
          version: "develop"
          commit: null

to

        - name: fddetdataformats
          version: v1.0.2
          commit: null

...where the commit field will be automatically calculated by the make-release-repo.py script run during the GitHub Actions which build releases.

In addition to the release.yaml file, there also needs to be a dbt-build-order.cmake file in the configs/fddaq/fddaq-vX.Y.Z/ directory you created. Copy it from the corresponding fddaq-develop/nddaq-develop subdirectory. Update this file when there's a package being added/removed, or if the dependency tree changed and a package needs to be built earlier (though generally, packages would be added removed outside the release period).

Checks before doing test builds

It's worth doing a couple of checks before starting any test builds. These checks include:

  • Check version tags match with the version numbers listed in CMakeLists.txt; The script scripts/checkout-daq-package.py in this repo can help here.

python3 scripts/checkout-daq-package.py -i <path-to-release-config-yaml> -a -c -o $( mktemp -d )

will checkout all the DAQ packages used in the release into a randomly-named directory and verify if the version tags match what's in CMakeLists.txt

  • (Optional) Check if developers got their dependencies in CMakeLists.txt to match those in cmake/<pkgname>Config.cmake.in files; see this section of the daq-cmake instructions for more

Building candidate releases

  • Once the release configuration is ready, one can start the CI build for candidate releases. Go to the "Actions" tab of daq-release repo on GitHub. From the list of workflows on the left, select "Alma9 build v5 candidate release", then click the "run workflow" button. A drop-down menu will show up. Put in the version of the core release in the vX.Y.Z format, the version of the fddaq release, and the candidate release number (start with 1, count up with later candidate releases). Click "Run workflow" to start the build.
  • Once the build is completed successfully, verify if the same version tags shown in the GitHub Action log match those in the tag collector spreadsheet
  • To publish the candidate release to cvmfs:
    • Log in to oasiscfs05.fnal.gov as cvmfsdunedaqdev
    • Get the publish_release_to_cvmfs.sh script (git clone this repo or use curl, e.g.)
    • Run the script without arguments for instructions; in a nutshell, it will publish the most recent release of a given specification (e.g., the most recent Alma9 stable build)
    • Run it with the desired specifications (e.g. publish_release_to_cvmfs.sh candidate fddaq alma9)
    • After running the script, the release will take ~20 minutes before it appears on cvmfs
  • After the candidate release is deployed and available on cvmfs, do the following simple tests:
    • Set up a work area based on the candidate release
    • If it's a far detector release, take the "Nightly v5 integration tests" Workflow and run it with the optional candidate release tag field filled in with the name of the candidate release. Performance should be what's expected and at the very least the minimal_system_quick_test should pass.
    • The above tests should be run on at least one NP04 DAQ server, and one Fermilab server

Building the stable release

  • The release will be cut at the end of the testing period. The build of the final stable release can be done in a similar way as the candidate releases. Choose "Build stable release" in the workflows list, and trigger the build by specifying release name used in configs and the number (starts from 1, increment it if second deployment to cvmfs is needed).
  • Deploying the stable release to cvmfs is the same as for a candidate release except you want to log in to oasiscfs05.fnal.gov as cvmfsdunedaq instead of cvmfsdunedaqdev and of course you'll want to pass stable rather than candidate to the publishing script
  • Do similar tests as shown in the section above for candidate releases, though fill in the "stable release tag" field of the integration test workflow rather than the "candidate release tag" field
  • If there is a new version of daq-buildtools for the release, it will need to be deployed to cvmfs too. Otherwise, creating a symbolic link in cvmfs to the latest tagged version will be sufficient, e.g. that setup_dbt fddaq-vX.Y.Z would give you the daq-buildtools used during the release period of fddaq-vX.Y.Z. How to do this is described in the documentation on cvmfs.
  • After the stable release is rolled out, there will be remaining prep release and patch branches used in the production of the release. The software coordination team and the release coordinator should get in touch to establish if anything should be kept out of the merge to develop. The software coordination team will do the merge across all relevant repositories. Developers should handle any partial merge (cherry-pick).
  • Also make sure that if the version of daq-cmake and/or any Python packages was bumped for the release, that these version increments make it into the nightly for the relevant development line.
  • The last step of making a stable release is to create release tags for all packages used by the release. To do so, use the script scripts/create-release-tag.py:
    • make sure daq-release is tagged for the new release, and the version is updated in the release YAML file. It will be tagged by the create-release-tag.py script;
    • Make sure you're able to push changes to repos without needing to enter your password; gh auth login should set you up to be able to do this if you have a GitHub Personal Access Token with the appropriate permissions
    • scripts/create-release-tag.py -h to show the usage of the script;
    • scripts/create-release-tag.py -a -t <release-tag> -i <release YAML file> to tag all packages used by the release; do this both for the core release and the detector release, using the correct release tag for each of the two categories
    • -d to delete release tags if found, -f to recreate release tags.