Add uwtools for global-workflow-env, build openssl in the stack

[rickgrubin-noaa]

Description

Add uwtools for global-workflow-env. Note: newer versions of uwtools will be available when [email protected] is available; see: Update uwtools versions and add py-python-dateutils dependency #1

Remove system openssl from configs/sites/tier1/orion/packages.yaml, as py-crytpography will not build against it; this breaks CI/CD weekly build / test automation for spack-stack@develop on orion / oneapi.

Dependencies

None

Issues addressed

None

Applications affected

N/A as applications are built against stack release versions, not branch develop.

Systems affected

orion

Testing

• CI: Note whether the automatic tests (GitHub actions tests that run automatically for every commit) pass or not
  • GitHub actions CI tests pass
  • GitHub actions CI tests do not pass (provide explanation)
  • GitHub actions CI tests skipped (provide explanation if necessary)
• New tests added: List and describe any new tests added to GitHub actions
  • ...
• Additional testing: Add information on any additional tests conducted
  • verify py-crytpography successfully loads / runs

$ export MODULEPATH=/apps/spack-managed-x86_64_v3-v1.0/modulefiles/Core:$MODULEPATH
$ module use /work2/noaa/epic/rgrubin/envs/orion/ue-oneapi-2024.2.1/install/modulefiles/Core
$ module use /work2/noaa/epic/rgrubin/envs/orion/ue-oneapi-2024.2.1/install/modulefiles/gcc/12.2.0
$ module use /work2/noaa/epic/rgrubin/envs/orion/ue-oneapi-2024.2.1/install/modulefiles/intel-oneapi-mpi/2021.13-ofiyixw/gcc/12.2.0
$ module load stack-oneapi stack-intel-oneapi-mpi stack-python

$ module load py-cryptography
$ python3 -c "import cryptography; print(cryptography.__version__)"
42.0.8

$ module load uwtools
$ uw --version
uw version 2.7.2 build 0

Checklist

• This PR addresses one issue/problem/enhancement or has a very good reason for not doing so.
• These changes have been tested on the affected systems and applications.
• All dependency PRs/issues have been resolved and this PR can be merged.
• All necessary updates to the documentation on readthedocs are included in this PR
  • For site config updates, check in particular doc/source/PreConfiguredSites.rst and doc/source/MaintainersSection.rst
• All necessary updates to the spack-stack wiki will be made when this PR is merged

[RatkoVasic-NOAA]

LGTM

[climbfuji]

Error in CI:

==> Error: Version requirement 2.7.2 on uwtools for uwtools cannot match any known version from package.py or externals

[rickgrubin-tomorrow]

Error in CI:

==> Error: Version requirement 2.7.2 on uwtools for uwtools cannot match any known version from package.py or externals

uwtools/package.py

version("2.7.2", sha256="56816d543664792258bfa7dfb7e4cc66f794959dc92dc3710021f40a2b8571a4")

spack-stack@develop isn't using spack@spack-stack-dev ?

spack-stack/.gitmodules

[submodule "spack"]
  path = spack
  url = https://github.com/jcsda/spack
  branch = spack-stack-dev

Apparently not, somewhat to my surprise, but rather spack @ 43381fe, and unfortunately Update uwtools #557 won't be merged because of the delay in having it reviewed ~3 weeks ago.

I'll change v2.7.2 to v2.7.1 for now, and it will all go away when the move to spack-packages is made.

[climbfuji]

We can also merge the 2.7.2 PR if it's needed, since there is a corresponding PR in the new spack-packages repo. I didn't know this was urgent.

[climbfuji]

I am confused, is this just a matter of forwarding the spack submodule ptr? You can easily do this here.

[rickgrubin-tomorrow]

We can also merge the 2.7.2 PR if it's needed, since there is a corresponding PR in the new spack-packages repo. I didn't know this was urgent.

It's not urgent; it does allow the folks who are relying on global-workflow-env to integrate uwtools instead of having it outside the environment they're using. As they've been getting along to date without this change, it can wait.

CI/CD failures are fixed by the change to orion's common/packages.yaml, so not directly related.

CI/CD does test spack-stack@develop and when that's cloned:

Submodule path 'spack': checked out '43381fe761f7280ac91fbdda462f86a062979843'

This is a chicken-and-egg problem, and as I agree with your previous statement about not merging Update uwtools #557.

If you're OK with the following:

change this PR to remove the uwtools changes, and simply fix openssl / py-cryptography so as to fix CI/CD and have a positive future basis for when automation work becomes prioritized again

that's what I'll do.

[climbfuji]

Let's just merge the spack PR and the equivalent spack-packages PR now; then you can update the submodule pointer for spack here and we are all set?

[climbfuji]

@rickgrubin-noaa New hash for spack is 29265cf

[rickgrubin-noaa]

@rickgrubin-noaa New hash for spack is 29265cf

Thanks; that's concretized and building.

Have to leave [email protected] -- trying v2.8.2 triggers the dependency for [email protected]:, which in turn requires py-python-dateutil/package.py to need

version(
        "2.9.0.post0", sha256="37dd54208da7e1cd875388217d5e00ebd4179249f90fb72437e91a35459a0ad3"
    )

That's already in the spack-packages repo recipe, so should be ready for [email protected]

[rickgrubin-noaa]

@rickgrubin-noaa New hash for spack is 29265cf

spack-stack@uwtools-gw-ue with spack@29265cf successfully builds on orion -- merging here ought to fix CI, yes?

$ git branch
  develop
  release/1.9.0
* uwtools-gw-ue

$ git submodule status
 607d9b4633395b4be88d87a1df88b46302d48d0c doc/CMakeModules (v1.0.0-32-g607d9b4)
 29265cff4abb9a7f48e19165ee0bdaf6ceb8d317 spack (spack-stack-1.9.1-1650-g29265cff4a)