Merge pull request #364 from ESCOMP/noresm

Bring in recent changes from NorESM

Changes here include:

`datm:`
- correct modes are now used for specific RYF JRA modes like `JRA-RYF9091`, rather than using the generic `CORE_IAF_JRA ` mode

`docn:`
- introduce a new mode for docn to read in multiple ocean levels to send to CISM
- introduce a new mode for docn to have both SST forcing for CAM and multiple ocean level temperature and salinity to send to CISM.

`dlnd:`
- refactor lnd_comp_nuopc to allow more than one datamode. The original datamode is now a new file dlnd_datamode_glc_forcing_mod.F90.
- add capability in CDEPS to have  lnd2rof to have ungridded dimensions that are set flexibly. This is needed in NorESM to permit non-water tracers to be advected by MOSART.
- introduce a new data mode, dlnd_datamode_rof_forcing_mod.F90, that permits the ingestion of forcing for a prognostic river model. This capability is in NorESMhub/CDEPS now but not in  ESCOMP/CDEPS.
- For glc forcings, send missing values instead of 0s over ocean


Are changes expected to change answers (bfb, different to roundoff, more substantial): YES, including some more substantial changes:
- DATM now uses correct modes for specific RYF JRA modes like `JRA-RYF9091`, rather than using the generic `CORE_IAF_JRA ` mode
- DLND now sends missing values instead of 0s over ocean (creates FILLDIFFs in coupler history files in T compsets)

Testing performed (e.g. aux_cdeps, CESM prealpha, etc):
- CESM prealpha

Author: billsacks

.github/workflows/extbuild.yml

@@ -19,8 +19,8 @@ jobs:
       CPPFLAGS: "-I/usr/include -I/usr/local/include "
       LDFLAGS: "-L/usr/lib/x86_64-linux-gnu "
       # Versions of all dependencies can be updated here - these match tag names in the github repo
-      ESMF_VERSION: v8.6.1
-      ParallelIO_VERSION: pio2_6_2
+      ESMF_VERSION: v8.9.0
+      ParallelIO_VERSION: pio2_6_6
     steps:
       - id: checkout-CDEPS
         uses: actions/checkout@v4
@@ -39,8 +39,8 @@ jobs:
         id: cache-PARALLELIO
         uses: actions/cache@v4
         with:
-          path: ${GITHUB_WORKSPACE}/pio
-          key: ${{ runner.os }}-${{ env.ParallelIO_VERSION }}-parallelio2
+          path: /home/runner/work/CDEPS/CDEPS/pio
+          key: ${{ runner.os }}-${{ env.ParallelIO_VERSION }}-parallelio
       - name: Build ParallelIO
         if: steps.cache-PARALLELIO.outputs.cache-hit != 'true'
         uses: NCAR/ParallelIO/.github/actions/parallelio_cmake@9390e30e29d4ebbfbef0fc72162cacd9e8f25e4e
@@ -49,7 +49,7 @@ jobs:
           enable_fortran: True
           install_prefix: ${GITHUB_WORKSPACE}/pio
       - name: Install ESMF
-        uses: esmf-org/install-esmf-action@v1
+        uses: esmf-org/install-esmf-action@v1.0.2
         env:
           ESMF_COMPILER: gfortran
           ESMF_BOPT: g

cime_config/stream_cdeps.py

@@ -76,7 +76,8 @@ def create_stream_xml(
         data_list_file,
         user_mods_file,
         available_neon_data=None,
-        available_plumber_data=None
+        available_plumber_data=None,
+        lnd2rof_nonh2o_number=0,
     ):
         """
         Create the stream xml file and append the required stream input data to the input data list file
@@ -230,6 +231,7 @@ def create_stream_xml(
                         datavars = child.xml_element.text.strip()
                         datavars = self._resolve_values(case, datavars)
                         datavars = self._sub_glc_fields(datavars, case)
+                        datavars = self._sub_rof_fields(datavars, case, lnd2rof_nonh2o_number=lnd2rof_nonh2o_number)
                         datavars = self._add_xml_delimiter(datavars.split("\n"), "var")
                         if stream_vars[node_name]:
                             stream_vars[node_name] = (
@@ -520,11 +522,11 @@ def _sub_glc_fields(self, datavars, case):
 
         Returns a string.
 
-        Example: If `_sub_fields` is called with an array containing two
+        Example: If `_sub_glc_fields` is called with an array containing two
         elements, each of which contains two strings, and glc_nec=3:
              foo               bar
              s2x_Ss_tsrf%glc   tsrf%glc
-         then the returned array will be:
+        then the returned array will be:
              foo               bar
              s2x_Ss_tsrf00     tsrf00
              s2x_Ss_tsrf01     tsrf01
@@ -547,6 +549,56 @@ def _sub_glc_fields(self, datavars, case):
                 new_lines.append(line)
         return "\n".join(new_lines)
 
+    def _sub_rof_fields(self, datavars, case, lnd2rof_nonh2o_number=0):
+        """Substitute indicators with given values in a list of fields.
+        Replace any instance of the following substring indicators with the
+        appropriate values:
+            %lnd2rof_nonh2o_number = two-digit number from 0 through the number
+             of non-h2o tracers sent from the dlnd to mosart.
+
+        The difference between this function and `_sub_paths` is that this
+        function is intended to be used for variable names (especially from the
+        `strm_datvar` defaults), whereas `_sub_paths` is intended for use on
+        input data file paths.
+
+        Returns a string.
+
+        Example: If `_sub_rof_fields` is called with an array containing only one
+        element, and lnd2rof_nonh2o_number = 2
+             foo               bar
+	     lndImp_Flrl_rofsur_nonh2o%rof Flrl_rofsur_nonh2o%rof
+        then the returned array will be:
+             foo               bar
+	     lndImp_Flrl_rofsur_nonh2o Flrl_rofsur_nonh2o
+
+        Example: If `_sub_rof_fields` is called with an array containing two
+        elements, each of which contains two strings, and lnd2rof_nonh2o_number = 2
+             foo               bar
+	     lndImp_Flrl_rofsur_nonh2o%rof Flrl_rofsur_nonh2o%rof
+        then the returned array will be:
+             foo               bar
+	     lndImp_Flrl_rofsur_nonh2o%01 Flrl_rofsur_nonh2o%01
+	     lndImp_Flrl_rofsur_nonh2o%02 Flrl_rofsur_nonh2o%02
+        """
+        lines = datavars.split("\n")
+        new_lines = []
+        for line in lines:
+            if not line:
+                continue
+            if "%rof" in line:
+                if lnd2rof_nonh2o_number == 0:
+                    lnd2rof_indices = []
+                else:
+                    lnd2rof_indices = range(1,lnd2rof_nonh2o_number + 1)
+                for i in lnd2rof_indices:
+                    if len(lnd2rof_indices) == 1:
+                        new_lines.append(line.replace("%rof", "".format(i)))
+                    else:
+                        new_lines.append(line.replace("%rof", "{:02d}".format(i)))
+            else:
+                new_lines.append(line)
+        return "\n".join(new_lines)
+
     @staticmethod
     def _days_in_month(month, year=1):
         """Number of days in the given month (specified as an int, 1-12).
@@ -664,8 +716,11 @@ def _sub_paths(
                             date_string = (year_format + "-{:02d}-{:02d}").format(
                                 adjusted_year, adjusted_month, adjusted_day
                             )
-                            new_line = line.replace(match.group(0), date_string)
-                            new_lines.append(new_line)
+                            new_file = line.replace(match.group(0), date_string)
+                            if os.path.exists(new_file):
+                                new_lines.append(new_file)
+                            else:
+                                print(f"   WARNING:not adding missing file {new_file}")
                 elif match.group("month"):
                     for month in range(1, 13):
                         date_string = (year_format + "-{:02d}").format(year, month)

datm/atm_comp_nuopc.F90

@@ -356,6 +356,10 @@ subroutine InitializeAdvertise(gcomp, importState, exportState, clock, rc)
     if ( trim(datamode) == 'CORE2_NYF'    .or. &
          trim(datamode) == 'CORE2_IAF'    .or. &
          trim(datamode) == 'CORE_IAF_JRA' .or. &
+         trim(datamode) == 'CORE_RYF6162_JRA' .or. &
+         trim(datamode) == 'CORE_RYF8485_JRA' .or. &
+         trim(datamode) == 'CORE_RYF9091_JRA' .or. &
+         trim(datamode) == 'CORE_RYF0304_JRA' .or. &
          trim(datamode) == 'CLMNCEP'      .or. &
          trim(datamode) == 'CPLHIST'      .or. &
          trim(datamode) == 'GEFS'         .or. &
@@ -372,7 +376,7 @@ subroutine InitializeAdvertise(gcomp, importState, exportState, clock, rc)
        call datm_datamode_core2_advertise(exportState, fldsExport, flds_scalar_name, &
             flds_co2, flds_wiso, flds_presaero, flds_presndep, rc)
        if (ChkErr(rc,__LINE__,u_FILE_u)) return
-    case ('CORE_IAF_JRA')
+    case ('CORE_IAF_JRA', 'CORE_RYF6162_JRA', 'CORE_RYF8485_JRA', 'CORE_RYF9091_JRA', 'CORE_RYF0304_JRA')
        call datm_datamode_jra_advertise(exportState, fldsExport, flds_scalar_name, &
             flds_co2, flds_wiso, flds_presaero, flds_presndep, rc)
        if (ChkErr(rc,__LINE__,u_FILE_u)) return
@@ -621,7 +625,7 @@ subroutine datm_comp_run(gcomp, importState, exportState, target_ymd, target_tod
        case('CORE2_NYF','CORE2_IAF')
           call datm_datamode_core2_init_pointers(exportState, sdat, datamode, factorfn_mesh, factorfn_data, rc)
           if (ChkErr(rc,__LINE__,u_FILE_u)) return
-       case('CORE_IAF_JRA')
+       case ('CORE_IAF_JRA', 'CORE_RYF6162_JRA', 'CORE_RYF8485_JRA', 'CORE_RYF9091_JRA', 'CORE_RYF0304_JRA')
           call datm_datamode_jra_init_pointers(exportState, sdat, rc)
           if (ChkErr(rc,__LINE__,u_FILE_u)) return
        case('CLMNCEP')
@@ -646,7 +650,7 @@ subroutine datm_comp_run(gcomp, importState, exportState, target_ymd, target_tod
           call shr_get_rpointer_name(gcomp, 'atm', target_ymd, target_tod, rpfile, 'read', rc)
           if (ChkErr(rc,__LINE__,u_FILE_u)) return
           select case (trim(datamode))
-          case('CORE2_NYF','CORE2_IAF','CORE_IAF_JRA','CLMNCEP','CPLHIST','ERA5','GEFS','SIMPLE')
+          case('CORE2_NYF','CORE2_IAF','CORE_IAF_JRA','CORE_RYF6162_JRA','CORE_RYF8485_JRA','CORE_RYF9091_JRA','CORE_RYF0304_JRA','CLMNCEP','CPLHIST','ERA5','GEFS','SIMPLE')
              call dshr_restart_read(restfilm, rpfile, logunit, my_task, mpicom, sdat, rc)
              if (ChkErr(rc,__LINE__,u_FILE_u)) return
           case default
@@ -686,7 +690,7 @@ subroutine datm_comp_run(gcomp, importState, exportState, target_ymd, target_tod
        call datm_datamode_core2_advance(datamode, target_ymd, target_tod, target_mon, &
             sdat%model_calendar, factorfn_mesh, rc)
        if (ChkErr(rc,__LINE__,u_FILE_u)) return
-    case('CORE_IAF_JRA')
+    case('CORE_IAF_JRA','CORE_RYF6162_JRA','CORE_RYF8485_JRA','CORE_RYF9091_JRA','CORE_RYF0304_JRA')
        call datm_datamode_jra_advance(exportstate, target_ymd, target_tod, sdat%model_calendar, rc)
        if (ChkErr(rc,__LINE__,u_FILE_u)) return
     case('CLMNCEP')
@@ -714,7 +718,7 @@ subroutine datm_comp_run(gcomp, importState, exportState, target_ymd, target_tod
        call shr_get_rpointer_name(gcomp, 'atm', target_ymd, target_tod, rpfile, 'write', rc)
        if (ChkErr(rc,__LINE__,u_FILE_u)) return
        select case (trim(datamode))
-       case('CORE2_NYF','CORE2_IAF','CORE_IAF_JRA','CLMNCEP','CPLHIST','ERA5','GEFS','SIMPLE')
+       case('CORE2_NYF','CORE2_IAF','CORE_IAF_JRA','CORE_RYF6162_JRA','CORE_RYF8485_JRA','CORE_RYF9091_JRA','CORE_RYF0304_JRA','CLMNCEP','CPLHIST','ERA5','GEFS','SIMPLE')
           call dshr_restart_write(rpfile, case_name, 'datm', inst_suffix, target_ymd, target_tod, logunit, &
                my_task, sdat, rc)
           if (ChkErr(rc,__LINE__,u_FILE_u)) return

datm/cime_config/config_component.xml

@@ -10,7 +10,7 @@
        This file may have atm desc entries.
   -->
   <description modifier_mode="1">
-    <desc atm="DATM[%QIA][%WISOQIA][%CRUJRA2024][%CRUv7][%GSWP3v1][%MOSARTTEST][%NLDAS2][%CPLHIST][%1PT][%NYF][%IAF][%JRA][%JRA-1p4-2018][%JRA-1p5-2023][%JRA-RYF8485][%JRA-RYF9091][%JRA-RYF0304][%SIMPLE]"> Data driven ATM </desc>
+    <desc atm="DATM[%QIA][%WISOQIA][%CRUJRA2024][%CRUv7][%GSWP3v1][%MOSARTTEST][%NLDAS2][%CPLHIST][%1PT][%NYF][%IAF][%JRA][%JRA-1p4-2018][%JRA-1p5-2023][%JRA-RYF6162][%JRA-RYF8485][%JRA-RYF9091][%JRA-RYF0304][%SIMPLE]"> Data driven ATM </desc>
     <desc option="QIA"> QIAN data set </desc>
     <desc option="WISOQIA">QIAN with water isotopes</desc>
     <desc option="CRUJRA2024"> CRUJRA 2024 data set </desc>
@@ -24,6 +24,7 @@
     <desc option="JRA">interannual JRA55 forcing</desc>
     <desc option="JRA-1p4-2018">interannual JRA55 forcing, v1.4, through 2018</desc>
     <desc option="JRA-1p5-2023">interannual JRA55 forcing, v1.5, through 2023</desc>
+    <desc option="JRA-RYF6162"> JRA55 Repeat Year Forcing v1.3 1961-1962</desc>
     <desc option="JRA-RYF8485"> JRA55 Repeat Year Forcing v1.3 1984-1985</desc>
     <desc option="JRA-RYF9091"> JRA55 Repeat Year Forcing v1.3 1990-1991</desc>
     <desc option="JRA-RYF0304"> JRA55 Repeat Year Forcing v1.3 2003-2004</desc>
@@ -42,7 +43,7 @@
 
   <entry id="DATM_MODE">
     <type>char</type>
-    <valid_values>CORE2_NYF,CORE2_IAF,CLM_QIAN,CLM_QIAN_WISO,1PT,CLMCRUJRA2024,CLMCRUNCEPv7,CLMGSWP3v1,CLMNLDAS2,CPLHIST,CORE_IAF_JRA,CORE_IAF_JRA_1p4_2018,CORE_IAF_JRA_1p5_2023,ERA5,SIMPLE</valid_values>
+    <valid_values>CORE2_NYF,CORE2_IAF,CLM_QIAN,CLM_QIAN_WISO,1PT,CLMCRUJRA2024,CLMCRUNCEPv7,CLMGSWP3v1,CLMNLDAS2,CPLHIST,CORE_IAF_JRA,CORE_IAF_JRA_1p4_2018,CORE_IAF_JRA_1p5_2023,CORE_RYF6162_JRA,CORE_RYF8485_JRA,CORE_RYF9091_JRA,CORE_RYF0304_JRA,ERA5,SIMPLE</valid_values>
     <default_value>CORE2_NYF</default_value>
     <group>run_component_datm</group>
     <file>env_run.xml</file>
@@ -56,6 +57,10 @@
       <value compset="DATM%JRA">CORE_IAF_JRA</value>
       <value compset="DATM%JRA-1p4-2018">CORE_IAF_JRA_1p4_2018</value>
       <value compset="DATM%JRA-1p5-2023">CORE_IAF_JRA_1p5_2023</value>
+      <value compset="DATM%JRA-RYF6162">CORE_RYF6162_JRA</value>
+      <value compset="DATM%JRA-RYF8485">CORE_RYF8485_JRA</value>
+      <value compset="DATM%JRA-RYF9091">CORE_RYF9091_JRA</value>
+      <value compset="DATM%JRA-RYF0304">CORE_RYF0304_JRA</value>
       <value compset="DATM%QIA">CLM_QIAN</value>
       <value compset="DATM%WISOQIA">CLM_QIAN_WISO</value>
       <value compset="DATM%CRUJRA2024">CLMCRUJRA2024</value>

datm/cime_config/namelist_definition_datm.xml

@@ -61,6 +61,9 @@
       <value datm_mode="CORE_IAF_JRA_1p5_2023" >
         CORE_IAF_JRA_1p5_2023.GCGCS.PREC,CORE_IAF_JRA_1p5_2023.GISS.LWDN,CORE_IAF_JRA_1p5_2023.GISS.SWDN,CORE_IAF_JRA_1p5_2023.NCEP.Q_10,CORE_IAF_JRA_1p5_2023.NCEP.SLP_,CORE_IAF_JRA_1p5_2023.NCEP.T_10,CORE_IAF_JRA_1p5_2023.NCEP.U_10,CORE_IAF_JRA_1p5_2023.NCEP.V_10
       </value>
+      <value datm_mode="CORE_RYF6162_JRA">
+        CORE_RYF6162_JRA.GISS.LWDN,CORE_RYF6162_JRA.GISS.SWDN,CORE_RYF6162_JRA.GCGCS,CORE_RYF6162_JRA.NCEP
+      </value>
       <value datm_mode="CORE_RYF8485_JRA">
         CORE_RYF8485_JRA.GISS.LWDN,CORE_RYF8485_JRA.GISS.SWDN,CORE_RYF8485_JRA.GCGCS,CORE_RYF8485_JRA.NCEP
       </value>
@@ -90,7 +93,7 @@
     <type>char</type>
     <category>datm</category>
     <group>datm_nml</group>
-    <valid_values>CLMNCEP,CORE2_NYF,CORE2_IAF,CORE_IAF_JRA,ERA5,SIMPLE,CPLHIST,1PT</valid_values>
+    <valid_values>CLMNCEP,CORE2_NYF,CORE2_IAF,CORE_IAF_JRA,CORE_RYF6162_JRA,CORE_RYF8485_JRA,CORE_RYF9091_JRA,CORE_RYF0304_JRA,ERA5,SIMPLE,CPLHIST,1PT</valid_values>
     <desc>
       general method that operates on the data.
       ----datamode = "CPLHIST"----
@@ -115,6 +118,14 @@
       Clm Dyn doi 10.1007/s00382-008-0441-3.
       ----datamode = "CORE_IAF_JRA"----
       JRA55 intra-annual year forcing
+      ----datamode = "CORE_RYF6162_JRA"----
+      JRA55 Repeat Year Forcing 1961-1962
+      ----datamode = "CORE_RYF8485_JRA"----
+      JRA55 Repeat Year Forcing 1984-1985
+      ----datamode = "CORE_RYF9091_JRA"----
+      JRA55 Repeat Year Forcing 1990-1991
+      ----datamode = "CORE_RYF0304_JRA"----
+      JRA55 Repeat Year Forcing 2003-2004
       ----datamode = "CLMNCEP"----
       In conjunction with NCEP climatological atmosphere data, provides the
       atmosphere forcing favored by the Land Model Working Group when