Add landice gridcomp

[weiyuan-jiang]

Add landice grid comp to GEOSldas.

By default, the nc4-formatted tile file will be read, if it is available in the bcs, which results in non-0-diff changes compared to BASELINE simulations that used the txt-formatted nc4 file. The PR should be 0-diff if TILE_FILE_FORMAT is set to TXT in LDAS.rc. Added labels for 0-diff and non-0-diff to reflect this situation.

Successfully 0-diff tested on 16 May 2025 by @gmao-rreichle
(For tests with aggressive builds, there are roundoff differences for cdcr1 in the catparam files.)

Successfully verified "land+landice" and "landice only" functionality on 19 May 2025 by @gmao-qliu.

Dependencies:
GEOS-ESM/GEOS_Util#137 (for remapping; merged on 15 May 2025; in release v2.1.8)
GEOS-ESM/GEOSgcm_GridComp#1094 (nc4-formatted tile file; merged on 15 May 2025; in develop and release v2.7.4)

[weiyuan-jiang]

Now this branch is debuggable and we should focus on the import variables. The crash happens here:

forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
GEOSldas.x 000000000086283A Unknown Unknown Unknown
libpthread-2.22.s 00002AB4E6578CE0 Unknown Unknown Unknown
GEOSldas.x 00000000007B56B7 geos_utilsmod_mp_ 555 GEOS_Utilities.F90
GEOSldas.x 00000000007B53BE geos_utilsmod_mp_ 612 GEOS_Utilities.F90
GEOSldas.x 000000000047FCF2 geos_landicegridc 3169 GEOS_LandIceGridComp.F90
GEOSldas.x 000000000046C693 geos_landicegridc 2201 GEOS_LandIceGridComp.F90

[gmao-rreichle]

Now this branch is debuggable and we should focus on the import variables. The crash happens here:

forrtl: severe (174): SIGSEGV, segmentation fault

Line 3169 of GEOS_LandiceGridComp is:

DQS = GEOS_QSAT(TS(:,N), PS, PASCALS=.TRUE.,RAMP=0.0) - QS(:,N)

This calculation concerns the specific humidity at the surface. Either the restarts didn't get processed right or - more likely - the connection to the met forcing isn't right. Ideally, we could use the debugger to run it to the crash and then view the values of the variables. If the debugger doesn't work, we'd have to add print statements.

[weiyuan-jiang]

This branch should work with this branch. It runs now but please check the variables that are recalculated when it is offline. @lcandre2 @gmao-rreichle

[gmao-rreichle]

This branch should work with this branch. It runs now ...

Thanks, @weiyuan-jiang! I added the branch name into components.yaml for now. @lcandre2, when you get a chance, please give the branch a try.

[weiyuan-jiang]

This branch should work with this branch. It runs now ...

Thanks, @weiyuan-jiang! I added the branch name into components.yaml for now. @lcandre2, when you get a chance, please give the branch a try.

@lcandre2 To try this branch, you need to setup it up manually. 1) generate a compatible landice restart file. 2) Copy the landice restart file to where catchment resides and make a link to it at input/restart 4) add a line LANDICE_INTERNAL_RESTART_FILE: ../input/restart/landice_internal_rst
to LDAS.rc

Eventually, we need to make some changes of ldas_setup to make it happen automatically.

[lcandre2]

Thanks @weiyuan-jiang! I'll try to get to this by Wednesday.

[lcandre2]

@weiyuan-jiang, I am past the HISTORY.rc problems, but now I am getting an error in the louissurface subroutine in the landice gridcomp (line 1995)... I think you have a run that works, could you point me to your path, so I can figure out what is going on?
eta: /discover/nobackup/lcandre2/outputs/ldas_landice/landice_test003

[weiyuan-jiang]

@lcandre2 Here is my run /discover/nobackup/wjiang/LDAS_exp/model/AddLandice
I will check your run too

[weiyuan-jiang]

After examining the codes, I think several import variables are missing:
THATM, AHATM, CTATM, CQATM
We need to know how to get them. @lcandre2 @gmao-rreichle

Never mind, those variables are not used in offline

[lcandre2]

Promised figures -- For this 1 month run (in January), the warm (0 *C) layer temperatures propagate down the snow column over the month, which does suggest a possible problem with the heat content, and of course something wrong occurring at the surface. More to come. Sorry for the inelegant plots.

[lcandre2]

I'm stumped @weiyuan-jiang, @gmao-rreichle. Issues and questions below.

1. Trying to run an experiment with no HISTORY.rc (or an experiment with no landice outputs) outputs to examine the restart files, the model generates a catch_internal_rst but crashes before the first landice_internal_rst output is generated.

This experiment is located here: /discover/nobackup/lcandre2/outputs/ldas_landice/landice_test013_nooutputs

2. Trying to run an experiments with instantaneous 1D *.bin outputs, the model does not create *.bin files, but it produces *nc4 files (completely unprompted by the HISTORY.rc file and there is no postprocessing assigned by the exe file). These nc4 files are 1D with the length and lat and lon of the land tiles and empty landice variables despite a ctrl file that has the appropriate number of landice tiles. And confusing to me, this experiment goes past the point where the previous experiment stops (to 3h instead of 1h), despite the same 1h job segments -- appropriately producing catch_internal_rst files but not landice_internal_rst files.

This experiment is located here: /discover/nobackup/lcandre2/outputs/ldas_landice/landice_test014_instant_binoutput/

So my questions are:

1. Something seems to be going wrong when writing the restart files (exactly what, I dont know), but why would the model run past the generation of landice_internal_rst in the second experiment?
2. Am I failing to do something in the HISTORY.rc file to produce *.bin outputs? Or is this something that originates from writing the outputs?

[weiyuan-jiang]

To generate restart files, it needs some changes:

1. in the LDAS.rc, add a line LANDICE_INTERNAL_CHECKPOINT_FILE: landice_internal_checkpoint
2. in lenkf.j, one block to deal with the check point is necessary right under {MODEL" block
   set rstf = 'landice'
   if (-f ${rstf}${ENSID}_internal_checkpoint ) then
   set tmp_file = $EXPDIR/output/$EXPDOMAIN/rs/$ENSDIR/Y${eYEAR}/M${eMON}/${EXPID}.${rstf}internal_rst.${eYEAR}${eMON}${eDAY}${eHour}${eMin}
   /bin/mv ${rstf}${ENSID}_internal_checkpoint $tmp_file
   /bin/rm -f $EXPDIR/input/restart/${rstf}${ENSID}_internal_rst
   /bin/ln -rs $tmp_file $EXPDIR/input/restart/${rstf}${ENSID}_internal_rst
   endif

My run is successful /discover/nobackup/wjiang/LDAS_exp/model/landice_test013_nooutputs

[biljanaorescanin]

GEOSldas regression tests all passed.