Update testing for configurations with active Greenland ice sheet #7892

stephenprice · 2025-11-19T00:14:42Z

Update existing testing support for IG and BG cases that uses 20km, uniform resolution Greenland mesh, to use newer variable resolution 4-to-40km mesh. All existing and new landice developer tests have been confirmed to pass. Changes to mali section of ./cime_config/config_archive.xml should now allow mali hist files to be compared against a baseline (prev. support used a hist.am file for this). A follow-up PR will fully remove support for Greenland 20km mesh (draft PR 7898).

[NML]
[BFB] for all current tests, but does change landice tests

Swap out low-res 20km mesh with var. res. 4-to-40km mesh; generalize shell commands and namelist file changes (not specific to a single mesh resolution); exclude regional stats generation for MALI (which prohibits comparison between multiple MALI hist. files); add ERS test for BG config.

Add reg exp matching to cime archiving script to pull in wider range of MPAS hist files in testing; Correct pe layouts for Chrys and PM testing.

Updated pes support for testing on Chrys and PM-cpu; reduce no. of days for ERS tests; remove no longer needed test mod support (due to default timestepping setting changes); rename test mod dir names; update default pe layouts for IG cases

matthewhoffman

@stephenprice , thanks for tackling this longstanding cleanup work. I have a handful of questions/comments. Some are in files I'm not super familiar with, so in some cases I'm guessing a little bit. I've reviewed by inspection but haven't tried running any tests with the branch.

cime_config/allactive/config_pesall.xml

cime_config/testmods_dirs/config_pes_tests.xml

matthewhoffman · 2025-11-26T13:57:10Z

cime_config/config_archive.xml

+    <hist_file_ext_regex>\w+</hist_file_ext_regex>
+    <hist_file_ext_regex>\w+\.\w+</hist_file_ext_regex>


I'm not an expert in regex or familiar with how this file works, but I wonder if there is a more specific search we can employ or if both versions are necessary. Maybe not. @jonbob , do you have experience with this?

What I was told a while back was that this should work but that there is some CIME bug preventing it from fully working. For the moment, it at least works to generate hist file baselines for MALI, which can then be later compared against newly generated baselines. It still does not work for including MALI hist files in a restart comparison unfortunately.

@stephenprice -- it doesn't include MALI hist files in an ERS test? Or not MALI restart files?

There are a few diff. issues here. First, those lines of code are supposed to allow both mali .hist and .am files to be compared against a set of baseline files. As discussed here though (in section titled "Notes from Jason Boutte..."). In my experimenting however, this does not work and only the .am files get compared. This is why I've set the user_nl_mali to exclude .am files from outputs for these tests. But I figure it does not hurt to include that functionality for if / when this problem gets attention and gets fixed (currently, we would at least be comparing newly generated mali .hist files to baselines during SMS testing).

It's possible that this is by design, but from what I can tell, in my ERS tests nothing gets compared aside from coupler history files. It would be nice if mpas component hist files were also compared. I think this is considered not necessary because the mpas components generate and pass fields that populate the cpl hist files. But right now, we pass very little to the cpl, and for short runs, most of those fields are going to contain all zeroes, which does not feel like a very rigorous restart test (for mali at least). It's unclear to me why hist files from other components, including mpas components, are not included in the comparison for ERS tests. Should they be?

components/elm/cime_config/config_pes.xml

matthewhoffman · 2025-11-26T14:02:04Z

components/elm/cime_config/config_pes.xml

+          <ntasks_rof>64</ntasks_rof>
+          <ntasks_ice>64</ntasks_ice>
+          <ntasks_ocn>64</ntasks_ocn>
+          <ntasks_glc>960</ntasks_glc>


same as previous comment about danger of having one component with so many more tasks than any other.

See response to above comment.

I'll revisit this test and see how it performs w/ a smaller set of pes (e.g., 256 or 512).

This has been updated for both pm-cpu and Chrys layouts so that all components use 256 pes (4 nodes on Chrys, 2 nodes on pm-cpu). PACE plots (for Chrys and pm-cpu) confirm that this is reasonably well balanced use for all components.

components/elm/cime_config/config_pes.xml

components/mpas-albany-landice/cime_config/testdefs/testmods_dirs/mali/gis/user_nl_mali

…tions Remove redundant 'gis' testmods dir under MALI component subdir; change pe layouts for testing to use 512 vs 256 pes; update description of default pe layout for IG case tests; update 'extra snowlayers' test to use 4-to-40km Greenland grid rather than no-longer-supported 20km grid

matthewhoffman

I'm approving as Steve has addressed all my comments. @jonbob , there still is the thread about history file comparisons in tests that I'm not totally sure if we came to a full understanding of. But the current changes seem to yield the desired behavior, so I'm happy to sign off on it despite not understanding the issues. I'm flagging that in case you want to look at that element more closely.

stephenprice · 2025-12-17T21:42:36Z

@jonbob -- I also confirmed that, when running the full landice test suite while using '--pesfile' to point to the correct 'config_pes_tests.xml' file, the behavior is as expected w.r.t. nodes used for the different tests in the suite (confirmed only on Chrys for now since pm-cpu is down today).

trhille

landice_developer tests pass on Chrysalis.

trhille · 2025-12-12T21:41:34Z

components/elm/cime_config/config_pes.xml

+  <grid name="a%ne30np4.pg2_l%r05_oi%null_r%r05_g%mpas.gis4to40km_w%null_z%null_m%IcoswISC30E3r5">
+    <mach name="pm-cpu|muller-cpu|alvarez-cpu">
+      <pes compset="any" pesize="any">
+        <comment>pm-cpu: IG-case testing config. using 4-to-40km (mde res) GIS init. cond.</comment>


Suggested change

<comment>pm-cpu: IG-case testing config. using 4-to-40km (mde res) GIS init. cond.</comment>

<comment>pm-cpu: IG-case testing config. using 4-to-40km (med res) GIS init. cond.</comment>

(I think this is what you meant)

Ack. Good catch. I'll fix that typo and push it up.

rljacob · 2026-01-08T17:50:22Z

@jonbob is this ready?

jonbob · 2026-01-08T17:58:57Z

@rljacob -- I think so. I'll test it today and merge

…to40km-grid' into next (PR #7892) Update testing for configurations with active Greenland ice sheet Update existing testing support for IG and BG cases that uses 20km, uniform resolution Greenland mesh, to use newer variable resolution 4-to-40km mesh. All existing and new landice developer tests have been confirmed to pass. Changes to mali section of cime_config/config_archive.xml should now allow mali hist files to be compared against a baseline (prev. support used a hist.am file for this). A follow-up PR will fully remove support for Greenland 20km mesh [BFB] for all current tests, but does change landice tests

jonbob · 2026-01-08T20:33:30Z

passes:

e3sm_landice_developer on chrysalis with expected NML DIFFs and DIFs related to missing baselines
SMS_D_Ld1.ne30pg2_r05_IcoswISC30E3r5.WCYCL1850.chrysalis_intel.allactive-wcprod

merged to next

jonbob · 2026-01-09T19:54:28Z

merged to master and expected NML and regular DIFFs (due to new baselines) blessed, except for pm-cpu which has not reported

jonbob · 2026-01-12T16:42:14Z

expected NML and regular DIFFs (due to new baselines) also blessed on pm-cpu

stephenprice added 3 commits November 13, 2025 17:17

More updates to testing support for Greenland configurations

77970bd

Add reg exp matching to cime archiving script to pull in wider range of MPAS hist files in testing; Correct pe layouts for Chrys and PM testing.

stephenprice requested review from matthewhoffman and trhille November 19, 2025 00:14

stephenprice assigned jonbob Nov 19, 2025

stephenprice added MPAS-albany-landice Concerning the MPAS-Albany land ice model Testing Anything related to unit/system tests Chrysalis pm-cpu Perlmutter at NERSC (CPU-only nodes) labels Nov 19, 2025

stephenprice mentioned this pull request Nov 20, 2025

Remove no-longer-needed support for 20km Greenland ice sheet mesh #7898

Open

stephenprice added the new baselines needed label Nov 20, 2025

matthewhoffman requested changes Nov 26, 2025

View reviewed changes

stephenprice added 3 commits December 2, 2025 17:12

Update default PE layout for IG case using GIS 1to10km mesh

43965dd

Minor corrections and clean-up of default pe setting files.

fc4ecd8

matthewhoffman approved these changes Dec 11, 2025

View reviewed changes

trhille approved these changes Dec 18, 2025

View reviewed changes

stephenprice mentioned this pull request Dec 18, 2025

Improve MALI performance for configurations with dynamic Greenland ice sheet #7926

Merged

Fix typo in PE layout description.

6b2926b

jonbob added the NML label Jan 8, 2026

jonbob merged commit 7ebddb9 into E3SM-Project:master Jan 9, 2026
5 of 6 checks passed

		<hist_file_ext_regex>\w+</hist_file_ext_regex>
		<hist_file_ext_regex>\w+\.\w+</hist_file_ext_regex>

	<comment>pm-cpu: IG-case testing config. using 4-to-40km (mde res) GIS init. cond.</comment>
	<comment>pm-cpu: IG-case testing config. using 4-to-40km (med res) GIS init. cond.</comment>

Update testing for configurations with active Greenland ice sheet #7892

Update testing for configurations with active Greenland ice sheet #7892

Uh oh!

Conversation

stephenprice commented Nov 19, 2025 • edited by jonbob Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

matthewhoffman left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

matthewhoffman left a comment

Choose a reason for hiding this comment

Uh oh!

stephenprice commented Dec 17, 2025

Uh oh!

trhille left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rljacob commented Jan 8, 2026

Uh oh!

jonbob commented Jan 8, 2026

Uh oh!

jonbob commented Jan 8, 2026

Uh oh!

Uh oh!

jonbob commented Jan 9, 2026

Uh oh!

jonbob commented Jan 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

stephenprice commented Nov 19, 2025 •

edited by jonbob

Loading

jonbob commented Jan 12, 2026 •

edited

Loading