Skip to content
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
ce969e5
Merge pull request #1 from ESCOMP/cam_development
jtruesdal May 7, 2020
f95085f
Merge pull request #5 from ESCOMP/cam_development
jtruesdal Nov 30, 2020
17430cd
Merge pull request #7 from ESCOMP/cam_development
jtruesdal Dec 9, 2020
55cc749
create copy of Brians mpas_fixes branch on my fork for PR
jtruesdal Dec 9, 2020
bf44227
updates to mpas defaults, longer timestep 600->900s, scalar transport…
jtruesdal Dec 15, 2020
8725316
replace mpasa120 ncdata grid file with a notopo default, the topograp…
jtruesdal Dec 16, 2020
744cef6
Enable CAM-MPAS to build with weak scaling update in infrastructure
MiCurry Dec 17, 2020
0f0bc86
Fill out physics_column_t array for MPAS physics, dynamics coupling
MiCurry Dec 17, 2020
c7784ea
Update MPAS d_p_coupling to use new weak scaling changes
MiCurry Dec 21, 2020
4273a26
Update MPAS p_d_coupling with new weak scaling changes
MiCurry Dec 28, 2020
e94dd8d
Peters interface fix to correct a error when calculating CAM pressure…
jtruesdal Jan 11, 2021
19cd4c4
updates to add sponge diffusion to top 3 layers - similar to CAM-SE s…
jtruesdal Jan 13, 2021
6f82fbf
Merge pull request #9 from jtruesdal/cam_development
jtruesdal Jan 15, 2021
face455
Merge pull request #10 from ESCOMP/cam_development
jtruesdal Jan 15, 2021
dff8425
bring up to date with ESCOMP cam_development head
jtruesdal Jan 15, 2021
6249eb8
MPAS timer updates
jtruesdal Jan 15, 2021
8d9066e
Miles port of weak scaling mods to MPAS
jtruesdal Jan 15, 2021
9368dbc
mpas 120/480 defaults for AMIP runs
jtruesdal Feb 3, 2021
3a83b5f
bug fix for mpas amip commit
jtruesdal Feb 3, 2021
12b2b24
update externals for 120/480 defaults
jtruesdal Feb 11, 2021
38498c2
add pressure level fields to default history output, rayleigh default…
jtruesdal Mar 8, 2021
8a35da0
merge in changes for weak scaling
jtruesdal Mar 8, 2021
97002e7
sync MPAS with cam development
jtruesdal Mar 12, 2021
06e8700
try sync again
jtruesdal Mar 12, 2021
db9548b
Merge branch 'mpas_fixes' of https://github.com/jtruesdal/CAM-3 into …
jtruesdal Mar 12, 2021
61b9ef5
error check for allocates and Externals.cfg corrections
jtruesdal Mar 15, 2021
0113536
levels back to 30 for simple models, missing IC's for other dycores
jtruesdal Mar 15, 2021
c35311e
PR updates, comments, new Makefile cpp define to allow ESMF finalize,…
jtruesdal Mar 31, 2021
ffe26ff
sycn mpas pr with cam6_3_017
jtruesdal Apr 14, 2021
fa1b9f1
Merge branch 'ESCOMP-cam_development' into mpas_fixes
jtruesdal Apr 14, 2021
11f0f44
Update MPAS-A dycore external from f69242b to 224740a
mgduda Apr 21, 2021
6b110d8
merge mgduda NUOPC/ESMF work
jtruesdal Apr 22, 2021
6f76adf
Merge branch 'mgduda_cam/mpas_nuopc' into mpas_fixes
jtruesdal Apr 22, 2021
4da11a1
update cime external to fix bug with MPAS restart write for 15-3 grid
jtruesdal Apr 23, 2021
c9383e6
Merge branch 'mpas_fixes' of https://github.com/jtruesdal/CAM-1 into …
jtruesdal Apr 23, 2021
24121b5
MPAS moved to optional component until merged with mainline MPAS deve…
jtruesdal May 18, 2021
cde52b2
PR updates from goldy and jesse
jtruesdal Jun 3, 2021
fa79940
change mixing to diffusion
jtruesdal Jun 3, 2021
02e503f
better error coverage for user errors specifying decomposions, line n…
jtruesdal Jun 3, 2021
0e87e44
Bringing mpas_fixes up to cam_development for PR
jtruesdal Jun 3, 2021
3852251
update ChangeLog and remove added space from mod
jtruesdal Jun 4, 2021
610c8cf
update date
jtruesdal Jun 4, 2021
462c073
update Changelog for merge/commit
jtruesdal Jun 4, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 0 additions & 15 deletions bld/build-namelist
Original file line number Diff line number Diff line change
Expand Up @@ -3701,21 +3701,6 @@ if ($dyn =~ /mpas/) {
add_default($nl, 'mpas_block_decomp_file_prefix');
}

# invoke MPAS utility to generate streams files
my $mpas_libdir = $cfg->get('mpas_libdir');

# Check that the executable file streams_gen is present. streams_gen is built
# at the same time as the MPAS library. This allows build-namelist
# to be called before the MPAS library is built (via CESM scripts).
if ( -x "$mpas_libdir/streams_gen") {

my $cmnd = "$mpas_libdir/streams_gen $mpas_libdir/Registry_processed.xml ".
"streams.atmosphere stream_list.atmosphere. listed ";

system($cmnd) == 0 or die
"$ProgName - ERROR: generating MPAS streams files via command:\n".
"$cmnd";
}
}

# Defaults for history output
Expand Down
8 changes: 4 additions & 4 deletions bld/namelist_files/namelist_defaults_cam.xml
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,8 @@
<ncdata nlev="32" analytic_ic="1" >atm/cam/inic/cam_vcoords_L32_c180105.nc</ncdata>

<!-- Vertical coordinates and grid only -->
<ncdata hgrid="mpasa480" nlev="32" analytic_ic="1" >atm/cam/inic/mpas/mpasa480_L32_v6.1.grid_c190924.nc</ncdata>
<ncdata hgrid="mpasa120" nlev="32" analytic_ic="1" >atm/cam/inic/mpas/mpasa120_L32_topo_grid_c201022.nc</ncdata>
<ncdata hgrid="mpasa480" nlev="32" analytic_ic="1" >atm/cam/inic/mpas/mpasa480_L32_notopo_grid_c201125.nc</ncdata>
<ncdata hgrid="mpasa120" nlev="32" analytic_ic="1" >atm/cam/inic/mpas/mpasa120_L32_notopo_grid_c201216.nc</ncdata>

<!-- Files with initial conditions -->
<ncdata dyn="fv" hgrid="0.23x0.31" nlev="26" ic_ymd="101" >atm/cam/inic/fv/cami_0000-01-01_0.23x0.31_L26_c100513.nc</ncdata>
Expand Down Expand Up @@ -2684,7 +2684,7 @@
<mpas_time_integration > SRK3 </mpas_time_integration>
<mpas_time_integration_order > 2 </mpas_time_integration_order>
<mpas_dt > 1800.0D0 </mpas_dt>
<mpas_dt hgrid="mpasa120" > 600.0D0 </mpas_dt>
<mpas_dt hgrid="mpasa120" > 900.0D0 </mpas_dt>

<mpas_split_dynamics_transport>.true.</mpas_split_dynamics_transport>
<mpas_number_of_sub_steps > 2 </mpas_number_of_sub_steps>
Expand Down Expand Up @@ -2712,7 +2712,7 @@
<mpas_scalar_advection > .true. </mpas_scalar_advection>
<mpas_positive_definite > .false. </mpas_positive_definite>
<mpas_monotonic > .true. </mpas_monotonic>
<mpas_coef_3rd_order > 0.25D0 </mpas_coef_3rd_order>
<mpas_coef_3rd_order > 1.0D0 </mpas_coef_3rd_order>
<mpas_smagorinsky_coef > 0.125D0 </mpas_smagorinsky_coef>
<mpas_mix_full > .true. </mpas_mix_full>
<mpas_epssm > 0.1D0 </mpas_epssm>
Expand Down
124 changes: 124 additions & 0 deletions doc/ChangeLog
Original file line number Diff line number Diff line change
@@ -1,5 +1,129 @@
===============================================================

Tag name:
Originator(s):
Date:
One-line Summary:
Github PR URL:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please fill these in

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.


Purpose of changes (include the issue number and title text for each relevant GitHub issue):

. Remove unused code in build-namelist for generating MPAS streams files.

. Fix to allow MPAS to run with 1 task.

. Fix to allow MPAS to run with NAG/debug.

. Replace bad grid file for running 480-km res with no topo.

. Fix for bug when MPAS dycore is substepping during physics timestep.

. Update calculation of hydrostatic pressure from MPAS state.

. Add ability to write initial file when using MPAS dycore.

Describe any changes made to build system:

Describe any changes made to the namelist:

List any changes to the defaults for the boundary datasets:

Describe any substantial timing or memory changes:

Code reviewed by:

List all files eliminated:

List all files added and what they do:

List all existing files that have been modified, and describe the changes:

bld/build-namelist
. remove code that executes the mpas streams_gen utility. It's not being
used.

bld/namelist_files/namelist_defaults_cam.xml
. replace mpas grid file mpasa480_L32_v6.1.grid_c190924.nc by
mpasa480_L32_notopo_grid_c201125.nc
. new defaults: longer time step, change 3rd-order scalar transport upwind coefficient
mpas_dt 600s => 900s
mpas_coef_3rd_order 0.25 => 1.0
Comment on lines 11 to 50
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please point to issues that describe the need for each of these changes (as noted in the instructions). I do not currently see any issues addressed by this PR, please link to them or open them as needed.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New issue: #385
MPAS is currently under development so in this case, I've opened a general issue for all of these items. I will be better about opening issues in the future as we anticipate more updates as the dycore port matures. While MPAS is under heavy development and in the interest of making efficient use of the reviewers' time, it is very likely that each of the next PR's will include several changes. I will make sure to update the issue, in the case of last-minute additions to a PR that weren't documented in the original.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While this looks like efficiency on the surface, it costs us a lot of time on the back end. The issues are our record and database for discussions of what needed fixing and why. Please trim #385 to match what is fixed in this PR and add a line to that effect in the PR title (see item 10 of the CAM SE PR procedures or the GitHub issue-closing syntax.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do.


src/dynamics/mpas/dyn_comp.F90
. Add target attribute to the module data arrays for height which are
pointed to by the hist_coords objects in the cam_history_support module.
NAG/debug requires this.
. remove the variable swap_time_level_ptrs
. cam_mpas_namelist_read
- allow the &decomposition namelist group to be missing by removing the
endrun call. When running with 1 task the partition file specified in
this group is not read.
. dyn_init
- change the dyn_out components with 2 time levels to point to the same
memory locations as the dyn_in components
- remove the code that copied dyn_in to dyn_out. The dyn_out pointers now point
to the same memory locations as the dyn_in pointers.
- remove setting swap_time_level_ptrs
. dyn_run
- after call to cam_mpas_run update the dyn_in/dyn_out pointers to point
to the MPAS current state in pool arrays with timeLevel=1
. read_inidat
- update computation of hydrostatic pressure profiles.
. set_base_state
- update computation of hydrostatic pressure used by base state.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Double check that all the modified files are accurately reflected here. (Externals.cfg is not listed though it now has change)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added documentation for Externals.cfg, diagnostics.F90 and listed subroutines that were removed when adopting weak scaling fixes. I'll have one more update when I run the official regression tests and will push this file before tagging.

src/dynamics/mpas/stepon.F90
. stepon_run1
- remove code to swap pointers after d_p_coupling
. shift_time_levels
- remove this routine. no longer needed.
. write_initial_file
- new subroutine to generate the initial filename, open a pio filehandle
to that file, and call the same mpas routines that are used to write to
the restart file.
. write_dynvar
- add conditional using write_inithist to call write_initial_file and
have MPAS write the initial file.


If there were any failures reported from running test_driver.sh on any test
platform, and checkin with these failures has been OK'd by the gatekeeper,
then copy the lines from the td.*.status files for the failed tests to the
appropriate machine below. All failed tests must be justified.

cheyenne/intel/aux_cam:

izumi/nag/aux_cam:

izumi/pgi/aux_cam:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The entry in namelist_definition.xml uses the word, mixing instead of the word diffusion. Please use the most appropriate term in both places (especially since namelist_definition.xml is our user-facing documentation).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@PeterHjortLauritzen below is the wording I currently have in namelist_definition.xml. I'm no longer sure which of the terminology is yours, mine or Michaels. Would you address Steve's concern (re mixing and diffusion) as well as any of your own and modify the following definition? I will update the ChangeLog and namelist definition file to use your version.

================

The coefficient for scaling the 2nd-order horizontal mixing in the mpas_cam absorbing layer.
A value of 1.0 will result in layered diffusion similar to CAM-SE and gives damping coefficients of 0.2216E7, 0.6482E6, 0.1927E6 in the top-most three layers on the dynamics variables u, w, and theta. The top 3 damping coefficients scale linearly with mpas_cam_coef. A value of 0.0 disables SE like 2nd-order diffusion in the absorbing layer and is the current default. In this case (no SE-like diffusion) we provide stability by activating Rayleigh damping on u in the absorbing layer with the namelist parameter mpas_rayleigh_damp_u.
Default: 0.0


CAM tag used for the baseline comparison tests if different than previous
tag:

Summarize any changes to answers, i.e.,
- what code configurations:
- what platforms/compilers:
- nature of change (roundoff; larger than roundoff but same climate; new
climate):

If bitwise differences were observed, how did you show they were no worse
than roundoff?

If this tag changes climate describe the run(s) done to evaluate the new
climate in enough detail that it(they) could be reproduced, i.e.,
- source tag (all code used must be in the repository):
- platform/compilers:
- configure commandline:
- build-namelist command (or complete namelist):
- MSS location of output:

MSS location of control simulations used to validate new climate:

URL for AMWG diagnostics output used to validate new climate:

===============================================================
===============================================================

Tag name: cam6_3_006
Originator(s): herrington, cacraig
Date: Dec 3, 2020
Expand Down
Loading