Implement tracer_data in CAM-SIMA; registry entries for prescribed ozone/BAM; NaN check in physics data check

[jimmielin]

Tag name (required for release branches):
Originator(s): @jimmielin

Description (include the issue title, and the keyword ['closes', 'fixes', 'resolves'] followed by the issue number):

• Port tracer_data, a generic reader for chemistry and aerosol data, from CAM to CAM-SIMA. The module is as close to the original tracer_data as possible, only removing CAM-specific features (i.e., chunking and physics buffer), FV dycore polar averaging, structured dycore latitude weighting, and a rmv_file flag that deletes files after being read -- closes CCPPize/move tracer_data to CAM-SIMA #432.
• Moves conservative regridding utility src/utils/horizontal_interpolate.F90 to CAM-SIMA (intended to be included in a sparse checkout from CAM in the future) -- currently only used for aircraft inputs, but could be used for all regridded data read from tracer_data in the future.
• Add string utilities increment_string, last_non_digit, and get_last_significant_char in support of tracer_data.
• Add registry entries for prescribed bulk (BAM) aerosol and prescribed ozone for radiation.
• Add tracer_data chemistry and aerosols (BAM) input regression test (Derecho GNU).
• Add a check for NaN values in physics check data -- closes NaNs in model state are not detected by physics data differences check #440.
  If any NaN values are detected in the model state, the difference count is incremented and the maximum difference will be denoted as NaN.
  The number of NaNs will be reported in the output. e.g.,

 ********** Physics Check Data Results **********

 TIMESTEP:            1

 Variable                                          # Diffs  Max Diff   Max Diff loc (rank, col, lev)
 --------                                          -------  --------   -----------------------------
 ocar1                                                  48       NaN   (0,1,25)
 (!) 48 NaN values in variable!
 bcar1                                                  48       NaN   (0,1,25)
 (!) 48 NaN values in variable!
 sslt1                                                  48       NaN   (0,1,25)

Describe any changes made to build system: N/A

Describe any changes made to the namelist: N/A

List any changes to the defaults for the input datasets (e.g. boundary datasets): N/A

List all files eliminated and why: N/A

List all files added and what they do:

A       src/utils/tracer_data.F90
  - new tracer_data utility in CAM-SIMA ported from CAM (will not be moved from CAM)

A       src/utils/horizontal_interpolate.F90
  - move horizontal_interpolate conservative regridder for chemistry data from CAM.

A       cime_config/testdefs/testmods_dirs/cam/outfrq_trcdata_bam_derecho/shell_commands
A       cime_config/testdefs/testmods_dirs/cam/outfrq_trcdata_bam_derecho/user_nl_cam
  - new tracer_data chemistry and aerosols input data regression test.

List all existing files that have been modified, and describe the changes:
(Helpful git command: git diff --name-status development...<your_branch_name>)

M       cime_config/testdefs/testlist_cam.xml
  - new tracer_data chemistry and aerosols input data regression test.

M       src/data/registry.xml
  - add prescribed ozone for radiation.
  - add prescribed BAM aerosol fields for radiation (used for input check).

M       src/physics/utils/physics_data.F90
  - check for NaNs in physics data check.

M       src/physics/utils/physics_grid.F90
  - make public the `dycore_unstructured` logical for tracer_data.

M       src/utils/string_utils.F90
  - new string utilities `increment_string`, `last_non_digit`, and `get_last_significant_char` in support of tracer_data.

If there are new failures (compared to the test/existing-test-failures.txt file),
have them OK'd by the gatekeeper, note them here, and add them to the file.
If there are baseline differences, include the test and the reason for the
diff. What is the nature of the change? Roundoff?

derecho/intel/aux_sima:

derecho/gnu/aux_sima:

If this changes climate describe any run(s) done to evaluate the new
climate in enough detail that it(they) could be reproduced:

CAM-SIMA date used for the baseline comparison tests if different than latest:

[nusbaume]

Thanks for bringing in this major new capability @jimmielin! I have some change requests, some of which would require non-trivial amounts of effort.

However, given that it is unclear what the future of all of this code will be, I am happy if you want to push back on some of my requests, or make them issues that we can tackle in the future. Just let me know!

[nusbaume]

I might change this code block a little to avoid extra function calls/math:

Suggested change

	! First, check if there are NaNs anywhere in the state
	if (shr_infnan_isnan(current_value(col))) then
	nan_count = nan_count + 1
	has_nan = .true.
	! Set max diff to NaN (if not already) to signal NaN was found
	if (.not. shr_infnan_isnan(max_diff(1))) then
	max_diff(1) = current_value(col) - buffer(col) ! = nan
	max_diff_col = col
	end if
	if (shr_infnan_isnan(current_value(col))) then
	nan_count = nan_count + 1
	if (.not. has_nan) then ! First NaN found for this variable
	has_nan = .true.
	! Set max diff to NaN (if not already) to signal NaN was found
	max_diff(1) = current_value(col) ! = nan
	max_diff_col = col
	end if

This same change request holds for the code block used in the 3d subroutine as well.

[jimmielin]

Thanks, updated!

[nusbaume]

I realize this will increase the scope of this PR slightly, but can we get rid of unstructured, and instead just replace it entirely with dycore_unstructured? I don't see any reason to keep two different variables that contain the same info.

[jimmielin]

Thanks for the suggestion! Changed in ffbc6a8

[nusbaume]

I realize this is a big ask, but can we move all of these new functions to src/core_utils/string_core_utils.F90, and then add unit tests for them?

You should be able to add the tests directly to test/unit/fortran/src/test_string_core_utils.pf and they will then run automatically as part of every PR's Github Actions. You can then point to them here like what is done for the other "core" functions above in order to avoid having to change any other file.

[jimmielin]

Thanks, moved to string_core_utils and added unit tests!

I still kept the public exports from string_utils.F90 that point to the actual functions in string_core_utils in line with what's being done now with other subroutines, but please let me know if you want me to remove these as well.

[nusbaume]

I don't think save is needed here.

[jimmielin]

Thanks, removed!

[nusbaume]

Might be worth adding the subroutine name to these endrun call messages? Otherwise I am not sure I would always know where these errors are coming from.

[jimmielin]

Thanks, added!

[nusbaume]

I believe this subroutine can be labeled pure (or even turned into a function as it only has one output).

[jimmielin]

Thanks, marked as pure!

[nusbaume]

I believe this subroutine can be labeled pure (or even turned into a function as it only has one output).

[jimmielin]

Thanks, marked as pure!

[nusbaume]

I believe this subroutine can be labeled pure (or even turned into a function as it only has one output).

[jimmielin]

Thanks, marked as pure!

[nusbaume]

Can we get rid of this subroutine entirely, and instead just use the one from ccpp_tuvx_utils?

[jimmielin]

It's the same subroutine, but ccpp_tuvx_utils is in physics/ncar_ccpp/to_be_ccppized/. Could I confirm if we are OK with CAM-SIMA code depending on atmos_phys code? Thanks!