A subgrid-scale mosaic/tiling approach to land cover in MPAS-Noah#26
A subgrid-scale mosaic/tiling approach to land cover in MPAS-Noah#26drnimbusrain wants to merge 2 commits intoMPAS-Dev:masterfrom
Conversation
… LSM. This commit describes implementation of a mosaic/tiling (or simply the "mosaic") approach (based on Li et al., 2013) in the MPAS-Noah land surface model to assess the impact of subgrid-scale variability in land cover composition. Currently, the default land cover composition in MPAS-Noah is based on a "dominant" approach, as each grid cell is assumed to be entirely composed of the most abundant tile. In the new mosaic approach, a certain number of tiles (set as N by the user), each representing a land cover category, is considered within each grid cell (Li et al., 2013). Citation: Li, D., E. Bou-Zeid, M. Barlage, F. Chen, and J. A. Smith (2013), Development and evaluation of a mosaic approach in the WRF-Noah framework, J. Geophys. Res. Atmos., 118, 11,918–11,935, doi:10.1002/2013JD020657.
|
Initial tests of MPAS5 Noah-Mosaic: Example of top three mosaic sorted landuse (LU) fraction (landusef) and associated LU category indices July 01, 2013 MPAS5.0 test simulation, N = 8 tiles: Mosaic sorted landusef (1) for NLCD40 Associated mosaic LU indices for NLCD40 Mosaic sorted landusef (2) for NLCD40 Associated mosaic LU indices (2) for NLCD40 Mosaic sorted landusef (3) for NLCD40 Associated mosaic LU indices (3) for NLCD40 ***Note that after the sorted top 3 LU categories, the landusef is at or below about 10% across the globe for NLCD40. This means that over 90% of LU categories are represented. |
|
Initial tests of MPAS5 Noah-Mosaic: Example of top three mosaic skin/vegetation temperature (TSK) and latent heat flux (LH) July 01, 2013 MPAS5.0 test simulation, N = 8 tiles: Mosaic TSK (1) for NLCD40 Mosaic LH (1) for NLCD40 Mosaic TSK (2) for NLCD40 Mosaic LH (2) for NLCD40 Mosaic TSK (3) for NLCD40 Mosaic LH (3) for NLCD40 ***Note: There are clear differences in the TSK and LH for the top N = 3 mosaic tiles, as well as for the tiles N = 5 - 8 (not shown). This impacts the weighted (by landuse fraction) average of TSK and LH (as well as numerous other surface variables in Noah LSM) for each grid cell. Comparative results against a baseline MPAS5-Noah (with mosaic off) to follow. Patrick |
|
Comparison of MPAS5 Noah-Dominant (identical code, mosaic turned off) against Noah-Mosaic for N = 8 and N = 15 tiles. Examples for 2-m temperature (T2), 2-m water vapor mixing ratio (Q2), 1st soil layer temperature (TSLB), 1st soil layer moisture (SMOIS), and upward moisture flux at the surface (QFX). T2: Noah-Dominant Q2: Noah-Dominant TSLB: Noah-Dominant SMOIS: Noah-Dominant QFX: Noah-Dominant ***Note: Significant differences between the Noah-Dominant and Noah-Mosaic approaches to T2, Q2, TSLB, SMOIS, and QFX. Evaluation of the dominant vs. mosaic approach is forthcoming. Patrick |
|
Evaluation of MPAS5 Noah-Dominant (identical code, mosaic turned off) against Noah-Mosaic for N = 8 and N = 15 tiles against METAR observations of T2, Q2, and WSPD10. Spatial Root Mean Square Error (RMSE) Comparisons: Global T2: Noah-Dominant Global Q2: Noah-Dominant Global WSPD10: Noah-Dominant ***Note: Significant reductions in RMSE globally and over CONUS for T2, Q2, and WSPD10 due to the mosaic approach. The most dramatic reductions in RMSE are for Q2 over CONUS, where RMSE is reduced by a factor of 2 or more. Select examples of time series comparisons (and statistical summaries) of dominant vs. mosaic (N=8) T2, Q2, and WSPD10 against METAR sites in CONUS that experience strong land cover heterogeneity across grid cells. Albany, NY (KALB) Baltimore, MD (KBWI) Detroit, MI (KDTW) Evansville, IN (KEVV) ***Note: Significant reductions in mean absolute error (MAE), and increases in the index of agreement (IOA) at these sites due to use of the mosaic approach. For example, the IOA for the mosaic/dominant at KALB, KBWI, KDTW, and KEVV are 0.42/0.03, 0.44/0.01, 0.30/-0.10, and 0.47/0.10, which are major changes. Patrick |
Fixed bug in line 3066: Changed dimension of TSLB_mosaic to correct value of "nMosaicCat3D"
|
Updated Analysis CONUS Q2: Noah-Dominant CONUS T2: Noah-Dominant CONUS Q2: Noah-Dominant CONUS T2: Noah-Dominant Clear improvements in model performance (i.e., reduced RMSE) over CONUS for both Q2 and T2 when implementing the new Noah-Mosaic in MPASv5.2. I am preparing full-year 2016 runs testing the Noah-Mosaic implementation in MPASv5.2 for a 46-12 km mesh, and will update the analysis here when complete. This may be a topic of a future manuscript. Patrick |
|
@mgduda Hi Mike, If you are still interested in my MPASv6 changes to the Noah LSM for tiled land cover characteristics, you can see my published paper on the subject in JGR-Atmospheres, just accepted: https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/2019JD032093 The results of the Noah LSM tiled LCC can improve predictions of temperature, humidity, and wind speed in MPAS. https://zenodo.org/record/3734982#.Xvzs5WhKiM8 Thank you, Patrick C. Campbell |
This merge introduces new infrastructure for OpenACC-friendly halo exchanges for
2-d and 3-d real fields, and it updates the halo exchange calls in the MPAS-Atmosphere
dynamics to use new routines for exchanging halos on both CPUs and GPUs.
Existing halo exchange routines that work on MPAS field types require the use of
complicated "multi-halo exchange lists" within the halo exchange routines themselves.
Transferring these data structures onto GPUs to allow direct GPU-to-GPU halo
exchanges is overly complicated with OpenACC, so new routines (collectively using
the interface `mpas_dmpar_exch_halo_acc`) have been developed.
The `mpas_dmpar_exch_halo_acc` routines make use of a simple array of integers to
describe the communication of elements with other MPI tasks, and are thus are more
easily mapped from host to GPU memory.
The example, below, illustrates the use of the new OpenACC-friendly routines as they
are applied in the MPAS-Atmosphere dynamics:
```
integer, dimension(:), pointer :: compactHaloInfo_c_v
type (field2DReal), pointer :: field_2d_ptr
real (kind=RKIND), dimension(:,:), pointer :: theta_m
! Build integer array of halo information for a given field dimensionality
call mpas_pool_get_field(state, 'rho_zz', field_2d_ptr, 1)
call mpas_dmpar_compact_halo_info(field_2d_ptr, compactHaloInfo_c_v)
! Exchange the halo for a field array using the halo information
call mpas_pool_get_array(state, 'theta_m', theta_m, 1)
call mpas_dmpar_exch_halo_acc(theta_m, compactHaloInfo_c_v)
! Halo information can be re-used as many times as needed before being deallocated
deallocate(compactHaloInfo_c_v)
```
Note that the `mpas_pool_get_array_gpu` routine contains an `!ACC ENTER DATA COPYIN`
directive internally to check if data are present on the GPU or should be copied.
This facilitates the GPU direct communication.
Merge branch 'gpu_halos_merge' into atmosphere/gpu
* gpu_halos_merge: (24 commits)
Replacing CPU halo exchange calls with CPU + GPU halo exachage calls.
Initialize commListPtr = 0 when building send/recv lists for OpenACC exchanges
Compute compactHaloInfo arrays needed by scalar transport in mpas_atm_dynamics_init
Move computation of compactHaloInfo_* arrays into mpas_atm_dynamics_init
Add empty mpas_atm_dynamics_{init,finalize} routines
Pre-process out log writes in mpas_dmpar_exch_halo_acc if running on GPUs
Convert all halo exchanges in atm_srk3() to mpas_dmpar_exch_halo_acc interface
Remove a few debugging write statements from mpas_dmpar_exch_halo_acc routines
Only use compactHaloInfo and field arrays in mpas_dmpar_exch_halo_acc routines
Build {send,recv}CommList exclusively from compactHaloInfo in mpas_dmpar_build_comm_lists_acc
Add endpoint IDs to compactHaloInfo array
Use compactHaloInfo to obtain product of inner dims in mpas_dmpar_build_comm_lists_acc
Add sendList and recvList information to compactHaloInfo array
Use constant layout of compactHaloInfo for all field types
Add mpas_dmpar_build_comm_lists_acc routine to create comm lists from compactHaloInfo
In mpas_dmpar_exch_halo_acc routines, get nHaloLayers from compactHaloInfo
Add 'array' argument to mpas_dmpar_exch_halo_acc routines
Get MPI communicator and rank info from compactHaloInfo in mpas_dmpar_exch_halo_acc
mpas_dmpar_exch_halo_acc routines now require compactHaloInfo argument
Add interface mpas_dmpar_compact_halo_info
...
Update metapackage to 0.1.12
) Description This PR accompanies JCSDA-internal/mpas-jedi#728. It solves single precision build for MPAS-Model: issue MPAS-Dev#26 To build source env-setup/gnu-openmpi-cheyenne.sh set(MPAS_DOUBLE_PRECISION "OFF" CACHE STRING ....) in mpas-bundle/CMakeLists.txt ecbuild (cmake); make Modified files M CMakeLists.txt M Makefile M cmake/PackageConfig.cmake.in
) Description This PR accompanies JCSDA-internal/mpas-jedi#728. It solves single precision build for MPAS-Model: issue MPAS-Dev#26 To build source env-setup/gnu-openmpi-cheyenne.sh set(MPAS_DOUBLE_PRECISION "OFF" CACHE STRING ....) in mpas-bundle/CMakeLists.txt ecbuild (cmake); make Modified files M CMakeLists.txt M Makefile M cmake/PackageConfig.cmake.in
) Description This PR accompanies JCSDA-internal/mpas-jedi#728. It solves single precision build for MPAS-Model: issue MPAS-Dev#26 To build source env-setup/gnu-openmpi-cheyenne.sh set(MPAS_DOUBLE_PRECISION "OFF" CACHE STRING ....) in mpas-bundle/CMakeLists.txt ecbuild (cmake); make Modified files M CMakeLists.txt M Makefile M cmake/PackageConfig.cmake.in
2 participants
This pull request demonstrates implementation of an optional mosaic/tiling (or simply the "mosaic") approach (based on Li et al., 2013) in the MPAS-Noah land surface model to assess the impact of subgrid-scale variability in land cover composition. Currently, the default land cover composition in MPAS-Noah is only based on a "dominant" approach, as each grid cell is assumed to be entirely composed of the most abundant tile. In this new mosaic approach, a certain number of tiles (set as N by the user), each representing a land cover category, is considered within each grid cell (Li et al., 2013). Stark differences between model predicted ground heat flux, sensible heat flux, surface temperature, 2-m temperature, and 2-m specific humidity are apparent between the Noah mosaic and dominant approaches. Li et al. (2013) also demonstrated an improved performance for the mosaic compared to the dominant approach in regional WRF simulations, and that results with the mosaic approach are less sensitive to the spatial resolution of the grid.
The motivation for implementation of the mosaic is that sub-grid scale representation of land cover is vital at coarse model resolutions typically applied at the global scale. Sub-grid scale land cover may also be further important to the gradually refining mesh (i.e., variable spatial resolution) inherent to MPAS.
The implementation of the mosaic approach requires the "real world" fractional land use variable (i.e., landusef) as input. In this implementation, I use the NLCD40 land cover data based landusef.
The mosaic approach has been implemented here as an optional setting, where the user sets the following atmosphere.namelist options:
config_landuse_data = NLCD40
num_land_cat = 40
config_surface_mosaic = true
config_mosaic_cat = 8
The user sets config_surface_mosaic = true to turn on the Noah mosaic option, and then sets config_mosaic_cat to control the number (N) of mosaic tiles (max = num_land_cat) considered within each grid cell. Independent tests in R found that at 12x12 km with NLCD40, setting N = 8 - 15 resulted in 90 - 99% of cells having 99% of their land cover categories represented. See Li et al. (2013) for more details setting N.
In addition, optional Noah mosaic variables may be included in the output via an additional stream name "mosaic" in the stream_list.atmosphere, where user defined variables are set in the stream_list.atmosphere.output.mosaic file.
For this implementation, the NLCD40 land use parameters must also be added to the Noah table (i.e., LANDUSE.TBL) for the summer and winter. An example of the updated LANDUSE.TBL is attached here.
LANDUSE.TBL.txt
Comparative results and model performance of a test simulation for July 2013 over a quasi-uniform 92_25 mesh, will be provided in the additional comments of this pull request.
Patrick
Citation: Li, D., E. Bou-Zeid, M. Barlage, F. Chen, and J. A. Smith (2013), Development and evaluation of a mosaic approach in the WRF-Noah framework, J. Geophys. Res. Atmos., 118, 11,918–11,935, doi:10.1002/2013JD020657.