Skip to content

Comments

Atmosphere/serialbox test#1

Draft
abishekg7 wants to merge 11 commits intodevelopfrom
atmosphere/serialbox_test
Draft

Atmosphere/serialbox test#1
abishekg7 wants to merge 11 commits intodevelopfrom
atmosphere/serialbox_test

Conversation

@abishekg7
Copy link
Owner

@abishekg7 abishekg7 commented Dec 9, 2024

Steps to use the python scripts for comparison

  1. Load the modules
1) ncarenv/23.09 (S)   2) craype/2.7.23   3) nvhpc/24.3   4) ncarcompilers/1.0.0   5) cray-mpich/8.1.27   6) parallel-netcdf/1.12.3   7) cuda/12.2.1
  1. Activate python env and set some paths
export SERIALBOX_ROOT=/glade/derecho/scratch/agopal/serialbox_2.6.2_2
source $SERIALBOX_ROOT/.venv/bin/activate

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$SERIALBOX_ROOT/install/lib:/glade/u/apps/common/23.08/spack/opt/spack/gcc/13.2.0/lib64

export PYTHONPATH=$SERIALBOX_ROOT/install/python:$PYTHONPATH
  1. Run compare script
python $SERIALBOX_ROOT/src/serialbox-python/compare/compare.py dir1/ser_data/MetaData-mpas_dycore.json dir2/ser_data/MetaData-mpas_dycore.json

link to hackmd page with more info: https://hackmd.io/uT-W_PEhTSWXZI7YkS0wWA

G. Dylan Dickerson and others added 11 commits August 29, 2024 13:39
This includes splitting a long do loop on iCell so an if condition on
rk_step is outside the do loop. Though this condition shouldn't lead to
warp divergence, it can still be helpful to do branch evaluation on the
device best suited for it (the CPU). Since there was also limited data
re-use in the long do loop, it may help to reduce kernel launch
overhead.

Also line up whitespace so the bdyMaskCell if condition is apparent and
clean up some lines that end with whitespace.
…work

This commit adds an initial port of this routine using OpenACC. More
changes are needed to improve performance.
This commit ensures the invariant fields used during this work routine
are present on the device from model startup to model shutdown.

It builds on the changes in PR MPAS-Dev#1176 to copyin invariant fields during
mpas_atm_dynamics_init and delete them from the device during
mpas_atm_dynamics_finalize.
These changes ensure that the other, non-invariant, fields are available
on the device during this routine. Some fields that are overwritten are
only created at the beginning, while others are copied in. Any fields
that were unmodified right-hand side fields are deleted at the end and
modified left-hand side fields are copied out.

Timing for these transfers are reported in the output log file in the
new timer: 'atm_recover_large_step_variables [ACC_data_xfer]'.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant