EMC_verif-global

Package that uses MET and METplus for verifying the GFS.

This verification package was created to support the efforts at EMC to move to using the METplus authoritative verification repository, as directed under NGGPS. Previously, EMC had been using the NCEP-EMC Global NWP Model Verification Package, which uses Verification Statistics DataBase (VSDB) as its verification code.

This package has been designed to initially recreate all the verification capabilities that were present within the VSDB Global Verification Package:

1. Grid-to-Grid Verification: The first step in this verification is producing regular and anomalous partial sums for various variables at various pressure levels, as well as surface fields. Gridded model forecasts are verified aganist a gridded analysis. The second step invloves creating statistical plots and scorecard from the archived regular and anomalous partial sum files. Additionally, these plots can be sent, along with a website template, to a web server for easy display.
2. Grid-to-Observations Verification: The first step in this verification is producing regular partial sums for various variables at various pressure levels, as well as surface fields. Gridded model forecasts are verified aganist observations. The second step invloves creating statistical plots from the archived regular partial sum files. Additionally, these plots can be sent, along with a website template, to a web server for easy display.
3. Precipitation Verification: The first step in this verification is producing contingency table counts for precipitation accumulations at various thresholds. Gridded model forecasts are verified against observations. At this time only 24 hour preciptation accumulations compared to CCPA are supported. The second step invloves creating statistical plots from the archived contingency table count files. Additionally, these plots can be sent, along with a website template, to a web server for easy display.
4. Satellite Verification: The first step in this verification is producing regular partial sums for SST and sea-ice coverage. Gridded model forecasts are verified against satellite analysis files. The second step involves creating statistical plots from the archive regular partial sum files. Additionally, these plots can be sent, along with a website template, to a web server for easy display.
5. 2D Lat-Lon Maps Verification: Model forecasts for various variables are compared amongst other models and observations. The mean errors are displayed on a lat-lon grid as well as zonal means.
6. Data Assimilation Verification: Similar to the 2D lat-lon maps verification, but the model GDAS analysis increments and ensemble mean and spread are plotted.

This package can be run standalone. It has also been incorporated to run the first steps for grid-to-grid, grid-to-observations, and precipitation verification within GFS version 16 Global Workflow (https://github.com/NOAA-EMC/global-workflow).

EMC_verif-global depends on the following prerequisites to be available on the system:

• workload management platform / scheduler - LSF or SLURM
• python version 3.11.7 or 3.12.2 or newer
• MET version 12.0.1 (https://github.com/dtcenter/MET)
• METplus version 6.0.0 (https://github.com/dtcenter/METplus)
• NCEPLIBS-grib_util
• NCEPLIBS-prod_util
• NetCDF (Network Common Data Form)
• NetCDF Operators (NCO)

EMC_verif-global is supported on the following machines:

• WCOSS2 (Cactus and Dogwood)
• Hera
• Orion
• Gaea-C6

For questions or issues, please e-mail Alicia Bentley (alicia.bentley@noaa.gov).

Running EMC-verif-global

1. Set up configuration file

The default configuration file for EMC_verif-global depends on whether a user is running step1 (stats) or step2 (plots). The step1 config file is located under parm/config/run_standalone_step1_stats.py. The step2 config file is located under parm/config/config.step2. Users can create a copy of these configuration files for their own verification purposes. Within these configuration files are switches to run the various types of verifications listed above. To run, set the switch to YES, if not set the switch to NO. Then, follows a section of settings that generally apply to running all the verification types. Finally, each verification type/switch has its own section of settings specifically related to it.

2. Run

To run step1, users will move to the parm/config/ directory. Here, run_standalone_step1_stats.py and standalone_step1_stats.append run EMC_verif-global for step1. There are several required run time arguments which include the following: machine name, component name, start date in YYYYMMDD format, and end date in YYYYMMDD format. An example run time command may look like below

$ python run_standalone_step1_stats.py gaeac6 grid2grid 20241121 20241122

To run step2 from the top directory, users will move to the /ush directory. Here is the script run_verif_global.sh that runs EMC_verif-global for step2. It takes one run time argument which is the path to the configuration file that the user wishes to run with. If none is given it will default to using config.step2. An example run time command may look like below

$ ./run_verif_global.sh ../parm/config/config.step2

Known issues

Pandas library versions 2.1.0 and greater will return warnings when running Python plotting scripts about incompatible dtypes. These warnings can be ignored, but note that future versions of Pandas (v3.0.0) will turn these warnings into errors. Avoid using Pandas v3.0.0 until EMC_verif-global has been updated to be compatible.