Adding Atmosphere Jobs in the ObsForge Workflow

[xincjin-NOAA]

Atmospheric observation conversion needs to be added to the ObsForge workflow to enable conversion of observation data (e.g., BUFR to IODA format).

[xincjin-NOAA]

Current Status:

Framework: Initial structure for the Atmos BUFR conversion job to ObsForge. ( PR. #134, Cory)

File Naming Standardization: (PRs #77, #142, #78)

• Removed bufr_ prefix from .py and .yaml files
• prepbufr retained as a special case

Configuration: New atmos_bufr_dump_config.yaml file to centralize observation metadata

Code Enhancements:

• MPI support
• Parallelized conversion execution
• Additional feature improvements

Performed individual tests for all observation types in the scripts directory with follow todo list:

[ ] - amsua and mhs need special inputs (Xin)
[ ] - avhrr not ready ?
[ ] - prepbufr: issue: try to get time information from directory ?
[ ] - ssmis: need pvlib ?

Components of the Atmos Config File

 # an example of default observation type
     obstype:
       method: script2netcdf
       input_file: obstype, a list
       output_file: obstype.nc
       mapping_file: obstype.yaml, a list
       script_file: [obstype.py](http://obstype.py/), a list
       mpi: [optional mpi settings], a number
       aux: [optional auxiliary files] , a list of the files needed for converter

NOTE: only special cases are needed to fill in otherwise just

obstype: null

[xincjin-NOAA]

Some updates:

• Set up and tested pvlib environment; resolved SSMIS environment issue related to the pvlib Python package. Though we cannot using virtual environment.

• Processed two AVHRR cases successfully.

• prepbufr issue was solved temporary by adding the required path information into the input path.

• Added code to process AMSUA, MHS, and PREPBUFR observations.

• Tested all observation jobs (e.g., SATWND, IASI) both individually and under MPI parallel execution.