feature/EVS-NWPS_move_STEP

[SamiraArdani-NOAA]

Note to developers: You must use this PR template!

Description of Changes

Please include a summary of the changes and the related GitHub issue(s). Please also include relevant motivation and context.

This PR updates the NWPS naming conventions by moving the ${STEP} variable earlier in the filename. In addition, the "last31days" plots were separated from "last90days" to align with the other operational components in EVS. Validation testing confirms the updates are functional for these changes.

Closes #918

Developer Questions and Checklist

• Is this a high priority PR? If so, why and is there a date it needs to be merged by?
  No.
• Do you have any planned upcoming annual leave/PTO?
  No.
• Are there any changes needed in the times when the jobs are supposed to run/kick-off?
  No.

• The code changes follow NCO's EE2 Standards.
• Developer's name is removed throughout the code and have used ${USER} where necessary throughout the code.
• References the feature branch for HOMEevs are removed from the code.
• J-Job environment variables, COMIN and COMOUT directories, and output follow what has been defined for EVS.
• Jobs over 15 minutes in runtime have restart capability.
  plots step takes longer than 15 min but restart capability is not required for components in dev.
• If applicable, changes in the dev/drivers/scripts or dev/modulefiles have been made in the corresponding ecf/scripts and ecf/defs/evs-nco.def?
• Jobs contain the appropriate file checking and don't run METplus for any missing data.
• Code is using METplus wrappers structure and not calling MET executables directly.
• Log is free of any ERRORs or WARNINGs.

Testing Instructions

Please include testing instructions for the PR assignee. Include all relevant input datasets needed to run the tests.

1. Environment Setup

• clone https://github.com/SamiraArdani-NOAA/EVS.git
• checkout branch feature/EVS-NWPS_move_STEP and set your HOMEevs to point to your local repository.
• ln -sf /lfs/h2/emc/vpppg/noscrub/emc.vpppg/verification/EVS_fix/* fix/
• In all driver scripts, Update the COMIN so that it points to the parallel outputs in emc.vpppg , evs_devonly:
  export COMIN=/lfs/h2/emc/vpppg/noscrub/emc.vpppg/${NET}_devonly/${evs_ver_2d}
KEEPDATA set to YES
SENDMAIL set to NO

2- Execute the renamed J-jobs:

Prep: qsub $HOMEevs/dev/drivers/scripts/prep/nwps/jevs_prep_nwps_wave_grid2obs.sh

Stats: qsub $HOMEevs/dev/drivers/scripts/stats/nwps/jevs_stats_nwps_wave_grid2obs.sh

plots: qsub $HOMEevs/dev/drivers/scripts/plots/nwps/jevs_plots_nwps_wave_grid2obs_last31days.sh

plots: qsub $HOMEevs/dev/drivers/scripts/plots/nwps/jevs_plots_nwps_wave_grid2obs_last90days.sh

[malloryprow]

@AliciaBentley-NOAA Added you as a reviewer since the related GitHub Issue was opened by you

[malloryprow]

I think the changes look good. Hopefully we can start testing on Monday!

[AliciaBentley-NOAA]

Thanks, @malloryprow and @SamiraArdani-NOAA! I hope that we can begin testing on Monday, too! If Mallory still doesn't have WCOSS2 access, either @AndrewBenjamin-NOAA or I can probably do the testing. We are also preparing to give a MEG webinar on Thursday.

Please note that I initially told Samira that we couldn't merge these PRs into the develop branch until EVS v2.0 becomes operational. Thinking about this more, I think that we can merge these PR into the develop branch before then because GLWU and NWPS are always removed from the release branches anyway and never included in operational code deliveries. We can merge these PRs into develop after they are tested. Thanks!

[AndrewBenjamin-NOAA]

@SamiraArdani-NOAA I began testing, starting with the stats jobs. There was on Fatal Error, and a Warning (see below). Text files were available for ndbc buoy data and grib2 files for nwps data were available in prep. Do I need to add any VDATE or vhr when I submit the prep jobs?

WARNING: NO NWPS forecast was available for valid date 20260308
+ 185 + echo 'WARNING: NO NWPS forecast was available for valid date 20260308'
WARNING: NO NWPS forecast was available for valid date 20260308
03/09 12:14:28.068 metplus.1f5b4f5c (command_builder.py:481) WARNING: Could not find observation file
03/09 12:14:28.246 metplus.1f5b4f5c (command_builder.py:264) ERROR: (runtime_freq_wrapper.py:478) A problem occurred trying to obtain input files
03/09 12:14:28.246 metplus.1f5b4f5c (run_util.py:298) ERROR: ASCII2NC had 1 error.
03/09 12:14:28.248 metplus.1f5b4f5c (run_util.py:347) ERROR: METplus has finished running as user andrew.benjamin(32436) but had 1 error.
-- FATAL ERROR: Job jevs_prep_nwps_wave_grid2obs.253566626.cbqs01 failed RETURN CODE 1
Job jevs_prep_nwps_wave_grid2obs.253566626.cbqs01 failed RETURN CODE 1
timeout: failed to run command 'ecflow_client': No such file or directory 

Logs: /lfs/h2/emc/vpppg/noscrub/andrew.benjamin/pr919/EVS/dev/drivers/scripts/prep/nwps/jevs_prep_nwps_wave_grid2obs.o253566626
Data: /lfs/h2/emc/stmp/andrew.benjamin/evs_test/prod/tmp/jevs_prep_nwps_wave_grid2obs.253566626.cbqs01
Prep: /lfs/h2/emc/vpppg/noscrub/andrew.benjamin/evs/v2.0/prep/nwps/wave.20260308

CC: @malloryprow @AliciaBentley-NOAA

[AliciaBentley-NOAA]

@AndrewBenjamin-NOAA It looks like NWPS prep doesn't run until ~14:48Z, so there is no prep directory for wave.20260308/ yet. Can you please either 1) start by testing an NWPS prep job (INITDATE=20260307 would be the most recent date that ran) or 2) test the stats job again on a previous VDATE (VDATE=20260307 is the last stats job that ran).

CC @SamiraArdani-NOAA @malloryprow

[AndrewBenjamin-NOAA]

@SamiraArdani-NOAA I reran prep for INITDATE=2020260307. The fatal errors were removed, but the warning remained.
WARNING: NO NWPS forecast was available for valid date 20260307

Prep data for ndbc and nwps still appear in prep

Logs: /lfs/h2/emc/vpppg/noscrub/andrew.benjamin/pr919/EVS/dev/drivers/scripts/prep/nwps/jevs_prep_nwps_wave_grid2obs.o253568916
Data: /lfs/h2/emc/stmp/andrew.benjamin/evs_test/prod/tmp/jevs_prep_nwps_wave_grid2obs.253568916.cbqs01
Prep: /lfs/h2/emc/vpppg/noscrub/andrew.benjamin/evs/v2.0/prep/nwps/wave.20260307

CC: @malloryprow @AliciaBentley-NOAA

[SamiraArdani-NOAA]

Hi @AndrewBenjamin-NOAA! Thanks for testing this PR. The WARNING messages are for specific WFOs that they do not have nwps runs for particular forecast hour and those are expected. I still see that your prep folder is populated with grib2 files for available forecast hours.
@AliciaBentley-NOAA @malloryprow

[malloryprow]

Question. Am I following correctly that WARNING: NO NWPS forecast was available for valid date 20260307 appears in the log if one WFO doesn't have NWPS data but other WFOs may?

[SamiraArdani-NOAA]

@malloryprow, Right! Perhaps it would be better to bring $WFO and $fcst in the warning message to make it more specific.
cc @AliciaBentley-NOAA @AndrewBenjamin-NOAA

[malloryprow]

@malloryprow, Right! Perhaps it would be better to bring $WFO and $fcst in the warning message to make it more specific. cc @AliciaBentley-NOAA @AndrewBenjamin-NOAA

I was thinking the same thing! Could we get that added?

[AliciaBentley-NOAA]

@SamiraArdani-NOAA @malloryprow I agree. Please add at least variables to indicate exactly what is missing. Thanks!

CC @AndrewBenjamin-NOAA

[SamiraArdani-NOAA]

@malloryprow and @AliciaBentley-NOAA , I will update the PR for this change. Thanks!

[SamiraArdani-NOAA]

@AndrewBenjamin-NOAA, I checked the outputs with those in emc.vpppg parallel and they are identical. The " prep" test looks as expected.

[malloryprow]

Ah sorry didn't notice this until I saw it with the latest commit... "valid date" be "init date" here

[malloryprow]

@SamiraArdani-NOAA I got my WCOSS2 access so I'll take over this PR's testing. I just ran prep.

jevs_prep_nwps_wave_grid2obs

Log: /lfs/h2/emc/vpppg/noscrub/mallory.row/verification/EVS_PRs/pr919/EVS/dev/drivers/scripts/prep/nwps/jevs_prep_nwps_wave_grid2obs.o253927769
DATA: /lfs/h2/emc/stmp/mallory.row/evs_test/prod/tmp/jevs_prep_nwps_wave_grid2obs.253927769.cbqs01
COMOUT: /lfs/h2/emc/vpppg/noscrub/mallory.row/verification/EVS_PRs/pr919/evs/v2.0/prep/nwps/wave[.20260310]

I'm seeing failures. It looks like METplus is running but there are problems (missing?) with the input files.

[SamiraArdani-NOAA]

@malloryprow, would you please run the prep again by pointing $COMIN to emc.vpppg instead of your username ? It might not ne the solution but just for double check. Also, congratulations for wcoss2 account 🎉

[malloryprow]

Ah, that'll do it!

It would be good to add checks to not run METplus if we don't have the input data. I know NWPS, but it is still good to keep the code to operational/production standards in case we were able to ever get it in operations. If you think that would be a quick fix, we can add it to this PR, even though it isn't exactly in the scope of the PR. Or I can make a note in the Fixes and Additions for the future.

Otherwise, things look much better now that I got COMIN right!

Log: /lfs/h2/emc/vpppg/noscrub/mallory.row/verification/EVS_PRs/pr919/EVS/dev/drivers/scripts/prep/nwps/jevs_prep_nwps_wave_grid2obs.o253934723
DATA: /lfs/h2/emc/stmp/mallory.row/evs_test/prod/tmp/jevs_prep_nwps_wave_grid2obs.253934723.cbqs01
COMOUT: /lfs/h2/emc/vpppg/noscrub/mallory.row/verification/EVS_PRs/pr919/evs/v2.0/prep/nwps/wave[.20260310]

[SamiraArdani-NOAA]

@malloryprow, I agree with you about checking the existence of the input files before running the METplus. If you do not mind, I prefer to fix it in another PR. I am glad that fixing $COMIN helped. I have checked the outputs and they are identical with emc.vpppg parallel. You can now run stats job.