Provide basic scheduler support through ESMX including heterogeneous job support

[theurich]

NRL is leveraging ESMX for Navy-ESPC v2.2 and up. Currently we use a custom scripting layer inside of ESPC to generate job scripts for CI testing. It would be nice provide generic support on the ESMX layer.

This would have to include the heterogeneous job support, where the executable is launched with different number of tasks per nodes on different nodes. Again this part is currently implemented on the ESPC scripting layer leveraging MPICH/IntelMPI machinefiles or SLURM heterogeneous jobs, depending on the targeted system.

It would be convenient to have all of this provided by ESMX so it can be leveraged by any project that uses ESMX (e.g. all of the model components making up ESPC).

[anntsay]

Dan: adding workflow. has similar thoughts on why not adding workflow in ESMX. may need a larger discussion regarding what other tool kits to include (like uwtools or our own workflow).
Gerhard: this ticket is solving a very specific issue of handling heterogenous jobs.
Dan: has some yaml tools in wrf-hydro coupling to parflow project.

Action: maybe Dan and Gerhard can look at their yaml solutions and incorporate/streamline in ESMX.

[anntsay]

Dan's related code: https://github.com/NCAR/wrf-hydro_parflow_setup/blob/main/usecase/wrfhydro-parflow-lw-2way.yaml