Start using hierarchical parallelism to limit vertical loops extents

[mwarusz]

This PR converts the following Omega modules to use hierarchical parallelism for limiting vertical loops extents:

• Auxiliary state
• Tendencies
• EOS

It includes some minor changes to the vertical coordinate module, but #309 has more substantial updates and needs to be merged first.

Checklist

• Documentation:
  • User's Guide has been updated
  • Developer's Guide has been updated
  • Documentation has been built locally and changes look as expected
• Building
  • CMake build does not produce any new warnings from changes in this PR
• Testing
  • CTest unit tests for new features have been added per the approved design.
  • Unit tests have passed. Please provide a relevant CDash build entry for verification.
  • Polaris test suite has passed

[mwarusz]

I have updated this PR to use hierarchical parallelism in the time stepping code to loop only over active layers.

[brian-oneill]

Looks good as a first step to integrating hierarchical parallelism in Omega. Ran ctests successfully on Chrysalis, pm-cpu, pm-gpu, and Frontier (cpu & gpu). Thanks @mwarusz !

[mwarusz]

@brian-oneill

I rebased this branch and updated the docs. I had to change the mesh used in time stepper tests because of issues with forcing constant NVertLayers when the input mesh has land boundaries.

[mark-petersen]

Ran a quick test. Passes CPU and GPU cTests on perlmutter.

[mark-petersen]

Thanks @mwarusz and @brian-oneill