Update Omega submodule to current develop

[xylar]

This merge updates the e3sm_submodules/Omega submodule from 24f2fbd to f2e951a.

This update includes the following MPAS-Ocean and MPAS-Frameworks PRs (check mark indicates bit-for-bit with previous PR in the list):

• (ocn) Add temporary global constants header file Omega#288 - non-bit-for-bit changes to manufactured_solution/convergence_both/del4.
• (ocn) Vertical coordinate updates and fixes Omega#309
• (ocn) Start using hierarchical parallelism to limit vertical loops extents Omega#314
• (ocn) Add vertical mixing coefficients module Omega#310

Checklist

• Testing comment in the PR documents testing used to verify the changes

[xylar]

Testing

I'm seeing non-bit-for-bit results for the manufactured_solution/convergence_both/del4 as mentioned above. I haven't definitively shown that this is from E3SM-Project/Omega#310 but I think it's likely.

I'm also seeing failures in ocean/spherical/icos/rotation_2d that seem to be #420. They occur both before and after the submodule update.

Polaris omega_pr suite

I ran the omega_pr suite on the previous Omega submodule as a baseline and on the current develop (identical to the new submodule) for comparison.

• Baseline workdir: /lcrc/group/e3sm/ac.xylar/polaris_0.9/chrysalis/test_20260115/omega-pr-baseline
• Baseline build: /lcrc/group/e3sm/ac.xylar/polaris_0.9/chrysalis/test_20260115/omega-pr-baseline/build
• PR build: /lcrc/group/e3sm/ac.xylar/polaris_0.9/chrysalis/test_20260115/omega-pr-develop/build
• PR workdir: /lcrc/group/e3sm/ac.xylar/polaris_0.9/chrysalis/test_20260115/omega-pr-develop
• Machine: chrysalis
• Partition: compute
• Compiler: intel
• Build type: <Debug|Release>
• Log: /lcrc/group/e3sm/ac.xylar/polaris_0.9/chrysalis/test_20260115/omega-pr-develop/polaris_omega_pr.o1125606
• Result:
  • Failures (1 of 7):
    • ocean/spherical/icos/rotation_2d
  • Diffs (1 of 7):
    • ocean/planar/manufactured_solution/convergence_both/del4

[xylar]

@katsmith133, could you verify that you think the non-bit-for-bit changes in manufactured_solution/convergence_both/del4 are to be expected and that they are likely due to E3SM-Project/Omega#310.

@cbegeman, I'm flagging you for review just in case you have any feedback. Feel free to remove yourself as a reviewer if you don't.

[cbegeman]

@katsmith133, could you verify that you think the non-bit-for-bit changes in manufactured_solution/convergence_both/del4 are to be expected and that they are likely due to E3SM-Project/Omega#310.

@cbegeman, I'm flagging you for review just in case you have any feedback. Feel free to remove yourself as a reviewer if you don't.

I don't think it's likely to be @katsmith133's PR. I see constant changes affecting the manufactured solution in E3SM-Project/Omega#288 but I would have expected that to affect all manufactured solution tests.

[katsmith133]

@cbegeman and @xylar, just acknowledging that I see this, but I am a bit tied up in other things right now. I can work on this early next week.

[xylar]

@cbegeman, good point. I could see why E3SM-Project/Omega#288 might be responsible. The differences I see are larger than machine precision:

layerThickness       Time index: 0
0:  l1: 3.69555097515178e+01  l2: 1.03017505127419e-01  linf: 4.79766074022336e-04
  FAIL /lcrc/group/e3sm/ac.xylar/polaris_0.9/chrysalis/test_20260115/omega-pr-develop/ocean/planar/manufactured_solution/del4/forward/25km_38s/output.nc
       /lcrc/group/e3sm/ac.xylar/polaris_0.9/chrysalis/test_20260115/omega-pr-baseline/ocean/planar/manufactured_solution/del4/forward/25km_38s/output.nc
normalVelocity       Time index: 0
0:  l1: 9.89667198754798e+00  l2: 1.65499010251692e-02  linf: 6.98835382923479e-05
  FAIL /lcrc/group/e3sm/ac.xylar/polaris_0.9/chrysalis/test_20260115/omega-pr-develop/ocean/planar/manufactured_solution/del4/forward/25km_38s/output.nc
       /lcrc/group/e3sm/ac.xylar/polaris_0.9/chrysalis/test_20260115/omega-pr-baseline/ocean/planar/manufactured_solution/del4/forward/25km_38s/output.nc
          baseline comp.:   FAIL
          runtime:          0:00:35

It is also not obvious to me why this would be the case for del4 but not the other manufactured_solution runs.

[xylar]

Testing

After adding Omega support in #452, I ran the e3sm_update utility and was able to show that:

• Add temporary global constants header file Omega#288 is responsible for the changes in manufactured_solution/convergence_both/del4 (good call @cbegeman)
• Vertical coordinate updates and fixes Omega#309, Start using hierarchical parallelism to limit vertical loops extents Omega#314, and Add vertical mixing coefficients module Omega#310 cause no non-bit-for-bit changes in omega_pr
• I ran into PIO error in Omega test on Chrysalis, Frontier and Aurora #420 a few times along the way, had to delete output.nc and rerun but was able to get past the issue after that.

Results are here if anyone wants to have a look:

/lcrc/group/e3sm/ac.xylar/polaris_0.9/chrysalis/test_20260124/omega_update

[xylar]

@katsmith133 and @cbegeman, I'd like to get this merged. Any concerns about the non-bit-for-bit differences in E3SM-Project/Omega#288, or should we just accept them?

[cbegeman]

@katsmith133 and @cbegeman, I'd like to get this merged. Any concerns about the non-bit-for-bit differences in E3SM-Project/Omega#288, or should we just accept them?

I'm not concerned about the non-BFB nature of that PR unless the manufactured solution del4 test error levels significantly change or the rate of convergence significantly changes. Can you/did you check that?

[xylar]

Oops! I looked at the results and I clearly confused myself. It is E3SM-Project/Omega#309 that is causing the non-bit-for-bit changes in del4.

@katsmith133, you're off the hook then. Nothing to do with either of your PRs.

@brian-oneill, could you confirm that this might be expected, i.e. that manufactured_solution/convergence_both/del4 might slightly improve in its convergence as a result of those changes?

@cbegeman, thanks for pushing for the plot. This made it clear I had the wrong PR in mind, and also that the convergence has improved!

[cbegeman]

Approving on the basis of successful testing (expected Omega fails) with this submodule hash and @xylar 's investigation into the source of discrepancies with the previous model version

[brian-oneill]

I think I found the cause. Turns out the outermost halo elements of the EdgeMask were not set correctly before, and E3SM-Project/Omega#309 fixed this. The stencil for the velocity hyperdiffusion is wide enough that this generated errors that leaked into the domain, slightly increasing the L2 errors.

[xylar]

Thanks @brian-oneill. It's good to have an explanation. I'll go ahead and merge this.