Bug between Flang 22 and NUOPC_FieldDictionarySetup

[mathomp4]

As @oehmke knows, we are trying to get MAPL (and GEOS) working with Flang. Due to bug issues, we currently need to use essentially flang main (aka 22) as flang 22 isn't released yet. However, we have built it on discover at NCCS as well as on a development server here at the GMAO.

We eventually got around one issue that was in pFUnit (Goddard-Fortran-Ecosystem/pFUnit#515), and have another we are diagnosing (which might be an ESMF bug, we aren't sure).

However, we did eventually track down one bug that @bena-nasa was able to make a reproducer for. The reproducer code is:

program  test_nuopc_dict
   use ESMF
   use NUOPC

   integer :: status
   call ESMF_Initialize()
   call NUOPC_FieldDictionarySetup(filename='test.yml', rc=status)
   print*,status
   call NUOPC_FieldDictionarySetup(filename='test.yml', rc=status)
   print*,status
   call ESMF_Finalize()
end program

and test.yml is:

field_dictionary:
  version_number: 0.0.1
  last_modified: 2018-03-14T11:01:19Z
  institution:   SI Team
  contact:       atanas.trayanov@nasa.gov
  source:        http://nowhere
  description:   just for testing

  entries:
    - standard_name: 'I_1 standard name'
      canonical_units: 'smoot'
      description: 'made up import'
    - standard_name: 'E_1 standard name'
      canonical_units: 'barn'
      description: 'made up export'
    - standard_name: 'Internal_1 standard name'
      canonical_units: '1'
      description: 'made up internal'

Though I suspect test.yml could be pared down further.

If you run this code with gfortran you get:

mpirun -np 1 ./test_nuopc_dict
           0
           0

The status of both NUOPC_FieldDictionarySetup is zero. However, with flang:

mpirun -np 1 ./test_nuopc_dict
 0
 32

and the log file says:

20251121 081915.265 INFO             PET0 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
20251121 081915.266 INFO             PET0 !!! THE ESMF_LOG IS SET TO OUTPUT ALL LOG MESSAGES !!!
20251121 081915.266 INFO             PET0 !!!     THIS MAY CAUSE SLOWDOWN IN PERFORMANCE     !!!
20251121 081915.266 INFO             PET0 !!! FOR PRODUCTION RUNS, USE:                      !!!
20251121 081915.266 INFO             PET0 !!!                   ESMF_LOGKIND_Multi_On_Error  !!!
20251121 081915.266 INFO             PET0 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
20251121 081915.266 INFO             PET0 Running with ESMF Version   : develop+g05f088affa
20251121 081915.266 INFO             PET0 ESMF library build date/time: "Nov 20 2025" "22:49:06"
20251121 081915.266 INFO             PET0 ESMF library build location : /discover/nobackup/projects/gmao/SIteam/ESMF_Testing/llvm_22.0.0gitdb43e_mpich_O_develop/esmf
20251121 081915.266 INFO             PET0 ESMF_COMM                   : mpich
20251121 081915.266 INFO             PET0 ESMF_MOAB                   : enabled
20251121 081915.266 INFO             PET0 ESMF_LAPACK                 : enabled
20251121 081915.266 INFO             PET0 ESMF_NETCDF                 : disabled
20251121 081915.266 INFO             PET0 ESMF_PNETCDF                : disabled
20251121 081915.266 INFO             PET0 ESMF_PIO                    : disabled
20251121 081915.266 INFO             PET0 ESMF_YAMLCPP                : enabled
20251121 081915.268 ERROR            PET0 src/addon/NUOPC/src/NUOPC_FieldDictionaryDef.F90:182 Unable to deallocate memory  - Passing error in return code
20251121 081915.268 ERROR            PET0 src/addon/NUOPC/src/NUOPC_FieldDictionaryApi.F90:316 Unable to deallocate memory  - Passing error in return code
20251121 081915.268 ERROR            PET0 src/addon/NUOPC/src/NUOPC_FieldDictionaryApi.F90:457 Unable to deallocate memory  - Passing error in return code
20251121 081915.268 INFO             PET0  Finalizing ESMF

My guess is this is a Flang bug since Intel, GNU, and NAG all seem happy. But reporting a bug to LLVM where we start with "First build ESMF" probably won't gain much traction. It's possible this is some issue with deallocating nested DDTs? Not sure.

While a workaround in ESMF would be nice, it would also be good if maybe a reproducer could be found that doesn't need ESMF. I don't know how NUOPC and FieldDictionary and HConfig and all work enough to "remove" the ESMF and YAML and etc to do that.

[mathomp4]

@theurich Am I right in thinking your latest commits in develop fix this? It seems to for me, but it's also possible I'm somehow using the wrong esmf.mk (i.e., picked ifort one or something).

[theurich]

@mathomp4 yes, all of the LLVM issues should be fixed now on develop. See e.g. last nights runs: https://github.com/esmf-org/esmf-test-artifacts/blob/89826effb79a02a74fd7741364d6d74ccc39e7ac/develop/llvm/22.0.0gitdb43e/O/mpich/llvm-flang-2025-10-14/summary.dat

There is one more LLVM related change coming in (activating OpenMP by default) today, and then I will put a beta tag on tomorrow. That's what I will use to update and close all of the LLVM tickets.

[mathomp4]

Thanks for all the work! I think then the last issues MAPL has with flang is on our side.

Next up: hoping someone will build a "pre-built" flang that doesn't need recent glibc. The main reason I keep having to build from scratch...

[theurich]

@mathomp4 we now have a beta tag that contains the LLVM fixes. It is https://github.com/esmf-org/esmf/releases/tag/v9.0.0b10.