GEOS-JEDI: Super-convergence

[rtodling]

This is the issue that I refer to as super convergence:

j3dfgat.jedi_run.log.20231005_03z.txt:jedi_run.csh: JEDI var converged in 40 iterations
j3dfgat.jedi_run.log.20231005_09z.txt:jedi_run.csh: JEDI var converged in 85 iterations
j3dfgat.jedi_run.log.20231005_15z.txt:jedi_run.csh: JEDI var converged in 89 iterations
j3dfgat.jedi_run.log.20231005_21z.txt:jedi_run.csh: JEDI var converged in 89 iterations
j3dfgat.jedi_run.log.20231006_03z.txt:jedi_run.csh: JEDI var converged in 101 iterations
j3dfgat.jedi_run.log.20231006_09z.txt:jedi_run.csh: JEDI var converged in 7 iterations
j3dfgat.jedi_run.log.20231006_15z.txt:jedi_run.csh: JEDI var converged in 101 iterations
j3dfgat.jedi_run.log.20231006_21z.txt:jedi_run.csh: JEDI var converged in 101 iterations
j3dfgat.jedi_run.log.20231007_03z.txt:jedi_run.csh: JEDI var converged in 101 iterations
j3dfgat.jedi_run.log.20231007_09z.txt:jedi_run.csh: JEDI var converged in 101 iterations
j3dfgat.jedi_run.log.20231007_15z.txt:jedi_run.csh: JEDI var converged in 101 iterations
j3dfgat.jedi_run.log.20231007_21z.txt:jedi_run.csh: JEDI var converged in 95 iterations
j3dfgat.jedi_run.log.20231008_03z.txt:jedi_run.csh: JEDI var converged in 101 iterations
j3dfgat.jedi_run.log.20231008_09z.txt:jedi_run.csh: JEDI var converged in 101 iterations
j3dfgat.jedi_run.log.20231008_15z.txt:jedi_run.csh: JEDI var converged in 101 iterations
j3dfgat.jedi_run.log.20231008_21z.txt:jedi_run.csh: JEDI var converged in 101 iterations
j3dfgat.jedi_run.log.20231009_03z.txt:jedi_run.csh: JEDI var converged in 10 iterations
j3dfgat.jedi_run.log.20231009_09z.txt:jedi_run.csh: JEDI var converged in 97 iterations
j3dfgat.jedi_run.log.20231009_15z.txt:jedi_run.csh: JEDI var converged in 101 iterations

As can be seen above there are times when the var stops at very few iterations. These cases, must be understood and used to debug the potential issue triggering these.

The cases above are when JEDI cycles without any input from GSI and the JEDI varBC results are not being cycled; that is, the values are what comes from the IODA converter for each case, as produced offline.

[rtodling]

This issue is largely related to how fv3-jedi does not handle 3d-pressures correctly. The most dominate observing system relying on 3d-pressue is GNSSRO - removing the RO from the yaml largely reduces this convergence issue - though obviously that is NOT the solution. The solution is in the update I have worked on that handles 3d-pressure correctly ... still needs to make it to JEDI dev officially.

[rtodling]

This may have been solved - see https://github.com/JCSDA-internal/fv3-jedi/pull/1374

Things are behaving in the jprePP experiment. I need to rerun a j3dfgat experiment to corroborate this being solved.