Better criteria for continuing PPE members

[hermanfugl]

As discussed in the PPE meeting today, filtering for RESTOM after 3 years drops members that may be on the way to adjusting to a better climate.

I plotted Gregory-like plots like @benmsanderson suggested for a set of variables we're struggling with: PRECT_global_ann, PRECT_amazon_ann, PRECT_SPac_SO_ann, TREFHT_land_global_ann, TREFHT_merid_slope_NH_winter, TREFHT_merid_slope_SH_winter (masks shown here).

Here are the plots: restom_extrapolation

Looks pretty convincing for e.g. land temperatures that some out-of-balance members would be better picks (for a given target) instead of our selected members (5, 7, 11, 19, 27, 33, 36, 38, 40, 41, 53):

The method works quite consistently but fails for some members: all_extrapolated_TREFHT_land_global_ann.png.
A couple more years would probably help but maybe it isn't necessary.

How would we combine the results for different variables to land on a good set of members (and make sure these members don't break anything else)?

@adagj @benmsanderson @maritsandstad @MichaelSchulzMETNO @oyvindseland

[maritsandstad]

Which ones to continue still differs depending on what we want to figure out. If we have reached the point where we think we can draw robust conclusions about what will happen, the answer is different then it is if we think we want to explore the overall behaviour a bit more from a slightly longer run. If it's the latter, I'd quite like to see a bit more from member 49...

[benmsanderson]

Oh, this is really nice - thanks Herman!

If we were just trying to get one variable right - we'd just pick runs where the extrapolated line intersects with the era diamond.

We're obviously trying to get multiple things right, but this approach allows us to predict where things will be at equilibrium.

2 possible strategies:

1. If we just worked with this 67 member ensemble, we could repeat the process for other target variables and build a combined skill score based on the distance of the extrapolated equilibrium state and the observational target.
   We'd need to normalise the distances for each variable in order to combine them together into an overall score.

The issue - I suspect - will be that other variables will be more noisy than the global land temperature, which seems to sit on a nice straight line as a function of restom.

But picking members where the extrapolated still score is maximised might be a reasonable strategy for choosing members to extend a bit.

2. If we have confidence in the extrapolations and we want to resample parameters, we could also use the projected equilibrium values as a target for a multilinear regression which predicts the stable levels of each variable as a function of the parameter values.
   The regression model would then be similar to what we used before too predict restom - but we'd be aiming to predict a wider set of relevant diagnostics.

Once we had that model - we can then simulate the equilibrium values for a big synthetic ensemble, and use that to predict configurations which maximise a multivariate skill score.

There are plenty of assumptions in all this - but your plots give me confidence that it might already work for temperature extrapolations when averaging over a large area.

Happy to chat tomorrow about how this might work in practise.

[hermanfugl]

I had a closer look at the predictability of different variables in a very out-of-balance member, extended to six years:

It seems as expected that the global mean temperature and precipitation are well-behaved. Nice to see that this also applies to the NH temperature gradient.

The Amazon is clearly too small to get a good signal, and I suppose the noise in the SH temperature gradient and the SPac/SO precipitation (which both looked linear in the three first years: TREFHT_merid_slope_SH, PRECT_SPac_SO_ann) could be due to some adjustment in the oceans/sea ice.

Edit: As a side note, this member's ECS estimated this way is very close to that in the beta09.