SST: outline the implementation

[SarahAlidoost]

The goal is to develop a model to predict "uncertainty-quantified adjustments":

• according to the proposal, paired datasets of “observations-like” and “climate-model-like” versions of ERA5 SST will be used.
• Machine learning (ML) methods will be used to “learn” the spatio/temporal adjustments and the uncertainties. See relevant studies at SST: relevant recent literature #1.

Learning framework:
I suggest the following framework:

• input = partial observed SST+ possibly other relevant variables/information
• target = adjusted Observed SST(x,y,t) ; x/y space and t time
• model = based on the recent studies, we might be able to extend a masked autoencoder (MAE) to work with a 3D cube (x, y, t) , see MAE: recent literature #3
• loss function: should be a probabilistic loss function to predict the uncertainty of the prediction. Here, we need the partial observed uncertainty information.

Workflow:

• data preparation; “observations-like” and “climate-model-like” (or adjusted observation) versions of ERA5 SST
• model construction: picking a MAE base code (based on Pytorch) and extend it to a cube
• training: train-test split, loss function, optimizers, GPU/CPU
• inference and validation: defining a validation strategy and defining the spatio/temporal scope
• visualization and documentation: used for demo and dissemination

[SarahAlidoost]

Regarding extending MAE for 3D cube, the library VideoMAE supports MAE for videos (x,y, t).

[meiertgrootes]

SST in ERA5 is provided as a foundation SST from two sources (switching to OSTIA in 2007). As such there is no diurnal variability in SST within ERA5. This may/will impact the degree to which we can use ERA5 as 'mock' observational data, as there is no temporal variability that would depend on ttime-of-day at observation contained

[SarahAlidoost]

In section 3.4.3 of the proposal, the last sentence: "Here, we use monthly means of the variables ta, wap, cl, clivi, clt and lwp.". The variable "SST" isnot there. --> This needs clarification.

[SarahAlidoost]

Also, in section 2: "To derive adjustments, information is needed about how SST and water vapor vary that is sufficient to quantify sub-daily and spatial variability and potential definitional differences. For this, we will use ERA5 reanalysis data that provide gap-free hourly fields of the surface skin temperature and atmosphere at ~30 km resolution. Although sub-surface (“foundation”) SST in ERA5 is a prescribed field, sub-daily and skin-effect processes are represented in the coupled re-analysis on an hourly basis. This means all forms of SST variability affecting sampling biases are present."

In ERA5, Prescribed foundation SST → ERA5 simulates hourly surface processes → Produces skin temp with sub-daily variability.

[meiertgrootes]

MAE is likely most promising. Multi-modal MAE, with possible multi-task approach could be a way to go if different input datasets are sufficiently closely sampled in time/space