Any possibility to mask the missing values when calculating EOF?

[matteodefelice]

Hi, unfortunately it's common to find datasets with NaN values, is it possible to mask in some way the NaN values when computing the EOF? Currently, the routine stops with a Error: There are missing values in the input data array

[jbedia]

Hi @matteodefelice, sorry for our slow reply. We have been actually looking at alternatives for dealing with missing data in the past, but we did not find a straightforward solution (there are of course ways of handling this...). Because we mainly use PCA for perfect condition predictors (reanalysis), we did not bother much about it so far. The routine is stopped before the variance/covariance matrix is built in the presence of NA/NaN to prevent subsequent errors. Of course, suggestions are welcome for new functionalities to be included.

[jorgebanomedina]

Hi @matteodefelice and @jbedia, sorry for the delay in solving this issue. We have included an imputation approach based on replacing missing data by the mean or median value, according to what has been found and commonly used in the literature.

[jbedia]

Hi @jorgebanomedina, no worries, it is great that you took the lead with this old issue. I would suggest to take the routine for NA filling out of this function in order to reuse it in other applications when needed.