Ensemble aggregation on dataset load?

[flicj191]

Hello all,
I have a question as I was struggling to find the documentation/code to confirm, I have found that when I load a Dataset that has multiple ensembles it loads a single cube, which looks like the mean of the ensembles.
Is this correct? Where can I find how it does this?

from esmvalcore.dataset import Dataset
var = Dataset(
    short_name='ts', mip='Amon', project='CMIP6',
    activity='CMIP',  dataset='CESM2',
    ensemble='*',
    institute='*', grid='*',
    exp='historical', timerange='*'
)
# number of datasets and list (11 ensembles)
len(list(var.from_files())), list(var.from_files())
var.load()  # a single cube

I just did a simple plot to check that it looks like the mean:

from esmvalcore.preprocessor import annual_statisitics, area_statistics
import iris.quickplot as qplt
import matplotlib.pyplot as plt

for ds in list(var.from_files()):
    cube = ds.load()
    cube = annual_statistics(cube, 'mean')
    qplt.plot(area_statistics(cube, 'mean'), label=ds['ensemble'], linewidth=0.5)

#do same for var dataset load
cube_var = annual_statistics(var.load(), 'mean')
qplt.plot(area_statistics(cube_var, 'mean'), label=var['dataset'], linewidth=1, color='k')
plt.legend()

[bouweandela]

Sorry, I did not see this earlier. I would recommend against using this; this will most likely concatenate the data from all ensembles (you can see exactly what happens on load in the Dataset.load method), so in the best case you'll end up with just a single ensemble member (whichever one is considered first by the concatenate preprocessor function), and in the worst case the resulting data will be a mix of multiple ensemble members.