Add "fast" recipes to RTW

[chrisbillowsMO]

Add the following "fast" recipes. If any fail, open a new separate issue and strike through from this list.

• recipe_albedolandcover
• recipe_aod_aeronet_assess
• recipe_autoassess_landsurface_permafrost
• recipe_autoassess_landsurface_soilmoisture
• recipe_autoassess_landsurface_surfrad
• recipe_autoassess_stratosphere

[sloosvel]

Here is a summary of all recipes that have been examined and in some cases added to the RTW, starting with B because A recipes are already handled in the comment above . I will open a PR with all the additions.

To add some recipes, it would be good if the RTW could handle the possibility of specifying the partition in which the job will be submitted.

Also it would be good if the max_parallel_tasks parameter could also be specified at a recipe level, instead of a at a global level. This is because some recipes can only run with the parameter set to 1. The recipes of this list have run with max_parallel_tasks set to 4.

B RECIPES

• recipe_bock20jgr_fig_1-4: Not fast
• recipe_bock20jgr_fig_6-7: Needs shared partition, time 48h, mem 50gb
• recipe_bock20jgr_fig8-10: Needs shared partition, time 48h, mem 50gb

---

C RECIPES

• recipe_capacity_factor: Added
• recipe_carvalhais14nat: Added
• recipe_climate_change_hotspot: Needs compute queue, constraint 512GB
• recipe_check_obs: missing datasets, needs compute queue, constraint 512GB
• recipe_climwip_brunner2019_med: Unable to convert from 'Unit('1.15740740740741e-05 kg.s-4')' to 'Unit('W m-2')'.
• recipe_climwip_brunner20esd: Not fast
• recipe_climwip_test_basic: Added
• recipe_climwip_test_performance_sigma: Added
• recipe_clouds_bias: Added
• recipe_clouds_ipcc: Added
• recipe_cmug_h2o: Added
• recipe_collins13ipcc: Needs compute queue, constraint 512GB, max_parallel_tasks=1
• recipe_combined_indices: diagnostic failure
• recipe_concatenate_exps: Added
• recipe_consecdrydays: Added
• recipe_correlation: Added
• recipe_cox18nature: Added
• recipe_cvdp: Added

---

D RECIPES

• cmorizers--recipe_daily_era5.yml: Not fast
• examples--recipe_decadal: Added
• testing--recipe_deangelis15nat_fig1_fast.yml: Added
• recipe_deangelis15nat.yml: Not fast
• recipe_diurnal_temperature_index.yml: Added

---

E RECIPES

• cmorizers--recipe_era5-land: Added
• examples--recipe_easy_ipcc: Not fast
• examples--recipe_extract_shape: Added
• recipe_eady_growth_rate: Not fast, need compute partition
• recipe_ecs: Not fast, needs compute partition
• recipe_ecs_constraints: Not fast, needs compute partition
• recipe_ecs_scatter: Added
• recipe_ensclus: Added, is at medium at the moment
• recipe_esacci_lst: Added
• recipe_esacci_oc: Added
• recipe_extreme_events: Not fast
• recipe_extreme_index: Needs compute partition
• recipe_eyring06jgr: Not fast, needs compute partition
• recipe_eyring13jgr_12: Not fast, needs compute partition

---

F RECIPES

• ipccwg1ar5ch9--recipe_flato13ipcc_figure_914: Added
• ipccwg1ar5ch9--recipe_flato13ipcc_figure_924:Added
• ipccwg1ar5ch9--recipe_flato13ipcc_figure_942: Not fast
• ipccwg1ar5ch9--recipe_flato13ipcc_figure_945a: Added
• ipccwg1ar5ch9--recipe_flato13ipcc_figure_96: Added
• ipccwg1ar5ch9--recipe_flato13ipcc_figure_98: Added
• ipccwg1ar5ch9--recipe_flato13ipcc_figures_926_927: Not fast
• ipccwg1ar5ch9--recipe_flato13ipcc_figures_92_95: Not fast
• ipccwg1ar5ch9--recipe_flato13ipcc_figures_938_941_cmip3: Not fast, max_parallel_task=1
• ipccwg1ar5ch9--recipe_flato13ipcc_figures_938_941_cmip6: Not fast, needs compute partition

---

G RECIPES

• hydrology--recipe_globwat: Added
• recipe_galytska23jgr: Not fast, needs compute partition
• recipe_gier2020bg: Added

---

H RECIPES

• hydrology--recipe_hydro_forcing: Added
• hydrology--recipe_hype: Added
• recipe_heatwaves_coldwaves: Added
• recipe_hyint: Not fast
• recipe_hyint_extreme_events: Added

---

I RECIPES

• ipccwg1ar6ch3--recipe_ipccwg1ar6ch3_atmosphere: Not fast, needs compute partition
• ipccwg1ar6ch3--recipe_ipccwg1ar6ch3_fig_3_19: Not fast, needs compute partition
• ipccwg1ar6ch3--recipe_ipccwg1ar6ch3_fig_3_42_a: Not fast, needs compute partition, constraint 512
• ipccwg1ar6ch3--recipe_ipccwg1ar6ch3_fig_3_42_b: Not fast, needs compute partition
• ipccwg1ar6ch3--recipe_ipccwg1ar6ch3_fig_3_43: Not fast, needs compute partition
• ipccwg1ar6ch3--recipe_ipccwg1ar6ch3_fig_3_9: Not fast, needs shared partition
• recipe_iht_toa: Added
• recipe_impact: Not fast

---

J RECIPES

• recipe_julia: Added

---

K RECIPES

• recipe_kcs: Added

---

L RECIPES

• clouds--recipe_lauer13jclim: Added
• clouds--recipe_lauer22jclim_fig1_clim: missing dataset CLARA-AVHRR
• clouds--recipe_lauer22jclim_fig1_clim_amip: missing dataset CLARA-AVHRR
• clouds--recipe_lauer22jclim_fig2_taylor: Missing data CLARA-AVHRR
• clouds--recipe_lauer22jclim_fig2_taylor_amip: Missing dataset CLARA-AVHRR
• clouds--recipe_lauer22jclim_fig3-4_zonal: Not fast, needs compute partition
• clouds--recipe_lauer22jclim_fig5_lifrac: Needs compute partition
• clouds--recipe_lauer22jclim_fig6_interannual: Missing data CLARA-AVHRR
• clouds--recipe_lauer22jclim_fig7_seas: Missing dataset CLARA-AVHRR
• clouds--recipe_lauer22jclim_fig8_dyn: Not fast
• clouds--recipe_lauer22jclim_fig9-11ab_scatter: Added
• clouds--recipe_lauer22jclim_fig9-11c_pdf: Added
• hydrology--recipe_lisflood: Added
• recipe_landcover: Added
• recipe_li17natcc: Added

---

M RECIPES

• examples--recipe_my_personal_diagnostic: Added
• hydrology--recipe_marrmot: Added
• model_evaluation--recipe_model_evaluation_basics: Not fast
• model_evaluation--recipe_model_evaluation_clouds_clim: Added
• model_evaluation--recipe_model_evaluation_clouds_cycles: Added
• model_evaluation--recipe_model_evaluation_precip_zonal: Added
• monitor--recipe_monitor.yml: Not fast
• monitor--recipe_monitor_with_refs: Added
• mpqb--recipe_mpqb_xch4: Needs compute partition
• recipe_martin18grl: Not fast
• recipe_meehl20sciadv: Needs compute partition
• recipe_miles_block: Not fast
• recipe_miles_eof: Not fast
• recipe_miles_regimes: Not fast
• recipe_modes_of_variability: Added
• recipe_multimodel_products: Added

---

N RECIPES

• examples--recipe_ncl: Added

---

O RECIPES

• recipe_ocean_Landschuetzer2016: Added
• recipe_ocean_amoc: missing data
• recipe_ocean_bgc: Added
• recipe_ocean_example: Not fast
• recipe_ocean_ice_extent: Added
• recipe_ocean_multimap: Not fast
• recipe_ocean_quadmap: Added
• recipe_ocean_scalar_fields:Added

---

P RECIPES

• examples--recipe_preprocessor_derive_test: Added
• examples--recipe_preprocessor_test: Added
• examples--recipe_python: Added
• testing--recipe_python_for_CI: Added
• hydrology--recipe_prcglobwb: Added
• recipe_perfmetrics_CMIP5: Needs compute partition
• recipe_perfmetrics_CMIP5_4cds: Needs compute partition
• recipe_perfmetrics_land_CMIP5: Needs compute partition
• recipe_portrait_CMIP: Added
• recipe_psyplot: Added
• recipe_pv_capacity_factor: Added

---

Q RECIPES

• recipe_quantilebias: Added

---

R RECIPES

• examples--recipe_r: Added
• recipe_radiation_budget: Added
• recipe_rainfarm: Added
• recipe_runoff_et: Add
• recipe_russell18jgr: Needs compute partition

---

S RECIPES

• recipe_schlund20esd: Needs compute partition, constraint=512G
• recipe_sea_surface_salinity: Needs compute partition
• recipe_seaborn: Added
• recipe_seaice: Added
• recipe_seaice_drift: Not fast
• recipe_seaice_feedback: Added
• recipe_shapeselect: Added
• recipe_smpi: Needs compute partition, max parallel tasks 1
• recipe_smpi_4cds: Needs compute partition, max parallel tasks 1
• recipe_snowalbedo: Added
• recipe_spei: Not fast

---

T RECIPES

• recipe_tcr: Added
• recipe_tebaldi21esd: Needs compute partition
• recipe_thermodyn_diagtool: Needs compute partition
• recipe_toymodel: Added

---

V RECIPES

• examples--recipe_variable_groups: Added
• recipe_validation: Added
• recipe_validation_CMIP6: Added

---

W RECIPES

• hydrology--recipe_wflow: Added
• recipe_weigel21gmd_figures_13_16: Added
• recipe_wenzel14jgr: Not fast, max parallel tasks 1
• recipe_wenzel16jclim: Needs compute partition
• recipe_wenzel16nat: Needs compute partition
• recipe_williams09climdyn_CREM: Added

---

Z RECIPES

• recipe_zmnam: Diagnostic failure. AttributeError: 'GeoContourSet' object has no attribute 'collections

[bouweandela]

Also it would be good if the max_parallel_tasks parameter could also be specified at a recipe level, instead of a at a global level. This is because some recipes can only run with the parameter set to 1.

@sloosvel Have you tested this recently? I've made some improvements to ESMValCore recently, see ESMValGroup/ESMValCore#2626 and ESMValGroup/ESMValCore#2610, that will hopefully make this less of a problem.

[sloosvel]

@sloosvel Have you tested this recently? I've made some improvements to ESMValCore recently, see ESMValGroup/ESMValCore#2626 and ESMValGroup/ESMValCore#2610, that will hopefully make this less of a problem.

I have not tested it yet, thanks for the heads-up!