You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: _episodes/first_example_recipe.md
+59-70Lines changed: 59 additions & 70 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,5 +1,4 @@
1
1
---
2
-
3
2
title: "Running a recipe (First example)"
4
3
teaching: 20
5
4
exercises: 25
@@ -16,8 +15,7 @@ objectives:
16
15
keypoints:
17
16
- "A recipe does not break by fiddling with it"
18
17
- "Log information is useful when interpreting the first warnings/errors"
19
-
- "The dataset section resembles the filename or directory structure (e.g. CMIP subdirectories)."
20
-
18
+
- "The dataset section resembles the filename or directory structure (e.g. CMIP subdirectories)"
21
19
---
22
20
23
21
This episode describes how ESMValTool recipes work, how to run a recipe and how to explore the recipe output. By the end of this episode, you should be able to run your first recipe, look at the recipe output, modify a recipe, explore and run some basic recipe debugging.
@@ -26,32 +24,32 @@ This episode describes how ESMValTool recipes work, how to run a recipe and how
26
24
27
25
Recipes are the instructions that you give to ESMValTool that tell it what you want to do. This includes four main sections: datasets, preprocessors, diagnostics and description.
28
26
29
-
- datasets: what datasets you want to use, including
30
-
- the time range and time resolution,
31
-
- the MIP, ensemble member,
32
-
- the experiment (i.e. historical, ssp125, etc.),
33
-
- and the grid type (CMIP6 only).
27
+
- datasets: what datasets you want to use, including
28
+
- the time range and time resolution,
29
+
- the MIP, ensemble member,
30
+
- the experiment (i.e. historical, ssp125, etc.),
31
+
- and the grid type (CMIP6 only).
34
32
35
-
- preprocessors: general operations applied to a dataset before handling it in a diagnostic, listing
36
-
- which preprocessor modules to apply,
37
-
- the order to apply them,
38
-
- and the preprocessor arguments.
33
+
- preprocessors: general operations applied to a dataset before handling it in a diagnostic, listing
34
+
- which preprocessor modules to apply,
35
+
- the order to apply them,
36
+
- and the preprocessor arguments.
39
37
40
-
This section can also be optional, if no preprocessing is needed.
38
+
This section can also be optional, if no preprocessing is needed.
41
39
42
-
- diagnostics: all the information about the diagnostic, including
43
-
- list of variables to evaluate (with their respective configurations),
44
-
- the desired diagnostic script to use,
45
-
- and additional diagnostic script options or arguments, if needed.
40
+
- diagnostics: all the information about the diagnostic, including
41
+
- list of variables to evaluate (with their respective configurations),
42
+
- the desired diagnostic script to use,
43
+
- and additional diagnostic script options or arguments, if needed.
46
44
47
-
Also include additional datasets beyond those included in the datasets section mentioned above, for instance variable specific observational data.
45
+
Also include additional datasets beyond those included in the datasets section mentioned above, for instance variable specific observational data.
48
46
49
-
- description: a brief description of the recipe, including
50
-
- who wrote the recipe and who maintains it,
51
-
- which project is responsible for it,
52
-
- and which publications, references are linked with the recipe.
47
+
- description: a brief description of the recipe, including
48
+
- who wrote the recipe and who maintains it,
49
+
- which project is responsible for it,
50
+
- and which publications, references are linked with the recipe.
53
51
54
-
Note that the authors, publications and references are to be named in the config-references.yml
52
+
Note that the authors, publications and references are to be named in the config-references.yml
55
53
56
54
The information you provide in the recipe is not only affecting the processes you are starting, but also the directory names your output will be structured in.
57
55
For additional reeds, please have a look at the recipe format description in the [ESMValTool manual](https://docs.esmvaltool.org/projects/esmvalcore/en/latest/recipe/overview.html#recipe-section-diagnostics).
@@ -64,10 +62,8 @@ Once you’ve set up your conda environment and installed ESMValTool (see episod
64
62
esmvaltool -c configuration recipe
65
63
~~~
66
64
67
-
68
65
To try your hand with a basic recipe, please work through this episode.
69
66
70
-
71
67
## Introduction to the example recipe
72
68
The recipe presented here is a simple, basic recipe that takes a single dataset and produces a time series plot.
73
69
@@ -120,7 +116,6 @@ Please download the following recipe into your ESMValTool working area with the
120
116
> ~~~
121
117
{: .solution}
122
118
123
-
124
119
> ## Explore the recipe
125
120
> Use the command and investigate the sample recipe.
126
121
> ~~~bash
@@ -129,50 +124,50 @@ Please download the following recipe into your ESMValTool working area with the
129
124
{: .challenge}
130
125
131
126
Please note the following sections:
132
-
- documentation: lines 4-20
127
+
- documentation: lines 4-20
133
128
134
-
The documentation consists of the following information:
135
-
- description: a short description of the recipe
136
-
- authors: a list of authors (linked to esmvaltool/config-references.yml)
137
-
- maintainer: a list of maintainers (linked to esmvaltool/config-references.yml)
138
-
- references: a list of references (linked to a bibtexfile in esmvaltool/references with the same name)
139
-
- projects: a list of projects (linked to esmvaltool/config-references.yml)
129
+
The documentation consists of the following information:
130
+
- description: a short description of the recipe
131
+
- authors: a list of authors (linked to esmvaltool/config-references.yml)
132
+
- maintainer: a list of maintainers (linked to esmvaltool/config-references.yml)
133
+
- references: a list of references (linked to a bibtexfile in esmvaltool/references with the same name)
134
+
- projects: a list of projects (linked to esmvaltool/config-references.yml)
140
135
141
136
142
-
- datasets: lines 22-23
137
+
- datasets: lines 22-23
143
138
144
-
The dataset definition consists of a list of python dictionaries with the information on the datasets.
145
-
- dataset name (key: dataset)
146
-
- project (key: project)
147
-
- experiment (key: exp)
148
-
- mip (for CMIP data, key: mip)
149
-
- ensemble member (key: ensemble)
150
-
- time range (e.g. key-value-pair: start_year: 1982, end_year: 1990)
151
-
- model grid (for CMIP6 data only, key: grid)
152
-
- alias (key: alias; use the alias for e.g. a more human readable name)
139
+
The dataset definition consists of a list of python dictionaries with the information on the datasets.
140
+
- dataset name (key: dataset)
141
+
- project (key: project)
142
+
- experiment (key: exp)
143
+
- mip (for CMIP data, key: mip)
144
+
- ensemble member (key: ensemble)
145
+
- time range (e.g. key-value-pair: start_year: 1982, end_year: 1990)
146
+
- model grid (for CMIP6 data only, key: grid)
147
+
- alias (key: alias; use the alias for e.g. a more human readable name)
153
148
154
149
155
-
- preprocessors: lines 25-28
150
+
- preprocessors: lines 25-28
156
151
157
-
The definition for different preprocessors or combinations.
158
-
If no preprocessing is needed, the preprocessor can be set to an empty python dictionary (`{}`).
159
-
Here, we produce annual means. The preprocessor is called with its name (here: prep_timeseries), later in the diagnostic (line 39).
160
-
(See episode #5 LINK for more details.)
152
+
The definition for different preprocessors or combinations.
153
+
If no preprocessing is needed, the preprocessor can be set to an empty python dictionary (`{}`).
154
+
Here, we produce annual means. The preprocessor is called with its name (here: prep_timeseries), later in the diagnostic (line 39).
155
+
(See episode #5 LINK for more details.)
161
156
162
157
163
-
- diagnostic section: lines 30-42
158
+
- diagnostic section: lines 30-42
164
159
165
-
The information of which diagnostic script to run with which variables.
166
-
The diagnostics section has some indents that are free to call.
167
-
- the first indent (here: diag_timeseries_temperature) is the diagnostic’s name (a string without whitespace), used for setting up the respective directories
168
-
- description: a short description of the diagnostic
169
-
- variables: a definition of all variables that are used in this diagnostic
170
-
- the next indent (here: timeseries_variable) is the variables’ names (a string without whitespace) for the diagnostic to use
171
-
- short_name: the variable name as listed in the dataset
172
-
- preprocessor: the preprocessor(s) applied to the variable before running the diagnostic
173
-
- scripts: a definition of all scripts that are used in this diagnostic
174
-
- the next indent (here: timeseries_diag) is the scripts’ names (a string without whitespace) for the script to use
175
-
- script: a executable script with a directory relative to the `esmvaltool/diag_scripts/` directory
160
+
The information of which diagnostic script to run with which variables.
161
+
The diagnostics section has some indents that are free to call.
162
+
- the first indent (here: diag_timeseries_temperature) is the diagnostic’s name (a string without whitespace), used for setting up the respective directories
163
+
- description: a short description of the diagnostic
164
+
- variables: a definition of all variables that are used in this diagnostic
165
+
- the next indent (here: timeseries_variable) is the variables’ names (a string without whitespace) for the diagnostic to use
166
+
- short_name: the variable name as listed in the dataset
167
+
- preprocessor: the preprocessor(s) applied to the variable before running the diagnostic
168
+
- scripts: a definition of all scripts that are used in this diagnostic
169
+
- the next indent (here: timeseries_diag) is the scripts’ names (a string without whitespace) for the script to use
170
+
- script: a executable script with a directory relative to the `esmvaltool/diag_scripts/` directory
176
171
177
172
> ## Please answer the following questions:
178
173
> What is the short_name of the variable being analyzed?
@@ -184,7 +179,6 @@ Please note the following sections:
184
179
> What do you think running this recipe will produce?
185
180
{: .challenge}
186
181
187
-
188
182
> ## What is the short_name of the variable being analyzed?
189
183
> thetaoga - Global Average Sea Water Potential Temperature
190
184
{: .solution}
@@ -220,10 +214,10 @@ Please note the following sections:
220
214
{: .solution}
221
215
222
216
Each time you run the ESMValTool, it will produce a new output directory within your specified work directory with the name of the recipe and the tagged runtime. This folder should contain four folders:
223
-
- run
224
-
- work
225
-
- preproc
226
-
- plots
217
+
- run
218
+
- work
219
+
- preproc
220
+
- plots
227
221
228
222
> ## Inspect the output:
229
223
> Now that you have run the esmvaltool command for the first time, please locate your output directory.
@@ -311,7 +305,6 @@ The snippets for the edits can be found below:
@@ -322,22 +315,18 @@ The snippets for the edits can be found below:
322
315
> Which computing machine are you using? Does your user-config.yml file reflect your machine's settings? Is the dataset’s name in the correct order?
323
316
{: .solution}
324
317
325
-
326
318
> ## Diagnostic path problems
327
319
> The directory path to your diagnostics code is set relative to the esmvaltool/diag_scripts subdirectory. Is the code placed in this subdirectory? Is it spelled correctly?
328
320
{: .solution}
329
321
330
-
331
322
> ## FX files not found
332
323
> There is no FX file for your corresponding dataset. Are your datasets’ names spelled correctly? Is there a FX file for this dataset?
333
324
{: .solution}
334
325
335
-
336
326
> ## The preprocessor works but the diagnostic fails
337
327
> If your preprocessor works fine but your diagnostic script fails, congratulations! A failed diagnostic means that you won’t need to re-run the preprocessor. In your “run/main_log.txt” run output, you should see a line that reads: “To re-run this diagnostic script, run:”, followed by a line with a command that will allow you to re-run your diagnostic script only. Append this line with the “-i” option after the python script you call to re-run your diagnostic.
338
328
{: .solution}
339
329
340
-
341
330
> ## Your recipe’s name/project/reference isn’t recognised by ESMValTool. Error message is `ValueError: Tag 'NAME' does not exist in section 'authors' of path/esmvaltool/config-references.yml`
342
331
> Most likely, you added your own name to the recipe in the description section, but didn’t add it to the esmvaltool/config-references.yml file, where the names are linked to an email address, institute, and ORCID Identity.
0 commit comments