Merge pull request #266 from ESMValGroup/update_episode_8

Update episode 8 after testing with esmvaltool 2.7

Author: SarahAlidoost

_episodes/08-diagnostics.md

@@ -2,7 +2,7 @@
 title: "Writing your own diagnostic script"
 teaching: 20
 exercises: 30
-compatibility: ESMValTool v2.4.0
+compatibility: ESMValTool v2.7.0
 
 questions:
 - "How do I write a new diagnostic in ESMValTool?"
@@ -43,13 +43,17 @@ Let's get started!
 
 ## Understanding an existing Python diagnostic
 
-After a development mode installation, a folder called ``ESMValTool`` is created
-in your working directory. This folder contains the source code of the tool. We
-can find the recipe ``recipe_python.yml`` and the python script
-``diagnostic.py`` in these directories:
+If you clone the ESMValTool repository, a folder called ``ESMValTool`` is
+created in your home/working directory, see the instructions in the lesson
+[Development and contribution]({{ page.root }}{% link
+_episodes/07-development-setup.md %}).
 
-- *path_to_ESMValTool/esmvaltool/recipes/examples/recipe_python.yml*
-- *path_to_ESMValTool/esmvaltool/diag_scripts/examples/diagnostic.py*
+The folder ``ESMValTool`` contains the source code of the tool. We can find the
+recipe ``recipe_python.yml`` and the python script ``diagnostic.py`` in these
+directories:
+
+- *~/ESMValTool/esmvaltool/recipes/examples/recipe_python.yml*
+- *~/ESMValTool/esmvaltool/diag_scripts/examples/diagnostic.py*
 
 Let's have look at the code in  ``diagnostic.py``.
 For reference, we show the diagnostic code in the dropdown box below.
@@ -189,10 +193,10 @@ There are four main sections in the script:
 >> that holds all the necessary
 >> information needed to run the diagnostic script such as the location of input
 >> data and various settings. We will next parse this ``cfg`` variable
->> in the  ``main`` function and extract information as needed 
+>> in the  ``main`` function and extract information as needed
 >> to do our analyses (e.g. in line 69).
 >> 3. The ``main`` function is called near the very end on line 107. So, it is mentioned
->> twice in our code - once where it is called by the top-level Python script and 
+>> twice in our code - once where it is called by the top-level Python script and
 >> second where it is defined.
 > {: .solution}
 {: .challenge}
@@ -205,10 +209,10 @@ There are four main sections in the script:
 >
 {: .callout}
 
-## Preprocesor-diagnostic interface
+## Preprocessor-diagnostic interface
 
 In the previous exercise, we have seen that the variable ``cfg`` is the input
-argument of the ``main`` function. The first thing passed to the diagnostic
+argument of the ``main`` function. The first argument passed to the diagnostic
 via the ``cfg`` dictionary is a path to a file called ``settings.yml``.
 The ESMValTool documentation page provides an overview of what is in this file, see
 [Diagnostic script interfaces][interface].
@@ -221,12 +225,13 @@ The ESMValTool documentation page provides an overview of what is in this file,
 > then run the recipe ``recipe_python.yml``:
 >
 > ~~~bash
-> esmvaltool run recipe_example.yml
+> esmvaltool run examples/recipe_python.yml
 > ~~~
 >
 > 1. Find one example of the file ``settings.yml`` in the ``run`` directory?
-> 2. Take a look at the ``input_files`` list. It contains pathes to some files
->    ``metadata.yml``. What information do you think is saved in those files?
+> 2. Open the file ``settings.yml`` and look at the ``input_files`` list.
+>    It contains paths to some files ``metadata.yml``. What information do you
+>    think is saved in those files?
 >
 >> ## Answer
 >>
@@ -252,7 +257,7 @@ ESMValTool provides many functions such as ``select_metadata`` (line 72),
 ``sorted_metadata`` (line 76), and ``group_metadata`` (line 80). As you can see
 in line 8, these functions are imported from ``esmvaltool.diag_scripts.shared``
 that means these are shared across several diagnostics scripts. A list of
-available functions and their description can be found in 
+available functions and their description can be found in
 [The ESMValTool Diagnostic API reference][shared].
 
 
@@ -266,9 +271,9 @@ available functions and their description can be found in
 >> There is a statement after use of ``select_metadata``, ``sorted_metadata``
 >> and ``group_metadata`` that starts with ``logger.info`` (lines 73, 77 and
 >> 83). These lines print output to the log files. In the previous exercise, we
->> ran the recipe ``recipe_python.yml``. If you look at the log
->> file ``path_to_recipe_output/run/map/script1/log.txt``, you can see the output
->> from each of these functions, for example:
+>> ran the recipe ``recipe_python.yml``. If you look at the log file
+>> ``recipe_python_#_#/run/map/script1/log.txt`` in ``esmvaltool_output``
+>> directory, you can see the output from each of these functions, for example:
 >>
 >>```
 >>2021-03-05 13:19:38,184 [34706] INFO     diagnostic,83  Example of how to group and
@@ -325,9 +330,9 @@ available functions and their description can be found in
 
 ## Diagnostic computation
 
-After grouping and selecting data, we can read individual attributes (such as filename) 
-of each item. Here we have grouped the input data  by ``variables`` 
-so we loop over the variables (line 89-93). Following this, is a call to the 
+After grouping and selecting data, we can read individual attributes (such as filename)
+of each item. Here we have grouped the input data  by ``variables``
+so we loop over the variables (line 89-93). Following this, is a call to the
 function ``compute_diagnostic`` (line 94). Let's have a look at the
 definition of this function in line 43 where the actual analysis on the data is done.
 
@@ -336,7 +341,7 @@ Here, ``compute_diagnostic`` uses
 [Iris](https://scitools-iris.readthedocs.io/en/latest/index.html) to read data
 from a netCDF file and performs an operation ``squeeze`` to remove any dimensions
 of length one. We can adapt this function to add our own analysis. As an
-example, here we calculate the bias using the average of the data using Iris cubes. 
+example, here we calculate the bias using the average of the data using Iris cubes.
 
 ~~~python
 def compute_diagnostic(filename):
@@ -391,7 +396,7 @@ def compute_diagnostic(filename):
 
 > ## Reading data using the netCDF4 package
 >
-> Yet another option to read the NetCDF file data is to use 
+> Yet another option to read the NetCDF file data is to use
 > the [netCDF-4 Python interface][netCDF] to the netCDF C library.
 >
 >> ## Answer
@@ -443,7 +448,7 @@ there:
 ```
 
 This way, we can pass arguments such as the type of
-plot ``pcolormesh`` and the colormap ``cmap:Reds`` from the recipe to the 
+plot ``pcolormesh`` and the colormap ``cmap:Reds`` from the recipe to the
 ``quickplot``  function in the diagnostic.
 
 > ## Passing arguments from the recipe to the diagnostic
@@ -477,25 +482,25 @@ plot ``pcolormesh`` and the colormap ``cmap:Reds`` from the recipe to the
 
 In our example, the function ``save_data`` in line 57 is used to save the Iris
 cube. The saved files can be found under the ``work`` directory in a ``.nc`` format.
-There is also the function ``save_figure`` in line 63 to save the plots under the 
-``plot`` directory in a ``.png`` format (or preferred format specified in your 
+There is also the function ``save_figure`` in line 63 to save the plots under the
+``plot`` directory in a ``.png`` format (or preferred format specified in your
 configuration settings). Again, you may choose your own method
-of saving the output. 
+of saving the output.
 
 ### Recording the provenance
 
 When developing a diagnostic script, it is good practice to record
 provenance. To do so, we use the function ``get_provenance_record`` (line 99).
 Let us have a look at the definition of this function in line 21 where we
 describe the diagnostic data and plot. Using the dictionary ``record``, it is
-possible to add custom provenance to our diagnostics output. 
-Provenance is stored in the *W3C PROV XML*
+possible to add custom provenance to our diagnostics output.
+Provenance is stored in the *[W3C PROV XML](https://www.w3.org/TR/prov-xml/)*
 format and also in an *SVG* file under the ``work`` and ``plot`` directory. For
 more information, see [recording provenance][provenance].
 
 ## Congratulations!
 
-You now know the basic diagnostic script structure and some available tools for putting 
-together your own diagnostics. Have a look at existing recipes and diagnostics in the 
+You now know the basic diagnostic script structure and some available tools for putting
+together your own diagnostics. Have a look at existing recipes and diagnostics in the
 repository for more examples of functions you can use in your diagnostics!
 {% include links.md %}