Minor changes and edits after proof reading

Author: rswamina

_episodes/10-diagnostics.md

@@ -1,7 +1,7 @@
 ---
 title: "Writing your own diagnostic script"
-teaching: TBD
-exercises: TBD
+teaching: 20
+exercises: 30
 
 questions:
 - "How do I write a new diagnostic in ESMValTool?"
@@ -16,48 +16,48 @@ keypoints:
   with preprocessor output."
 - "Existing diagnostics can be used as templates and modified to write new
   diagnostics."
-- "Many functions can be imported from ``esmvaltool.diag_scripts.shared`` and
+- "Helper functions can be imported from ``esmvaltool.diag_scripts.shared`` and
   used in your own diagnostic script."
 ---
 
 ## Introduction
 
-The diagnostic script is an important component of ESMValTool where the
+The diagnostic script is an important component of ESMValTool and it is where the
 scientific analysis or performance metric is implemented. With ESMValTool, you
 can adapt an existing diagnostic or write a new script from scratch.
 Diagnostics can be written in a number of open source
 languages such as Python, R, Julia and NCL but we will focus on understanding
 and writing Python diagnostics in this lesson.
 
 In this lesson, we will explain how to find an existing diagnostic and run it
-using ESMValTool installed in a editable/development mode. For a development
+using ESMValTool installed in editable/development mode. For a development
 installation, see the instructions in the lesson [Development and
 contribution]({{ page.root }}{% link _episodes/08-development-setup.md %}).
 Also, we will work with the recipe [recipe_python.yml][recipe] and the
 diagnostic script [diagnostic.py][diagnostic] called by this recipe that we have
 seen in the lesson [Running your first recipe]({{ page.root }}{% link
 _episodes/04-recipe.md %}).
 
-Let's get started.
+Let's get started!
 
 ## Understanding an existing Python diagnostic
 
-After development installation, a folder called ``ESMValTool`` has been created
+After a development mode installation, a folder called ``ESMValTool`` is created
 in your working directory. This folder contains the source code of the tool. We
 can find the recipe ``recipe_python.yml`` and the python script
 ``diagnostic.py`` in these directories:
 
 - *path_to_ESMValTool/esmvaltool/recipes/examples/recipe_python.yml*
 - *path_to_ESMValTool/esmvaltool/diag_scripts/examples/diagnostic.py*
 
-Let's have look into the codes of the ``diagnostic.py``.
+Let's have look at the code in  ``diagnostic.py``.
 For reference, we show the diagnostic code in the dropdown box below.
 There are four main sections in the script:
 
 - A description i.e. the ``docstring`` (line 1).
 - Import statements (line 2-16).
 - Functions that implement our analysis (line 21-101).
-- A typical python top-level script i.e. ``if __name__ == '__main__'`` (line
+- A typical Python top-level script i.e. ``if __name__ == '__main__'`` (line
   102-107).
 
 > ## diagnostic.py
@@ -175,7 +175,7 @@ There are four main sections in the script:
 >
 {:.solution}
 
-> ## What is the starting point of the diagnostic?
+> ## What is the starting point of a diagnostic?
 >
 > 1. Can you spot a function called ``main`` in the code above?
 > 2. What are its input arguments?
@@ -184,11 +184,15 @@ There are four main sections in the script:
 >> ## Answer
 >>
 >> 1. The ``main`` function is defined in line 66 as ``main(cfg)``.
->> 2. The variable ``cfg`` is a Python dictionary holding all the necessary
->> information needed to run the diagnostic script like the location of input
->> data and various settings. In the ``main`` function, we will next parse this
->> ``cfg`` variable and extract information as needed to do our analyses (e.g. in line 69).
->> 3. The ``main`` function called near the very end on line 107.
+>> 2. The input argument to this function is the variable ``cfg``, a Python dictionary t
+>> hat holds all the necessary
+>> information needed to run the diagnostic script such as the location of input
+>> data and various settings. We will next parse this ``cfg`` variable
+>> in the  ``main`` function and extract information as needed 
+>> to do our analyses (e.g. in line 69).
+>> 3. The ``main`` function is called near the very end on line 107. So, it is mentioned
+>> twice in our code - once where it is called by the top-level Python script and 
+>> second where it is defined.
 > {: .solution}
 {: .challenge}
 
@@ -211,8 +215,8 @@ The ESMValTool documentation page provides an overview of what is in this file,
 > ## What information do I need when writing a diagnostic script?
 >
 > From the lesson [Configuration]({{ page.root }}{% link _episodes/03-configuration.md %}),
-> we saw how to change the configurations before running a recipe.
-> Let's first set the option ``remove_preproc_dir`` to ``false`` in the configuration file,
+> we saw how to change the configuration settings  before running a recipe.
+> First we set the option ``remove_preproc_dir`` to ``false`` in the configuration file,
 > then run the recipe ``recipe_python.yml``:
 >
 > ~~~bash
@@ -232,37 +236,37 @@ The ESMValTool documentation page provides an overview of what is in this file,
 >> detailed information on your data including project (e.g., CMIP6, CMIP5),
 >> dataset names (e.g., BCC-ESM1, CanESM2), variable attributes (e.g.,
 >> standard_name, units), preprocessor applied and time range of the data. You
->> can use all of these information in your own diagnostics.
+>> can use all of this information in your own diagnostic.
 > >
 > >
 > {: .solution}
 {: .challenge}
 
 ## Diagnostic shared functions
 
-Looking at the codes of the ``diagnostic.py``, we see that ``input_data`` is
+Looking at the code in  ``diagnostic.py``, we see that ``input_data`` is
 read from the ``cfg`` dictionary (line 69). Now we can group the ``input_data``
 according to some criteria such as the model or experiment. To do so,
-ESMValTool provides many functions like ``select_metadata`` (line 72),
+ESMValTool provides many functions such as ``select_metadata`` (line 72),
 ``sorted_metadata`` (line 76), and ``group_metadata`` (line 80). As you can see
 in line 8, these functions are imported from ``esmvaltool.diag_scripts.shared``
-that means these are shared between several diagnostics scripts. A list of
+that means these are shared across several diagnostics scripts. A list of
 available functions and their description can be found in [Shared diagnostic
 script code][shared].
 
 > ## Extracting information needed for analyses
 >
 > We have seen the functions used for selecting, sorting and grouping data in the
-> script. What these functions do?
+> script. What do these functions do?
 >
 >> ## Answer
 >>
 >> There is a statement after use of ``select_metadata``, ``sorted_metadata``
 >> and ``group_metadata`` that starts with ``logger.info`` (lines 73, 77 and
 >> 83). These lines print output to the log files. In the previous exercise, we
->> ran the recipe ``recipe_python.yml``. If you looked at the content of the log
->> file ``path_to_recipe_output/run/map/script1/log.txt``, you can see the some
->> information on how each function works, for example:
+>> ran the recipe ``recipe_python.yml``. If you look at the log
+>> file ``path_to_recipe_output/run/map/script1/log.txt``, you can see the output
+>> from each of these functions, for example:
 >>
 >>```
 >>2021-03-05 13:19:38,184 [34706] INFO     diagnostic,83  Example of how to group and
@@ -317,16 +321,18 @@ script code][shared].
 
 ## Diagnostic computation
 
-After grouping and selecting data, we can read individual attributes such as the
-filename by looping over variables (line 89-93). Following this, we see the use
-of the function ``compute_diagnostic`` (line 94). Let's have a look at the
-definition of this function in line 43 where the analyses on the data is done.
+After grouping and selecting data, we can read individual attributes (such as filename) 
+of each item. Here we have grouped the input data  by ``variables`` 
+so we loop over the variables (line 89-93). Following this, is a call to the 
+function ``compute_diagnostic`` (line 94). Let's have a look at the
+definition of this function in line 43 where the actual analysis on the data is done.
 
+Note that output from the ESMValCore preprocessor is in the form of NetCDF files.
 Here, ``compute_diagnostic`` uses
 [Iris](https://scitools-iris.readthedocs.io/en/latest/index.html) to read data
-from a netCDF file and performs an operation ``squeeze`` to remove any dimension
+from a netCDF file and performs an operation ``squeeze`` to remove any dimensions
 of length one. We can adapt this function to add our own analysis. As an
-example, here we want to calculate the bias toward the averege of the data as:
+example, here we calculate the bias using the average of the data using Iris cubes. 
 
 ~~~python
 def compute_diagnostic(filename):
@@ -344,16 +350,16 @@ def compute_diagnostic(filename):
 
 > ## iris cubes
 >
-> Iris reads data into data structures called
+> Iris reads data from NetCDF files into data structures called
 > [cubes](https://scitools-iris.readthedocs.io/en/latest/userguide/iris_cubes.html).
 > The data in these cubes can be modified, combined with other cubes' data or
 > plotted.
 {: .callout}
 
 > ## Reading data using xarray
 >
-> Use the [xarrays](http://xarray.pydata.org/en/stable/) to read the data
-> instead of iris.
+> Alternately, you can use [xarrays](http://xarray.pydata.org/en/stable/) to read the data
+> instead of Iris.
 >
 >> ## Answer
 >>
@@ -381,17 +387,18 @@ def compute_diagnostic(filename):
 
 > ## Reading data using Scipy's netCDF library
 >
-> Use the [SciPy's netCDF library][netCDF] to read the data instead of iris.
+> Yet another option to read the NetCDF file data is to use 
+> the [SciPy's netCDF library][netCDF].
 >
 >> ## Answer
 >>
->> First, import ``netcdfx`` package at the top of the script as:
+>> First, import the ``netcdfx`` package at the top of the script as:
 >>
 >>~~~python
 >>from scipy.io import netcdfx
 >>~~~
 >>
->> Then, change the ``compute_diagnostic`` as:
+>> Then, change ``compute_diagnostic`` as:
 >>
 >>~~~python
 >>def compute_diagnostic(filename):
@@ -414,7 +421,7 @@ def compute_diagnostic(filename):
 Often, the end product of a diagnostic script is a plot or figure. The Iris cube
 returned from the ``compute_diagnostic`` function (line 94) is passed to the
 ``plot_diagnostic`` function (line 101). Let's have a look at the definition of
-this function in line 53 where we would plug in our plotting routine in the
+this function in line 53. This is where we would plug in our plotting routine in the
 diagnostic script.
 
 More specifically, the ``quickplot`` function (line 61) can be replaced with the
@@ -431,17 +438,17 @@ there:
           cmap: Reds
 ~~~
 
-In this way, we can further pass arguments for ``quickplot`` such as the type of
-plot ``pcolormesh`` and the colormap ``cmap:Reds`` from the recipe to the
-diagnostic.
+This way, we can pass arguments such as the type of
+plot ``pcolormesh`` and the colormap ``cmap:Reds`` from the recipe to the 
+``quickplot``  function in the diagnostic.
 
 > ## Passing arguments from the recipe to the diagnostic
 >
 > Change the type of the plot and its colormap and inspect the output figure.
 >
 >> ## Answer
 >>
->> In the recipe ``recipe_python.yml``, you should change ``plot_type`` and ``cmap``.
+>> In the recipe ``recipe_python.yml``, you could change ``plot_type`` and ``cmap``.
 >> As an example, we choose ``plot_type: pcolor`` and ``cmap: BuGn``:
 >>
 >> ~~~yaml
@@ -458,33 +465,35 @@ diagnostic.
 
 > ## ESMValTool gallery
 >
-> ESMValTool makes it possible to produce a wide array of such figures as seen
+> ESMValTool makes it possible to produce a wide array of plots and figures as seen
 > in the [gallery](https://docs.esmvaltool.org/en/latest/gallery.html).
 {: .callout}
 
 ### Saving the output
 
 In our example, the function ``save_data`` in line 57 is used to save the Iris
-cube. The saved files can be found under ``work`` directory in a ``.nc`` format.
-There is also the function ``save_figure`` in line 63 to save the plots under
-``plot`` directory in a ``.png`` format. Again, you may choose your own method
-of saving the output.
+cube. The saved files can be found under the ``work`` directory in a ``.nc`` format.
+There is also the function ``save_figure`` in line 63 to save the plots under the 
+``plot`` directory in a ``.png`` format (or preferred format specified in your 
+configuration settings). Again, you may choose your own method
+of saving the output. 
 
 ### Recording the provenance
 
-When developing a diagnostic script, we should make sure that it records the
+When developing a diagnostic script, it is good practice to record
 provenance. To do so, we use the function ``get_provenance_record`` (line 99).
-Let's have a look at the definition of this function in line 21 where we
+Let us have a look at the definition of this function in line 21 where we
 describe the diagnostic data and plot. Using the dictionary ``record``, it is
-possible to add custom provenance. Provenance is stored in the *W3C PROV XML*
+possible to add custom provenance to our diagnostics output. 
+Provenance is stored in the *W3C PROV XML*
 format and also in an *SVG* file under the ``work`` and ``plot`` directory. For
 more information, see [recording provenance][provenance].
 
 ## Congratulations!
 
-Now you know the diagnostic script structure and a few functions. There are many
-more functions to be used, but these should be enough to get you started! Look
-at other recipes and diagnostics for more examples.
+You now know the basic diagnostic script structure and some available tools for putting 
+together your own diagnostics. Have a look at existing recipes and diagnostics in the 
+repository for more examples of functions you can use in your diagnostics!
 {% include links.md %}
 
 [recipe]: https://github.com/ESMValGroup/ESMValTool/blob/master/esmvaltool/recipes/examples/recipe_python.yml