priyanka_review_update

Author: sdash77

guide/14-deep-learning/how_PSETAE_works.ipynb

@@ -1,16 +1,23 @@
 {
  "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# How PSETAE model works"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
     "## Table of Contents\n",
     "\n",
-    "* [Satellite Image Time-Series](#Satellite-Image-Time-Series)\n",
-    "* [Training Labels and Time-Series Satellite Imagery](#Training-Labels-and-Time-Series-Satellite-Imagery)\n",
-    "* [Time-Series Classification](#Time-Series-Classification)\n",
-    "  * [Pixel-Set Encoder](#Pixel-Set-Encoder)\n",
-    "  * [Temporal Attention Encoder](#Temporal-Attention-Encoder)\n",
+    "* [Satellite image time series](#Satellite-image-time-series)\n",
+    "* [Prepare training data for satellite image time series](#Prepare-training-data-for-satellite-image-time-series)\n",
+    "* [Architecture](#Architecture)\n",
+    "  * [Pixel-set encoder](#Pixel-set-encoder)\n",
+    "  * [Temporal attention encoder](#Temporal-attention-encoder)\n",
     "  * [Spectro-temporal classifier](#Spectro-temporal-classifier)\n",
     "* [Implementation in arcgis.learn](#Implementation-in-arcgis.learn) \n",
     "* [Summary](#Summary)\n",
@@ -21,14 +28,14 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Satellite Image Time-Series"
+    "## Satellite image time series"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Earth observation data cube or time-series is referred to as collection of satellite images of a location from different time-periods, stacked vertically resulting in a 3-dimensional structure. The collection have a common projection and a consistent timeline. Each location in the space-time is a vector of values across a timeline as shown in figure 1."
+    "Earth observation time series is referred to as collection of satellite images of a location from different time-periods, stacked vertically resulting in a 3-dimensional structure. The collection has a common projection and a consistent timeline. Each location in the space-time is a vector of values across a timeline as shown in figure 1:"
    ]
   },
   {
@@ -42,35 +49,35 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "<center> Figure 1. Time-series of satellite imagery </center>"
+    "<center> Figure 1. Time-series of satellite imagery. [2] </center>"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Combination of temporal component with spectral information allow for detection and classification of complex pattern in various applications such as crop type and condition classification, mineral and land-cover mapping etc"
+    "Combination of temporal component with spectral information allow for detection and classification of complex pattern in various applications such as crop type and condition classification, mineral and land-cover mapping etc."
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "In this guide, we will focus on PSETAE, a transformer based deep learning model originally developed by [Garnot et al](https://openaccess.thecvf.com/content_CVPR_2020/papers/Garnot_Satellite_Image_Time_Series_Classification_With_Pixel-Set_Encoders_and_Temporal_CVPR_2020_paper.pdf) for agricultural parcels classification into different crop types in satellite image time-series."
+    "In this guide, we will focus on `PSETAE`, a transformer based deep learning model originally developed by [Garnot et al](https://openaccess.thecvf.com/content_CVPR_2020/papers/Garnot_Satellite_Image_Time_Series_Classification_With_Pixel-Set_Encoders_and_Temporal_CVPR_2020_paper.pdf) for agricultural parcels classification into different crop types in satellite image time-series."
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Training Labels and Time-Series Satellite Imagery"
+    "## Prepare training data for satellite image time series"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "The [Export Training Data for Deep Learning](https://pro.arcgis.com/en/pro-app/latest/tool-reference/image-analyst/export-training-data-for-deep-learning.htm) is used to export training data for the model. The input satellite time-series is a [composite](https://pro.arcgis.com/en/pro-app/latest/tool-reference/data-management/composite-bands.htm) of rasters or [multi-dimensional raster](https://pro.arcgis.com/en/pro-app/latest/help/data/imagery/an-overview-of-multidimensional-raster-data.htm) from the required time periods or time steps. Here are the [steps](https://www.youtube.com/watch?v=HFbTFTnsMWM), to create multi-dimensional raster from collection of images. \n",
+    "The [Export Training Data for Deep Learning](https://pro.arcgis.com/en/pro-app/latest/tool-reference/image-analyst/export-training-data-for-deep-learning.htm) tool is used to export training data for the model. The input satellite time-series is a [composite](https://pro.arcgis.com/en/pro-app/latest/tool-reference/data-management/composite-bands.htm) of rasters or [multi-dimensional raster](https://pro.arcgis.com/en/pro-app/latest/help/data/imagery/an-overview-of-multidimensional-raster-data.htm) from the required time periods or time steps. Here are the [steps](https://www.youtube.com/watch?v=HFbTFTnsMWM), to create multi-dimensional raster from collection of images. \n",
     "\n",
     "Training labels can be created using the [Label objects for deep learning](https://pro.arcgis.com/en/pro-app/latest/help/analysis/image-analyst/label-objects-for-deep-learning.htm#:~:text=The%20Label%20Objects%20for%20Deep,is%20divided%20into%20two%20parts.) tool available inside `Classification Tools`. Pixels are labelled into different classes, based on the available information. Labelling of different crop types are shown in the figure. "
    ]
@@ -93,30 +100,23 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Time-Series Classification"
+    "## Architecture"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "PSETAE architecure is based on transfomers, originally developed for sequence-to-sequence modeling. The proposed architecture encodes time-series of multi-spectral images. The pixels under each class label is given by spectro-temporal tensor of size T x C x N, where T the number of temporal observation, C the number of spectral channels, and N the number of pixels.  \n",
+    "`PSETAE` model architecure is based on transfomers, originally developed for sequence-to-sequence modeling. The proposed architecture encodes time-series of multi-spectral images. The pixels under each class label is given by spectro-temporal tensor of size T x C x N, where T the number of temporal observation, C the number of spectral channels, and N the number of pixels.  \n",
     "\n",
-    "The architecture of PSETAE consists of a pixel-set encoder, temporal attention encoder and, classifier. The components are briefly described in following sections."
+    "The architecture of `PSETAE` model consists of a pixel-set encoder, temporal attention encoder and, classifier. The components are briefly described in following sections."
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Model architecture"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "### Pixel-Set Encoder"
+    "### Pixel-set encoder"
    ]
   },
   {
@@ -139,14 +139,14 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "<center> Figure 3. Pixel-Set Encoder </center> "
+    "<center> Figure 3. Pixel-Set Encoder. [1] </center> "
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Temporal Attention Encoder"
+    "### Temporal attention encoder"
    ]
   },
   {
@@ -171,7 +171,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "<center> Figure 4. Temporal Attention Encoder </center> "
+    "<center> Figure 4. Temporal Attention Encoder. [1] </center> "
    ]
   },
   {
@@ -194,7 +194,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Implementation in  `arcgis.learn`"
+    "## Implementation in `arcgis.learn`"
    ]
   },
   {
@@ -203,19 +203,17 @@
    "source": [
     "Input Raster - time-series raster is a [composite](https://pro.arcgis.com/en/pro-app/latest/tool-reference/data-management/composite-bands.htm) or [multi-dimensional](https://pro.arcgis.com/en/pro-app/latest/help/data/imagery/an-overview-of-multidimensional-raster-data.htm) raster from the required time periods or time steps. \n",
     "\n",
-    "Export - use the input raster to export the raster chips in `RCNN Masks` metadata format using [Export Training Data for Deep Learning](https://pro.arcgis.com/en/pro-app/latest/tool-reference/image-analyst/export-training-data-for-deep-learning.htm) tool available in `ArcGIS Pro`.\n",
-    "\n",
-    "The resulting path from from export tool is provided to [prepare_data](https://developers.arcgis.com/python/api-reference/arcgis.learn.toc.html#prepare-data) function in `arcgis.learn` to create a databunch.\n",
+    "Export - use the input raster to export the raster chips in `RCNN Masks` metadata format using [Export Training Data for Deep Learning](https://pro.arcgis.com/en/pro-app/latest/tool-reference/image-analyst/export-training-data-for-deep-learning.htm) tool available in `ArcGIS Pro`. The resulting path from from export tool is provided to [prepare_data](https://developers.arcgis.com/python/api-reference/arcgis.learn.toc.html#prepare-data) function in `arcgis.learn` to create a databunch.\n",
     "\n",
     "`data = prepare_data(path=r\"path/to/exported/data\", n_temporal, min_points, batch_size, n_temporal_dates, dataset_type=\"PSETAE\")`\n",
     "                    \n",
     "where,\n",
     "\n",
-    "* `n_temporal` - optional for multi-dimensional, required for composite raster. *Number of temporal observations or time steps or number of composited rasters*. \n",
-    "* `min_points` - optional. *Number of pixels equal to or multiples of 64 to sample from the each labelled region of training data i.e. 64, 128 etc*\n",
-    "* `batch_size` - optional. *suggested batch size for this model is around 128*\n",
-    "* `n_temporal_dates` - optional for multi-dimensional, required for composite raster. *The dates of the observations will be used for the positional encoding and should be stored as a list of date strings in YYYY-MM-DD format*.\n",
-    "* `dataset_type` - required. *type of dataset in-sync with the model*"
+    "- `n_temporal` - Optional for multi-dimensional, required for composite raster. Number of temporal observations or time steps or number of composited rasters. \n",
+    "- `min_points` - Optional. Number of pixels equal to or multiples of 64 to sample from the each labelled region of training data i.e. 64, 128, etc.\n",
+    "- `batch_size` - Optional. Suggested batch size for this model is around 128.\n",
+    "- `n_temporal_dates` - Optional for multi-dimensional, required for composite raster. The dates of the observations will be used for the positional encoding and should be stored as a list of date strings in YYYY-MM-DD format.\n",
+    "- `dataset_type` - Required. Type of dataset in-sync with the model."
    ]
   },
   {
@@ -224,7 +222,7 @@
    "source": [
     "By default, initialization of the `PSETAE` object as shown below:\n",
     "\n",
-    "`model = arcgis.learn.PSETAE(data=data)`\n",
+    "`model = arcgis.learn.PSETAE(data)`\n",
     "\n",
     "model parameters that can be passed using keyword arguments:\n",
     "\n",
@@ -262,7 +260,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
    "name": "python3"
   },
@@ -276,7 +274,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.7.11"
+   "version": "3.9.15"
   },
   "toc": {
    "base_numbering": 1,