How AutoDL works guide enhancements

Author: SurajBaloni

guide/14-deep-learning/how_AutoDL_works.ipynb

@@ -71,6 +71,22 @@
     "- CascadeRCNN\n",
     "- CascadeRPN\n",
     "- DCN\n",
+    "- Detectors\n",
+    "- DoubleHeads\n",
+    "- DynamicRCNN\n",
+    "- EmpiricalAttention\n",
+    "- FCOS\n",
+    "- FoveaBox\n",
+    "- FSAF\n",
+    "- GHM\n",
+    "- LibraRCNN\n",
+    "- PaFPN\n",
+    "- PISA\n",
+    "- RegNet\n",
+    "- RepPoints\n",
+    "- Res2Net\n",
+    "- SABL\n",
+    "- VFNet\n",
     "\n",
     "**Pixel Classification**\n",
     "\n",
@@ -81,7 +97,20 @@
     "- APCNet\n",
     "- CCNet\n",
     "- CGNet\n",
-    "- HRNet\n"
+    "- HRNet\n",
+    "- DeepLabV3Plus\n",
+    "- DMNet\n",
+    "- DNLNet\n",
+    "- EMANet\n",
+    "- FastSCNN\n",
+    "- FCN\n",
+    "- GCNet\n",
+    "- MobileNetV2\n",
+    "- NonLocalNet\n",
+    "- OCRNet\n",
+    "- PSANet\n",
+    "- SemFPN\n",
+    "- UperNet\n"
    ]
   },
   {
@@ -128,7 +157,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Prepare data for `AutoDL` class using `prepare_data()` in `arcgis.learn`"
+    "Prepare the data for `AutoDL` class using `prepare_data()` in `arcgis.learn`, the recommended value for the batch_size parameter is `None` as `AutoDL` class supports automatic evaluation of the batch_size based on the GPU capacity."
    ]
   },
   {
@@ -137,7 +166,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "data = prepare_data(\"path_to_data_folder\", batch_size=2)"
+    "data = prepare_data(\"path_to_data_folder\", batch_size=None)"
    ]
   },
   {
@@ -169,9 +198,10 @@
     "    The list of models that will be used in the training process. If the user does not provide the parameter value, the `AutoDL` class selects all of the supported networks, however the user can select one or more networks by passing the network names as string in a list.\n",
     "\n",
     "    - _Supported Object Detection models_\n",
-    "        - SingleShotDetector, RetinaNet, FasterRCNN, YOLOv3, ATSS, CARAFE, CascadeRCNN, CascadeRPN, DCN\n",
+    "        - SingleShotDetector, RetinaNet, FasterRCNN, YOLOv3, ATSS, CARAFE, CascadeRCNN, CascadeRPN, DCN, Detectors, DoubleHeads, DynamicRCNN, EmpiricalAttention, FCOS, FoveaBox, FSAF, GHM, LibraRCNN, PaFPN, PISA, RegNet, RepPoints, Res2Net, SABL, VFNet\n",
+    "        \n",
     "    - _Supported Object Detection models_\n",
-    "        - DeepLab, UnetClassifier, PSPNetClassifier, ANN, APCNet, CCNet, CGNet, HRNet\n",
+    "        - DeepLab, UnetClassifier, PSPNetClassifier, ANN, APCNet, CCNet, CGNet, HRNet, DeepLabV3Plus, DMNet, DNLNet, EMANet, FastSCNN, FCN, GCNet, MobileNetV2, NonLocalNet, OCRNet, PSANet, SemFPN, UperNet\n",
     "        \n",
     "\n",
     "- `verbose` (Optional Parameter): Optional Boolean.\n",
@@ -191,11 +221,12 @@
    "metadata": {},
    "source": [
     "- **Basic**\n",
-    "    - In this mode we iterate through all of the supported networks once with the default backbone, train it with the passed data, and calculate the network performance. At the end of each iteration, the function will save the model to the disk. The maximum number of epochs to train each network is 20, however, if the remaining time left to process the network is less than than the expected time (minimum time required to train the network), the program will automatically calculate the maximum number of epochs to train the network.\n",
+    "    - In this mode we iterate through all of the supported networks once with the default backbone, train it with the passed data, and calculate the network performance. At the end of each iteration, the function will save the model to the disk. Based on the alloted time, the program will automatically calculate the maximum number of epochs to train each network. However,the training will stop if the model stops improving for 5 epochs. A minimum difference of 0.001 in the validation loss is required for it to be considered as an improvement.\n",
     "    \n",
     "    \n",
     "- **Advanced**\n",
-    "    - To be used when the user wants to tune the hyper parameters of two best performing networks from the basic mode. This mode will divide the total time into two halves. In the first half, it works like basic mode, where it will iterate through all of the supported networks once. In the second half, it checks for the two best performing networks. The program then trains the selected networks with different supported backbones. At the end of each iteration, the function will save the model to the disk. The maximum number of epochs to train each network is 20, however, if the remaining time left to process the network is less than the expected time (minimum time required to train the network), the program will automatically calculate the number of epochs to train the network.\n",
+    "    - To be used when the user wants to tune the hyper-parameters of two best performing networks from the basic mode. This mode will divide the total time into two halves. In the first half, it works like basic mode, where it will iterate through all of the supported networks once. In the second half, it checks for the two best performing networks. The program then trains the selected networks with different supported backbones. At the end of each iteration, the function will save the model to the disk. Based on the alloted time, the program will automatically calculate the maximum number of epochs to train each network. However,the training will stop if the model stops improving for 5 epochs. A minimum difference of 0.001 in the validation loss is required for it to be considered as an improvement. \n",
+    "    - In this mode we use `optuna` to tune the hyper-paramaeters of the network. \n",
     " "
    ]
   },
@@ -212,7 +243,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "<p>When the <code>AutoDL</code> class is initialized, it calculates the number of images that can be processed in the given time and the time required to process the all of the data. The output of the cell above can then be used to analyze and update the <code>total_time_limit</code> and <code>networks</code> parameters while initializing the class. </p><p><strong>Here is an example of the output</strong> </p><ul><li>Given time to process the dataset is: 5.0 hours</li><li>Number of images that can be processed in the given time: 290</li><li>Time required to process the entire dataset of 3000 images is 52 hours</li></ul><p>This explains how many images can be processed to train all of the selected networks in the selected mode within the given time, as well as provides an estimate of the time that <code>AutoDL</code> will take to train all of the selected networks with the entire dataset.</p>"
+    "<p>When the <code>AutoDL</code> class is initialized, it calculates the number of images that can be processed in the given time and the time required to process the all of the data. The output of the cell above can then be used to analyze and update the <code>total_time_limit</code> and <code>networks</code> parameters while initializing the class. </p><p><strong>Here is an example of the output</strong> </p><ul><li>Given time to process the dataset is: 5.0 hours</li><li>Number of images that can be processed in the given time: 290</li><li>Time required to process the entire dataset of 3000 images is 52 hours</li></ul><p>This explains how many images can be processed to train all of the selected networks in the selected mode within the given time, as well as it provides an estimate of the time that <code>AutoDL</code> will take to train all of the selected networks with the entire dataset.</p>"
    ]
   },
   {
@@ -319,6 +350,30 @@
     "dl.score()"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Report \n",
+    "<a id='report'></a>"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "dl.report()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "This method will return an advanced html report of the networks trained by `AutoDL`. In the `basic` mode it shows the leaderboard of all the networks based on their performance during the training phase, and some important parameter details and charts for the best evaluated model. Additionally, in the `advanced` mode it shows details of all the `optuna` based trails with the hyper-tuned parameter details and the feature importance chart for each of the model evaluated during the `advanced` mode."
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -428,7 +483,7 @@
    "metadata": {},
    "source": [
     "- The load method is used to load a saved model from the disk using the `AutoDL` class. It accepts the following parameters:\n",
-    "    - path: Path to the ESRI Model Definition (.emd) file\n",
+    "    - path: Path to the ESRI Model Definition (emd or dlpk) file\n",
     "    - data: Returned data object from `prepare_data` function"
    ]
   },
@@ -552,7 +607,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.15"
+   "version": "3.9.16"
   }
  },
  "nbformat": 4,