model: pytorch: Add custom Neural Network & layer support and loss function entrypoints

Fixes: #794

Author: sakshamarora1

CHANGELOG.md

@@ -6,6 +6,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 ### Added
+- CLI and Python example usage of Custom Neural Network
+- PyTorch loss function entrypoint style loading
+- Custom Neural Network, last layer support for pre-trained models
 - Example usage of sklearn operations
 - Example Flower17 species image classification
 - Configloading ablity from CLI using "@" before filename

docs/tutorials/index.rst

@@ -13,3 +13,4 @@ or operations.
     operations
     sources/index
     dataflows/index
+    neuralnetwork

docs/tutorials/neuralnetwork.rst

@@ -0,0 +1,108 @@
+Custom Neural Network Tutorial
+==============================
+
+Rock Paper Scissors Hand Pose Classification
+--------------------------------------------
+
+This tutorial will show you how to train and test a PyTorch based custom neural network model made using DFFML.
+The dataset we will be using is the `rock-paper-scissors-dataset <http://www.laurencemoroney.com/rock-paper-scissors-dataset/>`_
+which contains images of hands in Rock/Paper/Scissors poses, each image is a 300x300 RGB image.
+
+The model we'll be using is :ref:`PyTorchNeuralNetwork <plugin_model_dffml_model_pytorch_pytorchnet>`
+which is a part of ``dffml-model-pytorch``, a DFFML plugin which allows you to use PyTorch
+via DFFML. We can install it with ``pip``. We will also be using image loading from
+``dffml-config-image`` and YAML file loading from ``dffml-config-yaml`` for creating our neural network.
+
+.. code-block:: console
+
+    $ pip install -U dffml-model-pytorch dffml-config-image dffml-config-yaml
+
+Download the dataset and verify with with ``sha384sum``.
+
+.. literalinclude:: /../examples/rockpaperscissors/dataset.sh
+
+.. code-block:: console
+
+    rps.zip: OK
+    rps-test-set.zip: OK
+    rps-validation.zip: OK
+
+Extract the datasets.
+
+.. code-block:: console
+
+    $ unzip rps.zip
+    $ unzip rps-test-set.zip
+    $ unzip rps-validation.zip -d rps-predict
+
+The dataset for training the model will be in the `rps` directory.
+The dataset for testing the model will be in the `rps-test-set` directory.
+The images we will be using for prediction on the neural network will be in the `rps-predict` directory.
+
+Now that we have our dataset ready, we can perform classification of the hand poses to predict whether
+it is rock, paper or scissors!
+
+We first create the neural network.
+
+The neural network can be created in 2 ways using DFFML:
+    1. By creating a dictionary of layers in YAML or JSON format passing the file via CLI (eg. @model.yaml).
+    2. By using the torch module to create the model and passing an instance of the network to the model config.
+
+
+Command Line
+------------
+
+We first create a YAML file to define the neural network with all the information about the layers along with
+the forward method which is passed as list of layers under the model name key:
+
+**model.yaml**
+
+.. literalinclude:: /../examples/rockpaperscissors/model.yaml
+
+.. seealso::
+    Sequential layers can also be created by indenting the layers under a key!
+    The layers definied inside the Sequential layer can be used again while defining the forward method in the
+    following syntax: `- key1.layer2`
+
+.. Note::
+    If the forward method is not given in the YAML file, forward method is automatically created by appending
+    the top level layers sequentially.
+
+Train the model.
+
+.. literalinclude:: /../examples/rockpaperscissors/train.sh
+
+.. code-block:: console
+
+    INFO:dffml.PyTorchNeuralNetworkContext:Training complete in 1m 42s
+    INFO:dffml.PyTorchNeuralNetworkContext:Best Validation Accuracy: 1.000000
+
+Assess the model's accuracy.
+
+.. literalinclude:: /../examples/rockpaperscissors/accuracy.sh
+
+The output is:
+
+.. code-block:: console
+
+    0.8763440860215054
+
+Predict with the trained model.
+
+.. literalinclude:: /../examples/rockpaperscissors/predict.sh
+
+Some of the Predictions:
+
+.. literalinclude:: /../examples/rockpaperscissors/output.txt
+
+
+Python API
+----------
+
+.. literalinclude:: /../examples/rockpaperscissors/python_example.py
+
+The output will be as follows:
+
+.. literalinclude:: /../examples/rockpaperscissors/python_output.txt
+
+The model predicts the hand poses correctly with great confidence!

docs/usage/flower17/flower17_pytorch.rst

@@ -4,11 +4,11 @@ FLOWER17 Species classification using Transfer Learning
 The model we'll be using is :ref:`AlexNet CNN Model <plugin_model_dffml_model_pytorch_alexnet>`
 which is a part of ``dffml-model-pytorch``, a DFFML plugin which allows you to use PyTorch
 via DFFML. We can install it with ``pip``. We will also be using image loading from
-``dffml-config-image``.
+``dffml-config-image`` and YAML file loading from ``dffml-config-yaml``.
 
 .. code-block:: console
 
-    $ pip install -U dffml-model-pytorch dffml-config-image
+    $ pip install -U dffml-model-pytorch dffml-config-yaml dffml-config-image
 
 There are 2 ways to perform Transfer Learning:
 
@@ -23,6 +23,10 @@ There are 2 ways to perform Transfer Learning:
 
 In this example, we will be fine-tuning the AlexNet model. (We set `trainable=True`)
 
+We first create a YAML file to define the last layer(s) to replace from the network architecture:
+
+.. literalinclude:: /../examples/flower17/pytorch-alexnet/layers.yaml
+
 Train the model.
 
 .. literalinclude:: /../examples/flower17/pytorch-alexnet/train.sh

examples/flower17/pytorch-alexnet/accuracy.sh

@@ -1,12 +1,17 @@
 dffml accuracy \
   -model alexnet \
+  -model-add_layers \
+  -model-layers @layers.yaml \
   -model-clstype str \
   -model-classifications \
     crocus windflower fritillary tulip pansy dandelion tigerlily sunflower \
     bluebell cowslip coltsfoot snowdrop daffodil lilyvalley iris buttercup daisy \
   -model-directory alexnet_model \
   -model-imageSize 224 \
+  -model-trainable \
   -model-enableGPU \
+  -model-normalize_mean 0.485 0.456 0.406 \
+  -model-normalize_std 0.229 0.224 0.225 \
   -model-features image:int:$((500*500)) \
   -model-predict label:str:1 \
   -sources f=dir \

examples/flower17/pytorch-alexnet/layers.yaml

@@ -0,0 +1,16 @@
+linear1:
+  name: Linear
+  in_features: 4096
+  out_features: 256
+relu:
+  name: ReLU
+dropout:
+  name: Dropout
+  p: 0.2
+linear2:
+  name: Linear
+  in_features: 256
+  out_features: 17
+logsoftmax:
+  name: LogSoftmax
+  dim: 1

examples/flower17/pytorch-alexnet/predict.sh

@@ -1,12 +1,17 @@
 dffml predict all \
   -model alexnet \
+  -model-add_layers \
+  -model-layers @layers.yaml \
   -model-clstype str \
   -model-classifications \
     crocus windflower fritillary tulip pansy dandelion tigerlily sunflower \
     bluebell cowslip coltsfoot snowdrop daffodil lilyvalley iris buttercup daisy \
   -model-directory alexnet_model \
   -model-imageSize 224 \
+  -model-trainable \
   -model-enableGPU \
+  -model-normalize_mean 0.485 0.456 0.406 \
+  -model-normalize_std 0.229 0.224 0.225 \
   -model-features image:int:$((500*500)) \
   -model-predict label:str:1 \
   -sources f=csv \

examples/flower17/pytorch-alexnet/train.sh

@@ -1,5 +1,7 @@
 dffml train \
   -model alexnet \
+  -model-add_layers \
+  -model-layers @layers.yaml \
   -model-clstype str \
   -model-classifications \
     crocus windflower fritillary tulip pansy dandelion tigerlily sunflower \
@@ -11,6 +13,8 @@ dffml train \
   -model-validation_split 0.2 \
   -model-trainable \
   -model-enableGPU \
+  -model-normalize_mean 0.485 0.456 0.406 \
+  -model-normalize_std 0.229 0.224 0.225 \
   -model-features image:int:$((500*500)) \
   -model-predict label:str:1 \
   -sources f=dir \

examples/flower17/test_flower17.py

@@ -62,10 +62,20 @@ async def test_shell_sklearn(self):
 
     async def test_shell_pytorch(self):
         with directory_with_data_files() as tempdir:
-            # Run training
-            subprocess.check_output(
-                ["bash", sh_filepath("pytorch-alexnet/train.sh")]
+            shutil.copy(
+                sh_filepath("pytorch-alexnet/layers.yaml"),
+                os.path.join(os.getcwd(), "layers.yaml"),
             )
+
+            # Run training
+            with open(sh_filepath("pytorch-alexnet/train.sh"), "r") as cmd:
+                cmd = cmd.read()
+                cmd = cmd.replace("\n", "")
+                cmd = cmd.replace("\\", "")
+                cmd = shlex.split(cmd)
+                cmd[cmd.index("-model-epochs") + 1] = "1"
+            await CLI.cli(*cmd[1:])
+
             # Check the Accuracy
             stdout = subprocess.check_output(
                 ["bash", sh_filepath("pytorch-alexnet/accuracy.sh")]
@@ -79,8 +89,7 @@ async def test_shell_pytorch(self):
                 cmd = cmd.replace("\\", "")
                 cmd = shlex.split(cmd)
                 # When passing to CLI, remove first argument(dffml) and -pretty at the end
-                cmd = cmd[1:-1]
-                records = await CLI.cli(*cmd)
+                records = await CLI.cli(*cmd[1:-1])
 
                 # Check the label for 1 record
                 self.assertIsInstance(

examples/rockpaperscissors/accuracy.sh

@@ -0,0 +1,14 @@
+dffml accuracy \
+  -model pytorchnet \
+  -model-features image:int:$((300*300*3)) \
+  -model-clstype str \
+  -model-classifications rock paper scissors \
+  -model-predict label:int:1 \
+  -model-network @model.yaml \
+  -model-directory rps_model \
+  -model-imageSize 150 \
+  -model-enableGPU \
+  -sources f=dir \
+    -source-foldername rps-test-set \
+    -source-feature image \
+    -source-labels rock paper scissors