large docs update

Author: C-Achard

docs/index.rst

@@ -10,6 +10,7 @@ Welcome to napari-cellseg-annotator's documentation!
     res/inference_module_guide
     res/training_module_guide
     res/cropping_module_guide
+    res/custom_model_template
 
 
 .. toctree::
@@ -24,6 +25,7 @@ Welcome to napari-cellseg-annotator's documentation!
     res/launch_review
     res/model_framework
     res/model_workers
+    res/model_instance_seg
     res/plugin_model_inference
     res/plugin_model_training
     res/utils

docs/res/cropping_module_guide.rst

@@ -46,7 +46,7 @@ you **change the position** of the cropped volumes and labels in the x,y and z p
 
 .. note::
     When you are done you can save the cropped volume and labels directly with the
-    **Quicksave** button on the lower left, which will save in the folder you picked the image from, or as
+    **Quicksave** button on the lower left, which will save in the folder Fyou picked the image from, or as
     a separate folder if you loaded a folder as a stack.
     If you want more options (name, format) you can save by selecting the layer and then
     using **File -> Save selected layer**, or simply **CTRL+S** once you have selected the correct layer.

docs/res/custom_model_template.rst

@@ -0,0 +1,37 @@
+.. _custom_model_guide:
+
+Advanced : Declaring a custom model
+=============================================
+
+To add a custom model, you will need a **.py** file with the following structure to be placed in the *src/napari_cellseg_annotator/models* folder:
+
+
+::
+
+    def get_net():
+        return ModelClass # should return the class of the model,
+        # for example SegResNet or UNET
+
+
+    def get_weights_file():
+        return "weights_file.pth" # name of the weights file for the model,
+        # which should be in *src/napari_cellseg_annotator/models/saved_weights*
+
+
+    def get_output(model, input):
+        out = model(input) # should return the model's output as [C, N, D,H,W]
+        # (C: channel, N, batch size, D,H,W : depth, height, width)
+        return out
+
+
+    def get_validation(model, val_inputs):
+        val_outputs = model(val_inputs) # should return the proper type for validation
+        # with single_window_inference from MONAI
+        return val_outputs
+
+
+    def ModelClass(x1,x2...):
+        # your Pytorch model here...
+        return results
+
+

docs/res/images/cropping_process_example.png

@@ -13,7 +13,7 @@ Model         Link to original paper
 ===========   ================================================================================================
 VNet          `Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation`_
 SegResNet     `3D MRI brain tumor segmentation using autoencoder regularization`_
-TRAILMAP      An emulation in Pytorch of the `TRAIlMAP project on GitHub`_
+TRAILMAP      An emulation in Pytorch of the `TRAILMAP project on GitHub`_
 ===========   ================================================================================================
 
 .. _Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation: https://arxiv.org/pdf/1606.04797.pdf
@@ -23,38 +23,58 @@ TRAILMAP      An emulation in Pytorch of the `TRAIlMAP project on GitHub`_
 Interface and functionalities
 --------------------------------
 
-When launching the module, you will be asked to provide an image folder containing all the volumes you'd like to be labeled.
-All images with the chosen (**.tif** or **.tiff** currently supported) extension in this folder will be labeled.
-You can then choose an output folder, where all the results will be saved.
+.. image:: images/inference_plugin_layout.png
+    :align: right
 
-.. note::
-    | The files will be saved using the following format :
-    |    ``{original_name}_{model}_{date & time}_pred{id}.file_ext``
-    | For example, using a VNet on the third image of a folder, called "volume_1.tif" will yield :
-    |   *volume_1_VNet_2022_04_06_15_49_42_pred3.tif*
+* Loading data : When launching the module, you will be asked to provide an image folder containing all the volumes you'd like to be labeled.
+  All images with the chosen (**.tif** or **.tiff** currently supported) extension in this folder will be labeled.
+  You can then choose an output folder, where all the results will be saved.
+
+
+
+* Model choice : You can then choose one of the selected models above, which will be used for inference.
+
+
+
+* Anisotropy :If you want to see your results without anisotropy when you have anisotropic images, you can specify that you have anisotropic data
+  and set the resolution of your image in micron, this wil save & show the results without anisotropy.
+
+
+
+* Thresholding : You can perform thresholding to binarize your labels, all values beneath the confidence threshold will be set to 0 using this.
+  If you wish to use instance segmentation it is recommended to use threshlding.
 
-You can then choose one of the selected models above, which will be used for inference.
 
-If you want to see your results without anisotropy when you have anistropic images, you can specify that you have anisotropic data
-and set the resolution of your image in micron, this wil save & show the results without anisotropy.
 
-You can perform thresholding to binarize your labels, all values beneath the confidence threshold will be set to 0 using this.
-If you wish to use instance segmentation it is recommended to use threshlding.
+* Viewing results : You can also select whether you'd like to see the results in napari afterwards; by default the first image processed will be displayed,
+  but you can choose to display up to ten at once. You can also request to see the originals.
+
+
 
-You can also select whether you'd like to see the results in napari afterwards; by default the first image processed will be displayed,
-but you can choose to display up to ten at once. You can also request to see the originals.
 
 When you are done choosing your parameters, you can press the **Start** button to begin the inference process.
 Once it has finished, results will be saved then displayed in napari; each output will be paired with its original.
+On the left side, a progress bar and a log will keep you informed on the process.
+
+
+
+.. note::
+    | The files will be saved using the following format :
+    |    ``{original_name}_{model}_{date & time}_pred{id}.file_ext``
+    | For example, using a VNet on the third image of a folder, called "somatomotor.tif" will yield the following name :
+    |   *somatomotor_VNet_2022_04_06_15_49_42_pred3.tif*
+
 
 .. hint::
-    | **Results** will be displayed using the **twilight shifted** colormap if raw or **turbo** if thresholded,
-    | whereas the **original** image will be shown in the **inferno** colormap.
+    | **Results** will be displayed using the **twilight shifted** colormap if raw or **turbo** if thresholding has been applied, whereas the **original** image will be shown in the **inferno** colormap.
     | Feel free to change the **colormap** or **contrast** when viewing results to ensure you can properly see the labels.
     | You'll most likely want to use **3D view** and **grid mode** in napari when checking results more broadly.
 
 .. image:: images/inference_results_example.png
 
+.. note::
+    You can save the log after the worker is finished to easily remember which parameters you ran inference with.
+
 Source code
 --------------------------------
 * :doc:`plugin_model_inference`

docs/res/images/inference_plugin_layout.png

@@ -1,4 +1,4 @@
-model_workers.py
+model_instance_seg.py
 ===========================================
 
 

docs/res/images/inference_results_example.png

@@ -6,21 +6,38 @@ Training module guide
 This module allows you to train pre-defined Pytorch models for cell segmentation.
 Pre-defined models are stored in napari-cellseg-annotator/models.
 
+.. important::
+    The machine learning models used by this program require all images of a dataset to all be of the same size.
+    Please ensure that all the images you are loading are of the **same size**, or to use the **"extract patches" (in augmentation tab)** with an appropriately small size
+    to ensure all images being used are of a proper size.
+
 The training module is comprised of several tabs.
-The first one will let you choose :
+
+
+1) The first one, **Data**, will let you choose :
 
 * The images folder
 * The labels folder
-* The model
-* The number of samples to extract from each of your image to ensure correct size and perform data augmentation.
 
-The second lets you define training parameters such as :
+2) The second tab, **Augmentation**, lets you define dataset and augmentation parameters such as :
+
+* Whether to use images "as is" (**requires all images to be of the same size and cubic**) or extract patches.
+
+* If you're extracting patches :
+    * The size of patches to be extracted (ideally, please use a value **close to a pwoer of two**, such as 120 or 60.
+    * The number of samples to extract from each of your image to ensure correct size and perform data augmentation. A larger number will likely mean better performances, but longer training and larger memory usage.
+* Whether to perform data augmentation or not (elastic deforms, intensity shifts. random flipping,etc). A rule of thumb for augmentation is :
+    * If you're using the patch extraction method, enable it if you have more than 10 samples per image with at least 5 images
+    * If you have a large dataset and are not using patches extraction, enable it.
 
-* The loss function used for training
-* The batch size
-* The number of epochs
+
+3) The third contains training related parameters :
+* The model to use for training
+* The loss function used for training (see table below)
+* The batch size (larger means quicker training and possibly better performance but increased memory usage)
+* The number of epochs (a possibility is to start with 60 epochs, and decrease or increase depending on performance.)
 * The epoch interval for validation (for example, if set to two, the module will use the validation dataset to evaluate the model with the dice metric every two epochs.)
-  If the dice metric is better on that validation interval, the model weights will be saved in the results folder.
+If the dice metric is better on that validation interval, the model weights will be saved in the results folder.
 
 The available loss functions are :
 
@@ -43,12 +60,18 @@ Tversky loss              `Tversky Loss from MONAI`_ with ``sigmoid=true``
 .. _Tversky Loss from MONAI: https://docs.monai.io/en/stable/losses.html#tverskyloss
 
 Once you are ready, press the Start button to begin training. The module will automatically load your dataset,
-perform data augmentation, select a CUDA device if one is present, and train the model.
+perform data augmentation if you chose to, select a CUDA device if one is present, and train the model.
 
 .. note::
     You can stop the training at any time by clicking on the start button again.
 
     **The training will stop after the next validation interval is performed, to save the latest model should it be better.**
 
+.. note::
+    You can save the log to record the losses and validation metrics numerical value at each step. This log is autosaved as well when training completes.
+
 After two validations steps have been performed, the training loss values and validation metrics will be automatically plotted
 and shown on napari every time a validation step completes.
+This plot automatically saved each time validation is performed for now. The final version is stored separately in the results folder.
+
+