changed csv format for easier access + updated plots + moved stats to worker + tooltips + trying to fix readme

Author: C-Achard

.github/workflows/test_and_deploy.yml

@@ -23,7 +23,7 @@
 #    strategy:
 #      matrix:
 #        platform: [windows-latest, macos-latest, ubuntu-latest]
-#        python-version: [3.9]
+#        python-version: [3.8, 3.9]
 #
 #    steps:
 #      - uses: actions/checkout@v2

README.md

@@ -1,13 +1,13 @@
 # napari-cellseg3d
 
-[![License](https://img.shields.io/pypi/l/napari-cellseg3d.svg?color=green)](https://github.com/C_Achard/napari-cellseg3d/raw/main/LICENSE)
-[![PyPI](https://img.shields.io/pypi/v/napari-cellseg3d.svg?color=green)](https://pypi.org/project/napari-cellseg3d)
+[![License](https://img.shields.io/pypi/l/napari-cellseg3d.svg?color=green)](https://github.com/C_Achard/napari-cellseg-annotator/raw/main/LICENSE)
+[![PyPI](https://img.shields.io/pypi/v/napari-cellseg3d.svg?color=green)](https://pypi.org/project/napari-cellseg-annotator)
 [![Python Version](https://img.shields.io/pypi/pyversions/napari-cellseg3d.svg?color=green)](https://python.org)
-[![tests](https://github.com/C_Achard/napari-cellseg3d/workflows/tests/badge.svg)](https://github.com/C_Achard/napari-cellseg3d/actions)
-[![codecov](https://codecov.io/gh/C_Achard/napari-cellseg3d/branch/main/graph/badge.svg)](https://codecov.io/gh/C_Achard/napari-cellseg3d)
-[![napari hub](https://img.shields.io/endpoint?url=https://api.napari-hub.org/shields/napari-cellseg3d)](https://napari-hub.org/plugins/napari-cellseg3d)
+[![tests](https://github.com/C_Achard/napari-cellseg3d/workflows/tests/badge.svg)](https://github.com/C_Achard/napari-cellseg-annotator/actions)
+[![codecov](https://codecov.io/gh/C_Achard/napari-cellseg3d/branch/main/graph/badge.svg)](https://codecov.io/gh/C_Achard/napari-cellseg-annotator)
+[![napari hub](https://img.shields.io/endpoint?url=https://api.napari-hub.org/shields/napari-cellseg3d)](https://napari-hub.org/plugins/cellseg-annotator-test)
 
-plugin for cell segmentation
+napari plugin providing cell segmentation tools : training, inference, review...
 
 ----------------------------------
 
@@ -23,8 +23,11 @@ https://napari.org/plugins/stable/index.html
 
 ## Requirements
 
-Requires manual installation of pytorch and MONAI.
-For Pytorch, please see [PyTorch]'s website for installation instructions.
+**Python >= 3.8 required**
+
+Requires manual installation of **pytorch** and **MONAI**.
+
+For Pytorch, please see [PyTorch's website for installation instructions].
 A CUDA-capable GPU is not needed but very strongly recommended, especially for training.
 
 If you get errors from MONAI regarding missing readers, please see [MONAI's optional dependencies] page for instructions on getting the readers required by your images.
@@ -44,7 +47,7 @@ pip install -e .
 
 ## Documentation
 
-You can generate docs by running ``make html`` in the docs folder
+You can generate docs by running ``make html`` in the *docs* folder
 
 ## Usage
 
@@ -97,5 +100,5 @@ If you encounter any problems, please [file an issue] along with a detailed desc
 [pip]: https://pypi.org/project/pip/
 [PyPI]: https://pypi.org/
 
-[PyTorch]: https://pytorch.org/get-started/locally/
+[PyTorch's website for installation instructions]: https://pytorch.org/get-started/locally/
 [MONAI's optional dependencies]: https://docs.monai.io/en/stable/installation.html#installing-the-recommended-dependencies

docs/res/guides/inference_module_guide.rst

@@ -40,10 +40,7 @@ Interface and functionalities
 
   | You can then choose one of the provided **models** above, which will be used for inference.
   | You may also choose to **load custom weights** rather than the pre-trained ones, simply ensure they are **compatible** (e.g. produced from the training module for the same model)
-
-.. note::
-    Currently the SegResNet model requires you to provide the size of the images the model was trained with due to the VAE module.
-    Provided weights use a size of 128, please leave it as is if you're not using custom weights.
+  |  If you choose to use a SegResNet with custom weights, you will have to provide the size of images it was trained on to ensure compatibility. (See note below)
 
 * **Inference parameters** :
 
@@ -106,6 +103,10 @@ Once it has finished, results will be saved then displayed in napari; each outpu
 On the left side, a progress bar and a log will keep you informed on the process.
 
 
+.. note::
+    Currently the SegResNet model requires you to provide the size of the images the model was trained with due to the VAE module.
+    Provided weights use a size of 128, please leave it as is if you're not using custom weights.
+
 
 .. note::
     | The files will be saved using the following format :

napari_cellseg3d/model_instance_seg.py

@@ -3,6 +3,7 @@
 
 import numpy as np
 from skimage.measure import label
+
 # from skimage.measure import marching_cubes
 # from skimage.measure import mesh_surface_area
 from skimage.measure import regionprops
@@ -13,6 +14,7 @@
 
 from napari_cellseg3d.utils import fill_list_in_between
 from napari_cellseg3d.utils import sphericity_axis
+
 # from napari_cellseg3d.utils import sphericity_volume_area
 
 
@@ -189,7 +191,7 @@ def volume_stats(volume_image):
 
     Returns:
         dict: Statistics described above
-        """
+    """
 
     properties = regionprops(volume_image)
     number_objects = np.amax(volume_image)
@@ -218,7 +220,9 @@ def fill(lst, n=len(properties) - 1):
 
     return {
         "Volume": volume,
-        "Centroid": [region.centroid for region in properties],
+        "Centroid x": [region.centroid[0] for region in properties],
+        "Centroid y": [region.centroid[1] for region in properties],
+        "Centroid z": [region.centroid[2] for region in properties],
         # "Sphericity (volume/area)": sphericity_va,
         "Sphericity (axes)": sphericity_ax,
         "Image size": fill([volume_image.shape]),

napari_cellseg3d/model_workers.py

@@ -44,6 +44,7 @@
 # local
 from napari_cellseg3d.model_instance_seg import binary_connected
 from napari_cellseg3d.model_instance_seg import binary_watershed
+from napari_cellseg3d.model_instance_seg import volume_stats
 
 """
 Writing something to log messages from outside the main thread is rather problematic (plenty of silent crashes...)
@@ -92,6 +93,7 @@ def __init__(
         use_window,
         window_infer_size,
         keep_on_cpu,
+        stats_csv,
     ):
         """Initializes a worker for inference with the arguments needed by the :py:func:`~inference` function.
 
@@ -118,6 +120,8 @@ def __init__(
 
             * keep_on_cpu: keep images on CPU or no
 
+            * stats_csv: compute stats on cells and save them to a csv file
+
         Note: See :py:func:`~self.inference`
         """
 
@@ -137,6 +141,7 @@ def __init__(
         self.use_window = use_window
         self.window_infer_size = window_infer_size
         self.keep_on_cpu = keep_on_cpu
+        self.stats_to_csv = stats_csv
 
         """These attributes are all arguments of :py:func:~inference, please see that for reference"""
 
@@ -215,6 +220,8 @@ def inference(self):
 
             * keep_on_cpu: keep images on CPU or no
 
+            * stats_csv: compute stats on cells and save them to a csv file
+
         Yields:
             dict: contains :
                 * "image_id" : index of the returned image
@@ -447,6 +454,14 @@ def method(image):
                         f"Instance segmentation results for image n°{image_id} have been saved as:"
                     )
                     self.log(os.path.split(instance_filepath)[1])
+
+                    if self.stats_to_csv:
+                        data_dict = volume_stats(
+                            instance_labels
+                        )  # TODO test with area mesh function
+                    else:
+                        data_dict = None
+
                 else:
                     instance_labels = None
 
@@ -457,6 +472,7 @@ def method(image):
                     "image_id": i + 1,
                     "original": original,
                     "instance_labels": instance_labels,
+                    "object stats": data_dict,
                     "result": out,
                     "model_name": self.model_dict["name"],
                 }

napari_cellseg3d/plugin_base.py

@@ -209,6 +209,9 @@ def __init__(self, viewer: "napari.viewer.Viewer", parent=None):
 
     def make_close_button(self):
         btn = ui.make_button("Close", self.remove_from_viewer)
+        btn.setToolTip(
+            "Close the window and all docked widgets. Make sure to save your work !"
+        )
         return btn
 
     def make_prev_button(self):

napari_cellseg3d/plugin_model_inference.py

@@ -12,7 +12,6 @@
 from napari_cellseg3d import interface as ui
 from napari_cellseg3d import utils
 from napari_cellseg3d.model_framework import ModelFramework
-from napari_cellseg3d.model_instance_seg import volume_stats
 from napari_cellseg3d.model_workers import InferenceWorker
 
 
@@ -587,6 +586,7 @@ def start(self):  # TODO update
                 use_window=self.use_window_inference,
                 window_infer_size=self.window_inference_size,
                 keep_on_cpu=self.keep_on_cpu,
+                stats_csv=self.stats_to_csv,
             )
 
             yield_connect_show_res = lambda data: self.on_yield(
@@ -703,21 +703,12 @@ def on_yield(data, widget):
 
                 instance_layer = viewer.add_labels(labels, name=name)
 
-                if widget.stats_to_csv:  # TODO move to worker
+                data_dict = data["object stats"]
+                if (
+                    widget.stats_to_csv and data_dict is not None
+                ):  # TODO move to worker
 
-                    cell_data = volume_stats(
-                        labels
-                    )  # TODO test with area mesh function
-                    # count = np.tile("", len(cell_data["Volume"])-1)
-                    # cell_data["Cell count"] = np.insert(count, 0, number_cells)
-                    # cell_data["Total cell volume"] = np.insert(
-                    #     count, 0, tot_cell_volume
-                    # )
-                    # cell_data["Cell volume ratio"] = np.insert(
-                    #     count, 0, tot_cell_volume / len(labels.flatten())
-                    # )
-
-                    numeric_data = pd.DataFrame(cell_data)
+                    numeric_data = pd.DataFrame(data_dict)
 
                     csv_name = f"/{method}_seg_results_{image_id}_{utils.get_date_time()}.csv"
                     numeric_data.to_csv(

napari_cellseg3d/plugin_model_training.py

@@ -319,6 +319,54 @@ def __init__(
         self.btn_model_path.setVisible(False)
         self.lbl_model_path.setVisible(False)
 
+        ############################
+        ############################
+        # tooltips
+        self.zip_choice.setToolTip(
+            "Checking this will save a copy of the results as a zip folder"
+        )
+        self.validation_percent_choice.setToolTip(
+            "Choose the proportion of images to retain for training.\nThe remaining images will be used for validation"
+        )
+        self.epoch_choice.setToolTip(
+            "The number of epochs to train for.\nThe more you train, the better the model will fit the training data"
+        )
+        self.loss_choice.setToolTip(
+            "The loss function to use for training.\nSee the list in the inference guide for more info"
+        )
+        self.sample_choice.setToolTip(
+            "The number of samples to extract per image"
+        )
+        self.batch_choice.setToolTip(
+            "The batch size to use for training.\n A larger value will feed more images per iteration to the model,\n"
+            " which is faster and possibly improves performance, but uses more memory"
+        )
+        self.val_interval_choice.setToolTip(
+            "The number of epochs to perform before validating data.\n "
+            "The lower the value, the more often the score of the model will be computed and the more often the weights will be saved."
+        )
+        self.learning_rate_choice.setToolTip(
+            "The learning rate to use in the optimizer. \nUse a lower value if you're using pre-trained weights"
+        )
+        self.augment_choice.setToolTip(
+            "Check this to enable data augmentation, which will randomly deform, flip and shift the intensity in images"
+            " to provide a more general dataset. \nUse this if you're extracting more than 10 samples per image"
+        )
+        [
+            w.setToolTip("Size of the sample to extract")
+            for w in self.patch_size_widgets
+        ]
+        self.patch_choice.setToolTip(
+            "Check this to automatically crop your images in smaller, cubic images for training."
+            "\nShould be used if you have a small dataset (and large images)"
+        )
+        self.use_deterministic_choice.setToolTip("Enable deterministic training for reproducibility."
+                                                 "Using the same seed with all other parameters being similar should yield the exact same results between two runs.")
+        self.use_transfer_choice.setToolTip("Use this you want to initialize the model with pre-trained weights or use your own weights.")
+        self.box_seed.setToolTip("Seed to use for RNG")
+        ############################
+        ############################
+
         self.build()
 
     def toggle_patch_dims(self):

notebooks/csv_cell_plot.ipynb

@@ -29,7 +29,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "dat1 = pd.read_csv(\"data_plot.csv\", index_col=False)"
+    "dat1 = pd.read_csv(\"plots_data.csv\", index_col=False)"
    ]
   },
   {
@@ -53,18 +53,9 @@
     "\n",
     "    data = pd.read_csv(data_path, index_col=False)\n",
     "\n",
-    "    x = [\n",
-    "        float(data[\"Centroid\"][i][1:-2].split(\",\")[0])\n",
-    "        for i in range(len(data[\"Centroid\"]))\n",
-    "    ]\n",
-    "    y = [\n",
-    "        float(data[\"Centroid\"][i][1:-2].split(\",\")[1])\n",
-    "        for i in range(len(data[\"Centroid\"]))\n",
-    "    ]\n",
-    "    z = [\n",
-    "        float(data[\"Centroid\"][i][1:-2].split(\",\")[2])\n",
-    "        for i in range(len(data[\"Centroid\"]))\n",
-    "    ]\n",
+    "    x = data[\"Centroid x\"]\n",
+    "    y = data[\"Centroid y\"]\n",
+    "    z = data[\"Centroid z\"]\n",
     "\n",
     "    x = np.floor(x)\n",
     "    y = np.floor(y)\n",
@@ -133,7 +124,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "plot_data(\"data_plot.csv\")"
+    "plot_data(\"plots_data.csv\")"
    ]
   },
   {
@@ -143,7 +134,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "dat2 = pd.read_csv(\"data_plot_2.csv\", index_col=False)"
+    "dat2 = pd.read_csv(\"plots_data2.csv\", index_col=False)"
    ]
   },
   {
@@ -153,7 +144,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "plot_data(\"data_plot_2.csv\", z_inv=True, x_inv=True)"
+    "plot_data(\"plots_data2.csv\", z_inv=True, x_inv=True)"
    ]
   },
   {
@@ -189,18 +180,9 @@
     "\n",
     "    init_notebook_mode()  # initiate notebook for offline plot\n",
     "\n",
-    "    x = [\n",
-    "        float(data[\"Centroid\"][i][1:-2].split(\",\")[0])\n",
-    "        for i in range(len(data[\"Centroid\"]))\n",
-    "    ]\n",
-    "    y = [\n",
-    "        float(data[\"Centroid\"][i][1:-2].split(\",\")[1])\n",
-    "        for i in range(len(data[\"Centroid\"]))\n",
-    "    ]\n",
-    "    z = [\n",
-    "        float(data[\"Centroid\"][i][1:-2].split(\",\")[2])\n",
-    "        for i in range(len(data[\"Centroid\"]))\n",
-    "    ]\n",
+    "    x = data[\"Centroid x\"]\n",
+    "    y = data[\"Centroid y\"]\n",
+    "    z = data[\"Centroid z\"]\n",
     "\n",
     "    fig = go.Figure(\n",
     "        data=go.Scatter3d(\n",
@@ -214,7 +196,7 @@
     "                sizemin=20,\n",
     "                size=data[\"Volume\"],\n",
     "                color=data[\"Sphericity (axes)\"],\n",
-    "                colorscale=\"Viridis\",\n",
+    "                colorscale=\"Turbo_r\",\n",
     "                colorbar_title=\"Sphericity\",\n",
     "                line_color=\"rgb(140, 140, 170)\",\n",
     "            ),\n",