More doc updates and minor changes

Author: constantinpape

build_doc.py

@@ -8,13 +8,15 @@
 
 def check_docs_completeness():
     """@private
-    All markdown and RST documentation files **MUST** be included in the module
+    All markdown and RST documentation files **SHOULD** be included in the module
     docstring at micro_sam/__init__.py
     """
     import micro_sam
 
-    markdown_doc_files = glob.glob("doc/**/*.md", recursive=True)
-    rst_doc_files = glob.glob("doc/**/*.rst", recursive=True)
+    # We don't search in subfolders anymore, to allow putting additional documentation
+    # (e.g. for bioimage.io mdoels) that should not be included in the main documentation here.
+    markdown_doc_files = glob.glob("doc/*.md", recursive=True)
+    rst_doc_files = glob.glob("doc/*.rst", recursive=True)
     all_doc_files = markdown_doc_files + rst_doc_files
     missing_from_docs = [f for f in all_doc_files if os.path.basename(f) not in micro_sam.__doc__]
     if len(missing_from_docs) > 0:
@@ -42,5 +44,3 @@ def check_docs_completeness():
     cmd.append("micro_sam")
 
     run(cmd)
-
-    # pdoc --docformat google --logo "https://raw.githubusercontent.com/computational-cell-analytics/micro-sam/master/doc/images/micro-sam-logo.png" micro_sam

doc/annotation_tools.md

@@ -104,15 +104,39 @@ Check out [this video](TODO) for a tutorial for how to use the tracking annotati
 
 ## Image Series Annotator
 
-<img src="https://raw.githubusercontent.com/computational-cell-analytics/micro-sam/master/doc/images/series-menu.png" width="1024">
+The image series annotation tool enables running the [2d annotator](#annotator-2d) or [2d annotator](#annotator-3d) for multiple images that are saved within an folder. This makes it convenient to annotate many images without having to close the tool. It can be started by
+- clicking `Image Series Annotator` in the plugin menu.
+- running `$ micro_sam.image_series_annotator` in the command line.
+- calling `micro_sam.sam_annotator.image_series_annotator` in a python script. Check out [examples/image_series_annotator.py](https://github.com/computational-cell-analytics/micro-sam/blob/master/examples/image_series_annotator.py) for details. 
 
-We also provide the `image series annotator`, which can be used for running the 2d annotator for several images in a folder. You can start by clicking `Image series annotator` in the GUI, running `micro_sam.image_series_annotator` in the command line or from a [python script](https://github.com/computational-cell-analytics/micro-sam/blob/master/examples/image_series_annotator.py).
+When starting this tool via the plugin menu the following interface opens:
 
+<img src="https://raw.githubusercontent.com/computational-cell-analytics/micro-sam/master/doc/images/series-menu.png" width="512">
 
-## Finetuning Tool
+You can select the folder where your image data is saved with `Input Folder`. The annotation results will be saved in `Output Folder`.
+You can specify a rule for loading only a subset of images via `pattern`, for example `*.tif` to only load tif images. Set `is_volumetric` if the data you want to annotate is 3d. The rest of the options are settings for the image embedding computation and are the same as for the embedding menu (see above).
+Once you click `Annotate Images` the images from the folder you have specified will be loaded and the annotation tool is started for them.
 
-<img src="https://raw.githubusercontent.com/computational-cell-analytics/micro-sam/master/doc/images/finetuning-menu.png" width="1024">
+This menu will not open if you start the image series annotator from the command line or via python. In this case the input folder and other settings are passed as parameters instead.
 
+Check out [this video](TODO) for a tutorial for how to use the image series annotator.
+
+
+## Finetuning UI
+
+We also provide a graphical tool for finetuning models on your own data. It can be started by clicking `Finetuning` in the plugin menu.
+
+**Note:** if you know a bit of python programming we recommend to use a script for model finetuning instead. This will give you more options to configure the training. See [these instructions](training-your-own-model) for details.
+
+When starting this tool via the plugin menu the following interface opens:
+
+<img src="https://raw.githubusercontent.com/computational-cell-analytics/micro-sam/master/doc/images/finetuning-menu.png" width="512">
+
+You can select the image data via `Path to images`. We can either load images from a folder or select a single file for training. By providing `Image data key` you can either provide a pattern for selecting files from a folder or provide an internal filepath for hdf5, zarr or similar fileformats.
+
+You can select the label data via `Path to labels` and `Label data key`, following the same logic as for the image data. We expect label masks stored in the same size as the image data for training. You can for example use annotations created with one of the `micro_sam` annotation tools for this, they are stored in the correct format!
+
+The `Configuration` option allows you to choose the hardware configuration for training. We try to automatically select the correct setting for your system, but it can also be changed. Please refer to the tooltips for the other parameters.
 
 ## Tips & Tricks
 

doc/contributing.md

@@ -1,4 +1,4 @@
-# How to contribute
+# Contribution Guide
 
 * [Discuss your ideas](#discuss-your-ideas)
 * [Clone the repository](#clone-the-repository)

doc/development.md renamed to doc/deprecated/development.md

@@ -1,3 +1,5 @@
+**This is outdated and we need to update it to correctly describe the current annotator design.**
+
 # For Developers
 
 This software consists of four different python (sub-)modules:

doc/finetuned_models.md

@@ -1,52 +1,53 @@
 # Finetuned models
 
-In addition to the original Segment anything models, we provide models that finetuned on microscopy data using the functionality from `micro_sam.training`.
-The models are hosted on zenodo. We currently offer the following models:
+In addition to the original Segment Anything models, we provide models that are finetuned on microscopy data.
+The additional models are available in the [bioimage.io modelzoo](https://bioimage.io/#/) and are also hosted on zenodo.
+
+We currently offer the following models:
 - `vit_h`: Default Segment Anything model with vit-h backbone.
 - `vit_l`: Default Segment Anything model with vit-l backbone.
 - `vit_b`: Default Segment Anything model with vit-b backbone.
 - `vit_t`: Segment Anything model with vit-tiny backbone. From the [Mobile SAM publication](https://arxiv.org/abs/2306.14289). 
-- `vit_b_lm`: Finetuned Segment Anything model for cells and nuclei in light microscopy data with vit-b backbone.
-- `vit_b_em_organelles`: Finetuned Segment Anything model for mitochodria and nuclei in electron microscopy data with vit-b backbone.
-- `vit_b_em_boundaries`: Finetuned Segment Anything model for neurites and cells in electron microscopy data with vit-b backbone.
+- `vit_l_lm`: Finetuned Segment Anything model for cells and nuclei in light microscopy data with vit-l backbone. ([zenodo](TODO), [bioimage.io](TODO))
+- `vit_b_lm`: Finetuned Segment Anything model for cells and nuclei in light microscopy data with vit-b backbone. ([zenodo](TODO), [bioimage.io](TODO))
+- `vit_t_lm`: Finetuned Segment Anything model for cells and nuclei in light microscopy data with vit-t backbone. ([zenodo](TODO), [bioimage.io](TODO))
+- `vit_l_em_organelles`: Finetuned Segment Anything model for mitochodria and nuclei in electron microscopy data with vit-l backbone. ([zenodo](TODO), [bioimage.io](TODO))
+- `vit_b_em_organelles`: Finetuned Segment Anything model for mitochodria and nuclei in electron microscopy data with vit-b backbone. ([zenodo](TODO), [bioimage.io](TODO))
+- `vit_t_em_organelles`: Finetuned Segment Anything model for mitochodria and nuclei in electron microscopy data with vit-t backbone. ([zenodo](TODO), [bioimage.io](TODO))
 
 See the two figures below of the improvements through the finetuned model for LM and EM data. 
 
 <img src="https://raw.githubusercontent.com/computational-cell-analytics/micro-sam/master/doc/images/lm_comparison.png" width="768">
 
 <img src="https://raw.githubusercontent.com/computational-cell-analytics/micro-sam/master/doc/images/em_comparison.png" width="768">
 
-You can select which of the models is used in the annotation tools by selecting the corresponding name from the `Model Type` menu:
+You can select which model to use for annotation by selecting the corresponding name in the embedding menu:
 
 <img src="https://raw.githubusercontent.com/computational-cell-analytics/micro-sam/master/doc/images/model-type-selector.png" width="256">
 
 To use a specific model in the python library you need to pass the corresponding name as value to the `model_type` parameter exposed by all relevant functions.
-See for example the [2d annotator example](https://github.com/computational-cell-analytics/micro-sam/blob/master/examples/annotator_2d.py#L62) where `use_finetuned_model` can be set to `True` to use the `vit_b_lm` model.
-
-Note that we are still working on improving these models and may update them from time to time. All older models will stay available for download on zenodo, see [model sources](#model-sources) below
+See for example the [2d annotator example](https://github.com/computational-cell-analytics/micro-sam/blob/master/examples/annotator_2d.py#L62).
 
 
-## Which model should I choose?
+## Choosing a Model 
 
 As a rule of thumb:
-- Use the `vit_b_lm` model for segmenting cells or nuclei in light microscopy.
-- Use the `vit_b_em_organelles` models for segmenting mitochondria, nuclei or other organelles in electron microscopy.
-- Use the `vit_b_em_boundaries` models for segmenting cells or neurites in electron microscopy.
+- Use the `vit_l_lm` or `vit_b_lm` model for segmenting cells or nuclei in light microscopy. The larger model (`vit_l_lm`) yields a bit better segmentation quality, especially for automatic segmentation, but needs more computational resources.
+- Use the `vit_l_em_organelles` or `vit_b_em_organelles` models for segmenting mitochondria, nuclei or other  roundish organelles in electron microscopy.
 - For other use-cases use one of the default models.
+- The `vit_t_...` models run much faster than other models, but yield inferior quality for many applications. It can still make sense to try them for your use-case if your working on a laptop and want to annotate many images or volumetric data. 
 
-See also the figures above for examples where the finetuned models work better than the vanilla models.
-Currently the model `vit_h` is used by default.
-
+See also the figures above for examples where the finetuned models work better than the default models.
 We are working on further improving these models and adding new models for other biomedical imaging domains.
 
 
-## Model Sources
+## Older Models
 
-Here is an overview of all finetuned models we have released to zenodo so far:
+Previous versions of our models are available on zenodo:
 - [vit_b_em_boundaries](https://zenodo.org/records/10524894): for segmenting compartments delineated by boundaries such as cells or neurites in EM.
 - [vit_b_em_organelles](https://zenodo.org/records/10524828): for segmenting mitochondria, nuclei or other organelles in EM.
 - [vit_b_lm](https://zenodo.org/records/10524791): for segmenting cells and nuclei in LM.
-- [vit_h_em](https://zenodo.org/records/8250291): this model is outdated.
-- [vit_h_lm](https://zenodo.org/records/8250299): this model is outdated.
+- [vit_h_em](https://zenodo.org/records/8250291): for general EM segmentation.
+- [vit_h_lm](https://zenodo.org/records/8250299): for general LM segmentation.
 
-Some of these models contain multiple versions.
+We do not recommend to use these models since our new models improve upon them significantly. But we provide the links here in case they are needed to reproduce older segmentation workflows.

doc/images/model-type-selector.png

@@ -5,9 +5,9 @@ The python library can be imported via
 import micro_sam
 ```
 
-The library
-- implements function to apply Segment Anything to 2d and 3d data more conveniently in `micro_sam.prompt_based_segmentation`.
-- provides more and improved automatic instance segmentation functionality in `micro_sam.instance_segmentation`.
+This library extends the [Segment Anything library](https://github.com/facebookresearch/segment-anything) and
+- implements functions to apply Segment Anything to 2d and 3d data in `micro_sam.prompt_based_segmentation`.
+- provides improved automatic instance segmentation functionality in `micro_sam.instance_segmentation`.
 - implements training functionality that can be used for finetuning on your own data in `micro_sam.training`.
 - provides functionality for quantitative and qualitative evaluation of Segment Anything models in `micro_sam.evaluation`.
 
@@ -18,24 +18,22 @@ import micro_sam.instance_segmentation
 # etc.
 ```
 
-This functionality is used to implement the interactive annotation tools and can also be used as a standalone python library.
-Some preliminary examples for how to use the python library can be found [here](https://github.com/computational-cell-analytics/micro-sam/tree/master/examples/use_as_library). Check out the `Submodules` documentation for more details.
+This functionality is used to implement the interactive annotation tools in `micro_sam.sam_annotator` and can be used as a standalone python library.
+We provide jupyter notebooks that demonstrate how to use it [here](https://github.com/computational-cell-analytics/micro-sam/tree/master/notebooks). You can find the full library documentation by scrolling to the end of this page. 
 
 ## Training your own model
 
 We reimplement the training logic described in the [Segment Anything publication](https://arxiv.org/abs/2304.02643) to enable finetuning on custom data.
-We use this functionality to provide the [finetuned microscopy models](#finetuned-models) and it can also be used to finetune models on your own data.
+We use this functionality to provide the [finetuned microscopy models](#finetuned-models) and it can also be used to train models on your own data.
 In fact the best results can be expected when finetuning on your own data, and we found that it does not require much annotated training data to get siginficant improvements in model performance.
-So a good strategy is to annotate a few images with one of the provided models using one of the interactive annotation tools and, if the annotation is not working as good as expected yet, finetune on the annotated data.
+So a good strategy is to annotate a few images with one of the provided models using our interactive annotation tools and, if the model is not working as good as required for your use-case, finetune on the annotated data.
 <!--
 TODO: provide link to the paper with results on how much data is needed
 -->
 
-The training logic is implemented in `micro_sam.training` and is based on [torch-em](https://github.com/constantinpape/torch-em). Please check out [examples/finetuning](https://github.com/computational-cell-analytics/micro-sam/tree/master/examples/finetuning) to see how you can finetune on your own data with it. The script `finetune_hela.py` contains an example for finetuning on a small custom microscopy dataset and `use_finetuned_model.py` shows how this model can then be used in the interactive annotation tools.
+The training logic is implemented in `micro_sam.training` and is based on [torch-em](https://github.com/constantinpape/torch-em). Check out [the finetuning notebook](https://github.com/computational-cell-analytics/micro-sam/blob/master/notebooks/micro-sam-finetuning.ipynb) to see how to use it.
 
-Since release v0.4.0 we also support training an additional decoder for automatic instance segmentation. This yields better results than the automatic mask generation of segment anything and is significantly faster.
-You can enable training of the decoder by setting `train_instance_segmentation = True` [here](https://github.com/computational-cell-analytics/micro-sam/blob/master/examples/finetuning/finetune_hela.py#L165).
-The script `instance_segmentation_with_finetuned_model.py` shows how to use it for automatic instance segmentation.
-We will fully integrate this functionality with the annotation tool in the next release.
+We also support training an additional decoder for automatic instance segmentation. This yields better results than the automatic mask generation of segment anything and is significantly faster.
+The notebook explains how to activate training it together with the rest of SAM and how to then use it.
 
-More advanced examples, including quantitative and qualitative evaluation, of finetuned models can be found in [finetuning](https://github.com/computational-cell-analytics/micro-sam/tree/master/finetuning), which contains the code for training and evaluating our microscopy models.
+More advanced examples, including quantitative and qualitative evaluation, of finetuned models can be found in [finetuning](https://github.com/computational-cell-analytics/micro-sam/tree/master/finetuning), which contains the code for training and evaluating [our models](finetuned-models).

doc/images/series-menu.png

@@ -323,7 +323,7 @@ def _create_settings(self):
 
         self.pattern = "*"
         _, layout = self._add_string_param(
-            "", self.pattern, tooltip=get_tooltip("image_series_annotator", "pattern")
+            "pattern", self.pattern, tooltip=get_tooltip("image_series_annotator", "pattern")
         )
         setting_values.layout().addLayout(layout)