Merge pull request #971 from computational-cell-analytics/dev

New release

Author: constantinpape

.github/workflows/test.yaml

@@ -31,7 +31,7 @@ jobs:
       - name: Setup micromamba
         uses: mamba-org/setup-micromamba@v1
         with:
-          environment-file: ${{ runner.os == 'Windows' && 'environment_cpu_win.yaml' || 'environment.yaml' }}
+          environment-file: 'environment.yaml'
           create-args: >-
             python=${{ matrix.python-version }}
 

README.md

@@ -19,7 +19,7 @@ We implement napari applications for:
 <img src="https://github.com/computational-cell-analytics/micro-sam/assets/4263537/dfca3d9b-dba5-440b-b0f9-72a0683ac410" width="256">
 <img src="https://github.com/computational-cell-analytics/micro-sam/assets/4263537/aefbf99f-e73a-4125-bb49-2e6592367a64" width="256">
 
-If you run into any problems or have questions regarding our tool please open an issue on Github or reach out via [image.sc](https://forum.image.sc/) using the tag `micro-sam` and tagging @constantinpape.
+If you run into any problems or have questions regarding our tool please open an [issue](https://github.com/computational-cell-analytics/micro-sam/issues/new/choose) on Github or reach out via [image.sc](https://forum.image.sc/) using the tag `micro-sam`, and tagging [@constantinpape](https://forum.image.sc/u/constantinpape/summary) and [@anwai98](https://forum.image.sc/u/anwai98/summary).
 
 
 ## Installation and Usage
@@ -31,7 +31,7 @@ Please check [the documentation](https://computational-cell-analytics.github.io/
 
 We welcome new contributions!
 
-If you are interested in contributing to micro-sam, please see the [contributing guide](https://computational-cell-analytics.github.io/micro-sam/micro_sam.html#contribution-guide). The first step is to [discuss your idea in a new issue](https://github.com/computational-cell-analytics/micro-sam/issues/new) with the current developers.
+If you are interested in contributing to `micro-sam`, please see the [contributing guide](https://computational-cell-analytics.github.io/micro-sam/micro_sam.html#contribution-guide). The first step is to [discuss your idea in a new issue](https://github.com/computational-cell-analytics/micro-sam/issues/new) with the current developers.
 
 
 ## Citation
@@ -50,12 +50,12 @@ There are a few other napari plugins build around Segment Anything:
 - https://github.com/hiroalchem/napari-SAM4IS
 
 Compared to these we support more applications (2d, 3d and tracking), and provide finetuning methods and finetuned models for microscopy data.
-[WebKnossos](https://webknossos.org/) also offers integration of SegmentAnything for interactive segmentation.
+[WebKnossos](https://webknossos.org/) and [QuPath](https://qupath.github.io/) also offer integration of Segment Anything for interactive segmentation.
 
 We have also built follow-up work that is based on `micro_sam`:
-- https://github.com/computational-cell-analytics/patho-sam - improves SAM for histopathology
-- https://github.com/computational-cell-analytics/medico-sam - improves it for medical imaging
-- https://github.com/computational-cell-analytics/peft-sam - studies parameter efficient fine-tuning for SAM
+- https://github.com/computational-cell-analytics/patho-sam - improves SAM for histopathology.
+- https://github.com/computational-cell-analytics/medico-sam - improves SAM for medical imaging.
+- https://github.com/computational-cell-analytics/peft-sam - studies parameter efficient fine-tuning for SAM.
 
 ## Release Oveverview
 

development/check_omero_connection.py

@@ -0,0 +1,101 @@
+import numpy as np
+from shapely import LineString
+from skimage.measure import find_contours
+
+import ezomero as ez
+
+from micro_sam.sam_annotator import annotator_2d
+
+
+def _load_image(conn, upload_segmentation_to_omero):
+    # Step 1: Use the connecton to access files.
+    # NOTE: The "project_id" info is located inside the metadata section of your project.
+    # eg. the "project_id" for "example_data_test" is set to 46.
+    dataset_ids = ez.get_dataset_ids(conn, project=46)
+
+    # - Now that we know the dataset ids, the next sub-groups where data is stored,
+    # we go ahead with accessing them.
+    for id in dataset_ids:
+        image_ids = ez.get_image_ids(conn, dataset=id)
+        print(image_ids)
+
+    # - Once we have identified the image ids, let's open just one of it.
+    # I will open one z-stack.
+    # NOTE: Our array is located at pixels.
+    image_id = 7540
+    image_obj, pixels = ez.get_image(
+        conn,
+        image_id=image_id,
+        no_pixels=False,  # if True, only fetches the meta-data, which is super fast.
+        axis_lengths=(512, 512, 40, 1, 1),  # fetches an ROI in XYZCT config, otherwise the full volume.
+    )
+
+    # Let's annotate stuff using micro-sam.
+    pixels = pixels.squeeze()  # Remove singletons on-the-fly.
+
+    # HACK: For segmentation, let's keep it simple and segment the last slice only.
+    pixels = pixels[-1, ...]
+
+    # Run the 2d annotator
+    viewer = annotator_2d(image=pixels, embedding_path="test_omero.zarr", return_viewer=True)
+
+    import napari
+    napari.run()
+
+    if upload_segmentation_to_omero:
+
+        # Store the segmentations locally for storing them either as polygons or something else.
+        segmentation = viewer.layers["committed_objects"].data
+
+        # Let's try converting them as polygons, store them as a list of polygons and put it back.
+        contours = find_contours(segmentation)[0]  # Get contours
+        contour_as_line = LineString(contours)  # Convert contours to line structure.
+        simple_line = contour_as_line.simplify(tolerance=1.0)  # Adjust tolerance to make the polygon.
+        simple_coords = np.array(simple_line.coords)
+
+        # Now, let's post a single ROI and see if it worked.
+        ez.post_roi(conn, image_id=image_id, shapes=[ez.rois.Polygon(simple_coords, z=40)], name="test_micro_sam_seg")
+
+
+if __name__ == "__main__":
+    import argparse
+    parser = argparse.ArgumentParser(description="Run `micro-sam` on OMERO data")
+    parser.add_argument(
+        "--host", default="omero-training.gerbi-gmb.de", type=str, help="The host server URL",
+    )
+
+    # NOTE: If the default port below is blocked, use the one below.
+    # host = "wss://omero-training.gerbi-gmb.de/omero-wss"
+    # port = 443  # the default port (4064) seems to work for me.
+    parser.add_argument(
+        "--port", default=4064, type=int, help="The choice of port for connecting to the server",
+    )
+    parser.add_argument(
+        "--username", default="tim2025_test", type=str, help="The username.",
+    )
+    parser.add_argument(
+        "--password", default="tim2025_test", type=str, help="The correspond user's password."
+    )
+    parser.add_argument(
+        "--omero_group", default="TiM2025_preparation", type=str, help="The OMERO-level group name."
+    )
+    parser.add_argument(
+        "--upload_segmentation", action="store_true", help="Whether to load segmentation as polygons to OMERO."
+    )
+    args = parser.parse_args()
+
+    # Inspired by https://github.com/I3D-bio/omero_python_workshop.
+
+    # Check connection to the test account.
+    conn = ez.connect(
+        user=args.username,
+        password=args.password,
+        group=args.omero_group,
+        host=args.host,
+        port=args.port,
+        secure=True
+    )
+    print(f"Is connected: {conn.isConnected()}")
+
+    # Visualize the image from the OMERO server.
+    _load_image(conn, upload_segmentation_to_omero=args.upload_segmentation)

doc/faq.md

@@ -166,6 +166,9 @@ NOTE: It is important to choose the correct model type when you opt for the abov
 The older model versions are still available on zenodo. You can find the download links for all of them [here](https://computational-cell-analytics.github.io/micro-sam/micro_sam.html#other-models).
 You can then use those models with the custom checkpoint option, see answer 15 for details.
 
+### 18. I would like to evaluate the instance segmentation quantitatively. Can you suggest how to do that?
+`micro-sam` supports a `micro_sam.evaluate` CLI, which computes the mean segmentation accuracy (introduced in the Pascal VOC challenge) of the predicted instance segmentation with the corresponding ground-truth annotations. Please see our paper (`Methods` -> `Inference and Evaluation` for more details about it) and `$ micro_sam.evaluate -h` for more details about the evaluation CLI.
+
 
 ## Fine-tuning questions
 

doc/installation.md

@@ -13,12 +13,10 @@ We do **not support** installing `micro_sam` with pip.
 
 `conda` is a python package manager. If you don't have it installed yet you can follow the instructions [here](https://conda-forge.org/download/) to set it up on your system.
 Please make sure that you are using an up-to-date version of conda to install `micro_sam`.
-You can also use [mamba](https://mamba.readthedocs.io/en/latest/), which is a drop-in replacement for conda, to install it. In this case, just replace the `conda` command below with `mamba`.
+You can also use [mamba](https://mamba.readthedocs.io/en/latest/), which is a drop-in replacement for conda, to install it. In this case, just replace the `conda` commands below with `mamba`.
 
 **IMPORTANT**: Do not install `micro_sam` in the base conda environment.
 
-**Installation on Linux and Mac OS:**
-
 `micro_sam` can be installed in an existing environment via:
 ```bash
 conda install -c conda-forge micro_sam
@@ -39,24 +37,10 @@ However, if you have an older operating system, or a CUDA version older than 12,
 conda install -c conda-forge micro_sam "libtorch=*=cuda11*"
 ```
 
-**Installation on Windows:**
-
-`pytorch` is currently not available on conda-forge for windows. Thus, you have to install it from the `pytorch` conda channel. In addition, you have to specify two specific dependencies to avoid incompatibilities.
-This can be done with the following commands:
-```bash
-conda install -c pytorch -c conda-forge micro_sam "nifty=1.2.1=*_4" "protobuf<5"
-```
-to install `micro_sam` in an existing environment and
-```bash
-conda create -c pytorch -c conda-forge -n micro-sam micro_sam "nifty=1.2.1=*_4" "protobuf<5"
-```
-
 ## From source
 
 To install `micro_sam` from source, we recommend to first set up an environment with the necessary requirements:
-- [environment.yaml](https://github.com/computational-cell-analytics/micro-sam/blob/master/environment.yaml): to set up an environment on Linux or Mac OS.
-- [environment_cpu_win.yaml](https://github.com/computational-cell-analytics/micro-sam/blob/master/environment_cpu_win.yaml): to set up an environment on windows with CPU support.
-- [environment_gpu_win.yaml](https://github.com/computational-cell-analytics/micro-sam/blob/master/environment_gpu_win.yaml): to set up an environment on windows with GPU support.
+- [environment.yaml](https://github.com/computational-cell-analytics/micro-sam/blob/master/environment.yaml): to set up an environment on Windows, Linux or Mac OS.
 
 To create one of these environments and install `micro_sam` into it follow these steps
 
@@ -75,7 +59,7 @@ cd micro-sam
 3. Create the respective environment:
 
 ```bash
-conda env create -f <ENV_FILE>.yaml
+conda env create -f environment.yaml
 ```
 
 4. Activate the environment:

doc/python_library.md

@@ -32,7 +32,7 @@ In fact the best results can be expected when finetuning on your own data, and w
 So a good strategy is to annotate a few images with one of the provided models using our interactive annotation tools and, if the model is not working as good as required for your use-case, finetune on the annotated data.
 We recommend checking out our [paper](https://www.nature.com/articles/s41592-024-02580-4) for details on the results on how much data is required for finetuning Segment Anything.
 
-The training logic is implemented in `micro_sam.training` and is based on [torch-em](https://github.com/constantinpape/torch-em). Check out [the finetuning notebook](https://github.com/computational-cell-analytics/micro-sam/blob/master/notebooks/sam_finetuning.ipynb) to see how to use it.
+The training logic is implemented in `micro_sam.training` and is based on [torch-em](https://github.com/constantinpape/torch-em). Check out [the finetuning notebook](https://github.com/computational-cell-analytics/micro-sam/blob/master/notebooks/sam_finetuning.ipynb) to see how to use it, or the training CLI (`micro_sam.train`), see `micro_sam.train -h` for details on how to use it.
 We also support training an additional decoder for automatic instance segmentation. This yields better results than the automatic mask generation of segment anything and is significantly faster.
 The notebook explains how to train it together with the rest of SAM and how to then use it.
 
@@ -52,4 +52,4 @@ Here is a list of resources, together with their recommended training settings,
 | GPU (NVIDIA A100)           | 80GB     | ViT Large  | 2          | *all*                        | 30               |
 | GPU (NVIDIA A100)           | 80GB     | ViT Huge   | 2          | *all*                        | 25               |
 
-> NOTE: If you use the [finetuning UI](#finetuning-ui) or `micro_sam.training.training.train_sam_for_configuration` you can specify the hardware configuration and the best settings for it will be set automatically. If your hardware is not in the settings we have tested choose the closest match. You can set the training parameters yourself when using `micro_sam.training.training.train_sam`. Be aware that the choice for the number of objects per image, the batch size, and the type of model have a strong impact on the VRAM needed for training and the duration of training. See the [finetuning notebook](https://github.com/computational-cell-analytics/micro-sam/blob/master/notebooks/sam_finetuning.ipynb) for an overview of these parameters.
+> NOTE: If you use the [finetuning UI](#finetuning-ui) or the training CLI (`micro_sam.train`) or `micro_sam.training.training.train_sam_for_configuration`, you can specify the hardware configuration and the best settings for it will be set automatically. If your hardware is not in the settings we have tested choose the closest match. You can set the training parameters yourself when using `micro_sam.training.training.train_sam`. Be aware that the choice for the number of objects per image, the batch size, and the type of model have a strong impact on the VRAM needed for training and the duration of training. See the [finetuning notebook](https://github.com/computational-cell-analytics/micro-sam/blob/master/notebooks/sam_finetuning.ipynb) for an overview of these parameters.

doc/start_page.md

@@ -32,7 +32,7 @@ After installing `micro_sam`, you can start napari from within your environment
 ```bash
 $ napari
 ```
-After starting napari, you can select the annotation tool you want to use from `Plugins -> SegmentAnything for Microscopy`. Check out the [quickstart tutorial video](https://youtu.be/gcv0fa84mCc) for a short introduction, the video of our [virtual I2K tutorial](https://www.youtube.com/watch?v=dxjU4W7bCis&list=PLdA9Vgd1gxTbvxmtk9CASftUOl_XItjDN&index=33) for an in-depth explanation and [the annotation tool section](#annotation-tools) for details.
+After starting napari, you can select the annotation tool you want to use from `Plugins -> Segment Anything for Microscopy`. Check out the [quickstart tutorial video](https://youtu.be/gcv0fa84mCc) for a short introduction, the video of our [virtual I2K tutorial](https://www.youtube.com/watch?v=dxjU4W7bCis&list=PLdA9Vgd1gxTbvxmtk9CASftUOl_XItjDN&index=33) for an in-depth explanation and [the annotation tool section](#annotation-tools) for details.
 
 The `micro_sam` python library can be imported via
 

environment.yaml

@@ -2,16 +2,16 @@ name: sam
 channels:
     - conda-forge
 dependencies:
-    - nifty >=1.2.1
+    - nifty >=1.2.3
     - imagecodecs
     - magicgui
-    - napari >=0.5.0
+    - napari >=0.5.0,<0.6.0
     - natsort
     - pip
     - pooch
     - pyqt
     - python-xxhash
-    - python-elf >=0.4.8
+    - python-elf >=0.6.1
     # Note: installing the pytorch package from conda-forge will generally
     # give you the most optmized version for your system, if you have a modern
     # enough OS and CUDA version (CUDA >= 12). For older versions, you can
@@ -20,11 +20,12 @@ dependencies:
     # - libtorch=*=cuda11*
     # or, to enforce a CPU installation, change to
     # - "pytorch=*=cpu*"
-    - pytorch >=2.4
+    - pytorch >=2.5
     - segment-anything
     - torchvision
-    - torch_em >=0.7.0
+    - torch_em >=0.7.8
     - tqdm
     - timm
+    - xarray <2025.3.0
     - pip:
         - git+https://github.com/ChaoningZhang/MobileSAM.git