Merge branch 'master' into sgn-density-and-volume

Author: constantinpape

.github/workflows/build_docs.yaml

@@ -0,0 +1,74 @@
+name: Build and Deploy Docs
+
+on:
+  push:
+    paths:
+      - "doc/*.md"   # Trigger on changes to any markdown file
+      - "flamigo_tools/**/*.py"   # Optionally include changes in Python files
+    branches:
+      - master       # Run the workflow only on pushes to the master branch
+  workflow_dispatch:
+
+# security: restrict permissions for CI jobs.
+permissions:
+  contents: read
+  pages: write          # to publish to Pages
+  id-token: write       # to authenticate to Pages
+
+concurrency:
+  group: "pages"
+  cancel-in-progress: true
+
+jobs:
+  build:
+    name: Build Documentation
+    runs-on: ubuntu-latest
+
+    steps:
+    - name: Checkout Code
+      uses: actions/checkout@v4
+
+    - name: Set up Micromamba
+      uses: mamba-org/setup-micromamba@v2
+      with:
+        micromamba-version: "latest"
+        environment-file: environment.yaml
+        init-shell: bash
+        cache-environment: true
+        post-cleanup: 'all'
+
+    - name: Install napari
+      shell: bash -l {0}
+      run: pip install napari pyqt5
+
+    - name: Install package
+      shell: bash -l {0}
+      run: pip install -e .
+
+    - name: Install pdoc
+      shell: bash -l {0}
+      run: pip install pdoc
+
+    - name: Generate Documentation
+      shell: bash -l {0}
+      run: pdoc flamingo_tools/ -d google -o _site
+
+    - name: Verify Documentation Output
+      run: ls -la _site/
+
+    - name: Upload Documentation Artifact
+      uses: actions/upload-pages-artifact@v3
+      with:
+        path: _site/
+
+  deploy:
+    name: Deploy Documentation
+    needs: build
+    runs-on: ubuntu-latest
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    steps:
+      - name: Deploy to GitHub Pages
+        id: deployment
+        uses: actions/deploy-pages@v4

README.md

@@ -1,7 +1,14 @@
-# Flamingo Tools
+# CochleaNet
 
-Data processing for light-sheet microscopy, specifically for data from [Flamingo microscopes](https://huiskenlab.com/flamingo/).
+CochleaNet is a software for the analysis of cochleae imaged in light-sheet microscopy. It is based on deep neural networks for the segmentation of spiral ganglion neurons, inner hair cells, and the detection of ribbon synapses.
+It was developed for imaging data from (clear-tissue) [flamingo microscopes](https://huiskenlab.com/flamingo/) and is also applicable to data from commercial microscopes.
 
+In addition to the analysis functionality, CochleaNet implements data pre-processing to convert data from flamingo microscopes into a format compatible with [BigStitcher](https://imagej.net/plugins/bigstitcher/) and to export image data and segmentation results to [ome.zarr](https://www.nature.com/articles/s41592-021-01326-w) and [MoBIE](https://mobie.github.io/).
+This functionality is applicable to any imaging data from flamingo microscopes, not only clear-tissue data or cochleae. We aim to also extend the segmentation and analysis functionality to other kinds of samples imaged in the flamingo in the future.
+
+For installation and usage instructions, check out [the documentation](https://computational-cell-analytics.github.io/cochlea-net/). For more details on the underlying methodology check out [our preprint](TODO).
+
+<!---
 The `flamingo_tools` library implements functionality for:
 - converting the lightsheet data into a format compatible with [BigDataViewer](https://imagej.net/plugins/bdv/) and [BigStitcher](https://imagej.net/plugins/bigstitcher/).
 - Cell / nucleus segmentation via a 3D U-net.
@@ -48,3 +55,4 @@ You can also check out the following example scripts:
 - `load_data.py`: Example script for how to load sub-regions from the converted data into python.
 
 For advanced examples to segment data with a U-Net, check out the `scripts` folder.
+--->

doc/documentation.md

@@ -0,0 +1,83 @@
+# CochleaNet
+
+CochleaNet is a software tool for the analysis of cochleae imaged in light-sheet microscopy.
+Its main components are:
+- A deep neural network for segmenting spiral ganglion neurons (SGNs) from parvalbumin (PV) staining.
+- A deep neural network for segmenting inner hair cells (IHCs) from VGlut3 staining.
+- A deep neural network for detecting ribbon synapses from CtBP2 staining.
+
+In addition, it contains functionality for data pre-processing and different kinds of measurements based on the network predictions, including:
+- Analyzing the tonotopic mapping of SGNs and IHCs in the cochlea.
+- Validating gene therapies and optogentic therapies (based on additional fluorescent stainings).
+- Analyzing SGN subtypes (based on additional fluorescent staining).
+- Visualizing segmentation results and derived analyses in [MoBIE](https://mobie.github.io/).
+
+The networks and analysis methods were primarily developed for high-resolution isotropic data from a [custom light-sheet microscope](https://www.biorxiv.org/content/10.1101/2025.02.21.639411v2.abstract).
+The networks will work best on the respective fluorescent stains they were trained on, but will work on similar stains. For example, we have successfully applied the network for SGN segmentation on a calretinin (CR) stain and the network for IHC segmentation on a myosin7a stain. 
+In addition, CochleaNet provides networks for the segmentation of SGNs and IHCs in anisotropic data from a [commercial light-sheet microscope](https://www.miltenyibiotec.com/DE-en/products/macs-imaging-and-spatial-biology/ultramicroscope-platform.html).
+
+For more information on CochleaNet, check out our [preprint](TODO).
+
+## Installation
+
+CochleaNet can be installed via `conda` (or [micromamba](https://mamba.readthedocs.io/en/latest/user_guide/micromamba.html)).
+To install it:
+- Download the CochleaNet github repository:
+```
+git clone https://github.com/computational-cell-analytics/cochlea-net
+```
+- Go to the directory:
+```
+cd cochlea-net
+```
+- Create an environment with the required dependencies:
+```
+conda env create -f environment.yaml
+```
+- Activate the environment:
+```
+conda activate cochlea-net
+```
+- Install the cochlea-net package:
+```
+pip install .
+```
+- (Optional): if you want to use the napari plugin you have to install napari:
+```
+conda install -c conda-forge napari pyqt
+```
+
+## Usage
+
+CochleaNet can be used via:
+- The [napari plugin](#napari-plugin): enables prediction with the pre-trained CochleaNet deep neural networks.
+- The [command line interface](#command-line-interface): enables data conversion, model prediction, and selected analysis workflows for large image data.
+- The [python library](#python-library): implements CochleaNet's functionality and can be used to implement flexible prediction and data analysis workflows for large image data. 
+
+**Note: the napari plugin was not optimized for processing large data. For processing large image data use the CLI or python library.**
+
+### Napari Plugin
+
+The napari plugin for segmentation (SGNs and IHCS) and detection (ribbon synapses) is available under `Plugins->CochleaNet->Segmentation/Detection` in napari:
+
+The segmentation plugin offers the choice of different models under `Select Model:` (see [Available Models](#available-models) for details). `Image data` enables to choose which image data (layer) the model is applied to. The segmentation is started by clicking the `Run Segmentation` button. After the segmentation has finished, a new segmentation layer with the result (here `IHC`) will be added:
+
+The detection model works similarly. It currently provides the model for synapse detection. The predictions are added as point layer (``):
+
+TODO Video.
+For more information on how to use napari, check out the tutorials at [www.napari.org](TODO).
+
+**To use the napari plugin you have to install `napari` and `pyqt` in your environment.** See [installation](#installation) for details.
+
+### Command Line Interface
+
+TODO
+
+### Python Library
+
+TODO
+
+
+## Available Models
+
+TODO

environment.yaml

@@ -1,17 +1,19 @@
-name: flamingo
+name: cochlea-net
 
 channels:
   - conda-forge
 
 dependencies:
   - cluster_tools
-  - scikit-image
+  - mobie_utils
+  - pip
   - pooch
   - pybdv
   - pytorch
+  - scikit-image
   - s3fs
   - torch_em
   - trimesh
   - z5py
-  # Don't install zarr v3, as we are not sure that it is compatible with MoBIE etc. yet
+  # Don't install zarr v3, which is not yet compatible with all dependencies.
   - zarr <3

flamingo_tools/__init__.py

@@ -1,2 +1,6 @@
+"""
+.. include:: ../doc/documentation.md
+"""
+
 from .data_conversion import convert_lightsheet_to_bdv, convert_lightsheet_to_bdv_cli
 from .test_data import create_test_data

flamingo_tools/classification/classification_gui.py

@@ -11,9 +11,12 @@
 from joblib import dump
 from magicgui import magic_factory
 
-import micro_sam.sam_annotator.object_classifier as classifier_util
-from micro_sam.object_classification import project_prediction_to_segmentation
-from micro_sam.sam_annotator._widgets import _generate_message
+try:
+    import micro_sam.sam_annotator.object_classifier as classifier_util
+    from micro_sam.object_classification import project_prediction_to_segmentation
+    from micro_sam.sam_annotator._widgets import _generate_message
+except ImportError:
+    micro_sam = None
 
 from ..measurements import compute_object_measures_impl
 

flamingo_tools/file_utils.py

@@ -1,18 +1,34 @@
+import os
 import warnings
 from typing import Optional, Union
 
 import imageio.v3 as imageio
 import numpy as np
+import pooch
 import tifffile
 import zarr
 from elf.io import open_file
 
+from .s3_utils import get_s3_path
+
 try:
     from zarr.abc.store import Store
 except ImportError:
     from zarr._storage.store import BaseStore as Store
 
 
+def get_cache_dir() -> str:
+    """Get the cache directory of CochleaNet.
+
+    The default cache directory is "$HOME/cochlea-net"
+
+    Returns:
+        The cache directory.
+    """
+    cache_dir = os.path.expanduser(pooch.os_cache("cochlea-net"))
+    return cache_dir
+
+
 def _parse_shape(metadata_file):
     depth, height, width = None, None, None
 
@@ -67,7 +83,9 @@ def read_tif(file_path: str) -> Union[np.ndarray, np.memmap]:
     return x
 
 
-def read_image_data(input_path: Union[str, Store], input_key: Optional[str]) -> np.typing.ArrayLike:
+def read_image_data(
+    input_path: Union[str, Store], input_key: Optional[str], from_s3: bool = False
+) -> np.typing.ArrayLike:
     """Read flamingo image data, stored in various formats.
 
     Args:
@@ -76,10 +94,16 @@ def read_image_data(input_path: Union[str, Store], input_key: Optional[str]) ->
             Access via S3 is only supported for a zarr container.
         input_key: The key (= internal path) for a zarr or n5 container.
             Set it to None if the data is stored in a tif file.
+        from_s3: Whether to read the data from S3.
 
     Returns:
         The data, loaded either as a numpy mem-map, a numpy array, or a zarr / n5 array.
     """
+    if from_s3:
+        assert input_key is not None
+        s3_store, fs = get_s3_path(input_path)
+        return zarr.open(s3_store, mode="r")[input_key]
+
     if input_key is None:
         input_ = read_tif(input_path)
     elif isinstance(input_path, str):

flamingo_tools/measurements.py

@@ -3,7 +3,7 @@
 import warnings
 from concurrent import futures
 from functools import partial
-from typing import List, Optional, Tuple
+from typing import List, Optional, Tuple, Union
 
 import numpy as np
 import pandas as pd
@@ -25,7 +25,7 @@
 
 def _measure_volume_and_surface(mask, resolution):
     # Use marching_cubes for 3D data
-    verts, faces, normals, _ = marching_cubes(mask, spacing=resolution)
+    verts, faces, normals, _ = marching_cubes(mask, spacing=(resolution,) * 3)
 
     mesh = trimesh.Trimesh(vertices=verts, faces=faces, vertex_normals=normals)
     surface = mesh.area
@@ -60,6 +60,9 @@ def _get_bounding_box_and_center(table, seg_id, resolution, shape, dilation):
         for bmin, bmax, sh in zip(bb_min, bb_max, shape)
     )
 
+    if isinstance(resolution, float):
+        resolution = (resolution,) * 3
+
     center = (
         int(row.anchor_z.item() / resolution[0]),
         int(row.anchor_y.item() / resolution[1]),
@@ -155,7 +158,7 @@ def _default_object_features(
         # The radius passed is given in micrometer.
         # The resolution is given in micrometer per pixel.
         # So we have to divide by the resolution to obtain the radius in pixel.
-        radius_in_pixel = background_radius / resolution
+        radius_in_pixel = background_radius / resolution if isinstance(resolution, (float, int)) else resolution[1]
         measures = _normalize_background(measures, image, background_mask, center, radius_in_pixel, norm, median_only)
 
     # Do the volume and surface measurement.
@@ -207,7 +210,7 @@ def _regionprops_features(seg_id, table, image, segmentation, resolution, backgr
     return features
 
 
-def get_object_measures_from_table(arr_seg, table, keyword="median"):
+def get_object_measures_from_table(arr_seg, table):
     """Return object measurements for label IDs wthin array.
     """
     # iterate through segmentation ids in reference mask
@@ -217,11 +220,11 @@ def get_object_measures_from_table(arr_seg, table, keyword="median"):
     if len(object_ids) < len(ref_ids):
         warnings.warn(f"Not all IDs were found in measurement table. Using {len(object_ids)}/{len(ref_ids)}.")
 
-    measure_values = [table.at[table.index[table["label_id"] == label_id][0], keyword] for label_id in object_ids]
+    median_values = [table.at[table.index[table["label_id"] == label_id][0], "median"] for label_id in object_ids]
 
     measures = pd.DataFrame({
         "label_id": object_ids,
-        keyword: measure_values,
+        "median": median_values,
     })
     return measures
 
@@ -252,7 +255,7 @@ def compute_object_measures_impl(
     image: np.typing.ArrayLike,
     segmentation: np.typing.ArrayLike,
     n_threads: Optional[int] = None,
-    resolution: Optional[Tuple[float, float, float]] = (0.38, 0.38, 0.38),
+    resolution: float = 0.38,
     table: Optional[pd.DataFrame] = None,
     feature_set: str = "default",
     background_mask: Optional[np.typing.ArrayLike] = None,
@@ -323,7 +326,7 @@ def compute_object_measures(
     image_key: Optional[str] = None,
     segmentation_key: Optional[str] = None,
     n_threads: Optional[int] = None,
-    resolution: Optional[Tuple[float, float, float]] = (0.38, 0.38, 0.38),
+    resolution: Union[float, Tuple[float, ...]] = 0.38,
     force: bool = False,
     feature_set: str = "default",
     s3_flag: bool = False,
@@ -375,8 +378,8 @@ def compute_object_measures(
         table = table[table["component_labels"].isin(component_list)]
 
     # Then, open the volumes.
-    image = read_image_data(image_path, image_key)
-    segmentation = read_image_data(segmentation_path, segmentation_key)
+    image = read_image_data(image_path, image_key, from_s3=s3_flag)
+    segmentation = read_image_data(segmentation_path, segmentation_key, from_s3=s3_flag)
 
     measures = compute_object_measures_impl(
         image, segmentation, n_threads, resolution, table=table, feature_set=feature_set,