Moar examples + Dataset.{compress,downsample} (#813)

* adds 2 new examples and Dataset.compress and Dataset.downsample

* add examples to docs

* fixes

* fixes?

* changelog

* Apply suggestions from code review

Co-authored-by: Jonathan Striebel <[email protected]>

* pr feedback

* pr feedback

* fix circular import

* add snapshot, minor changes

* explicit segment_ids

* fix

* Apply suggestions from code review

Co-authored-by: Jonathan Striebel <[email protected]>

Co-authored-by: Jonathan Striebel <[email protected]>
Co-authored-by: Jonathan Striebel <[email protected]>

Author: normanrz

docs/mkdocs.yml

@@ -77,6 +77,7 @@ nav:
   - Usage Examples:
       - Dataset Examples:
         - webknossos-py/examples/dataset_usage.md
+        - webknossos-py/examples/upload_tiff_stack.md
         - webknossos-py/examples/upload_image_data.md
         - webknossos-py/examples/download_image_data.md
         - webknossos-py/examples/remote_datasets.md
@@ -86,6 +87,7 @@ nav:
         - webknossos-py/examples/learned_segmenter.md
         - webknossos-py/examples/skeleton_synapse_candidates.md
         - webknossos-py/examples/calculate_segment_sizes.md
+        - webknossos-py/examples/download_segments.md
       - Administration Examples:
         - webknossos-py/examples/user_times.md
         - webknossos-py/examples/annotation_project_administration.md

docs/src/webknossos-py/examples/download_segments.md

@@ -0,0 +1,9 @@
+# Download segment masks
+
+This example [accesses a volume annotation on webKnososs](../../api/webknossos/annotation/annotation.md#Annotation.open_as_remote_dataset) and creates segment masks from selected segments. The segment masks are stored as tiff sequences. The segments have been selected from the "Segments" tab in webKnossos. The [BufferedSliceReader](../../api/webknossos/dataset/view.md#View.get_buffered_slice_reader) is used to efficiently request sections from the remote segmentation data.
+
+```python
+--8<--
+webknossos/examples/download_segments.py
+--8<--
+```

docs/src/webknossos-py/examples/upload_tiff_stack.md

@@ -0,0 +1,9 @@
+# Upload tiff stack
+
+This example shows how to [create a new dataset from a tiff sequence](../../api/webknossos/dataset/dataset.md#Dataset.from_images), [compress](../../api/webknossos/dataset/dataset.md#Dataset.compress) it, [downsample](../../api/webknossos/dataset/dataset.md#Dataset.downsample) it and [upload](../../api/webknossos/dataset/dataset.md#Dataset.upload) it to webKnossos.
+
+```python
+--8<--
+webknossos/examples/upload_tiff_stack.py
+--8<--
+```

webknossos/Changelog.md

@@ -30,6 +30,8 @@ For upgrade instructions, please check the respective *Breaking Changes* section
   [#811](https://github.com/scalableminds/webknossos-libs/pull/811)
 - `skeleton.save()` now also accepts paths with a `.zip` suffix. [#811](https://github.com/scalableminds/webknossos-libs/pull/811)
 - Added `annotation.get_volume_layer_segments()` to interact with information from the `Segments` tab in annotations. This method returns a dict from segment ids to an object containing optional segment `name`, `color` and `anchor_position`. [#812](https://github.com/scalableminds/webknossos-libs/pull/812)
+- Added convenience methods `Dataset.compress` and `Dataset.downsample` for compressing and downsampling all layers and mags in a dataset. [#813](https://github.com/scalableminds/webknossos-libs/pull/813)
+- Added examples for downloading segment masks from webKnossos and cubing & uploading tiff stacks. [#813](https://github.com/scalableminds/webknossos-libs/pull/813)
 
 ### Changed
 

webknossos/examples/download_segments.py

@@ -0,0 +1,36 @@
+import numpy as np
+from tifffile import imwrite
+
+import webknossos as wk
+
+ANNOTATION_ID = "634e8fe1010000b4006f3cf4"
+SEGMENT_IDS = [32, 667325]
+MAG = wk.Mag("4-4-1")
+
+
+def main() -> None:
+    dataset = wk.Annotation.open_as_remote_dataset(
+        ANNOTATION_ID, webknossos_url="https://webknossos.org"
+    )
+    mag_view = dataset.get_segmentation_layers()[0].get_mag(MAG)
+
+    z = mag_view.layer.bounding_box.topleft.z
+    with mag_view.get_buffered_slice_reader() as reader:
+        for slice_data in reader:
+            slice_data = slice_data[0]  # First channel only
+            for segment_id in SEGMENT_IDS:
+                segment_mask = (slice_data == segment_id).astype(
+                    np.uint8
+                ) * 255  # Make a binary mask 0=empty, 255=segment
+                segment_mask = segment_mask.T  # Tiff likes the data transposed
+                imwrite(
+                    f"l4_sample_segments/seg{segment_id:04d}_mag{MAG}_z{z:04d}.tiff",
+                    segment_mask,
+                )
+
+            print(f"Downloaded z={z:04d}")
+            z += MAG.z
+
+
+if __name__ == "__main__":
+    main()

webknossos/examples/upload_tiff_stack.py

@@ -0,0 +1,23 @@
+from pathlib import Path
+from time import gmtime, strftime
+
+import webknossos as wk
+
+
+def main() -> None:
+    time_str = strftime("%Y-%m-%d_%H-%M-%S", gmtime())
+    dataset = wk.Dataset.from_images(
+        str(Path(__file__).parent.parent / "testdata" / "tiff"),
+        f"tiff_dataset_{time_str}",
+        voxel_size=(12, 12, 12),
+    )
+    dataset.compress()
+    dataset.downsample()
+
+    remote_dataset = dataset.upload()
+    url = remote_dataset.url
+    print(f"Successfully uploaded {url}")
+
+
+if __name__ == "__main__":
+    main()