Merge pull request #27 from fractal-analytics-platform/ngio_02

Ngio 0.2 refactor

Author: jluethi

.github/workflows/build_and_test.yml

@@ -17,7 +17,7 @@ jobs:
     strategy:
       matrix:
         os: [ubuntu-22.04, macos-latest]
-        python-version: ["3.10", "3.11", "3.12"]
+        python-version: ["3.11", "3.12"]
         exclude:
           - os: macos-latest
             python-version: '3.10'
@@ -50,7 +50,7 @@ jobs:
           key: pooch-cache
 
       - name: Check if manifest has changed
-        run: fractal-manifest check --package fractal-helper-tasks
+        run: fractal-manifest check --package fractal-helper-tasks --fractal-server-2-13
 
       - name: Test tasks with pytest
         run: pytest --color=yes --cov --cov-report=xml --cov-report=term-missing -s --log-cli-level debug

.gitignore

@@ -109,4 +109,7 @@ ENV/
 
 # IDE settings
 .vscode/
-.idea/
+.idea/
+
+# Local dev files
+examples/test_rechunk_zarr.ipynb

README.md

@@ -9,66 +9,9 @@ Collection of Fractal helper tasks
 
 ## Development instructions
 
-This instructions are only relevant *after* you completed both the `copier
-copy` command and the git/GitLab/GitHub initialization phase - see
-[README](https://github.com/fractal-analytics-platform/fractal-tasks-template#readme)
-for details.
-
-1. It is recommended to work from an isolated Python virtual environment:
-```console
-# Create the virtual environment in the folder venv
-python -m venv venv
-# Activate the Python virtual environment
-source venv/bin/activate
-# Deactivate the virtual environment, when you don't need it any more
-deactivate
+To create the manifest:
 ```
-2. You can install your package locally as in:
-```console
-# Install only fractal_helper_tasks:
-python -m pip install -e .
-# Install both fractal_helper_tasks and development dependencies (e.g. pytest):
-python -m pip install -e ".[dev]"
+fractal-manifest create --package fractal_helper_tasks --fractal-server-2-13
 ```
 
-3. Enjoy developing the package.
-
-4. The template already includes a sample task ("Thresholding Task"). Whenever
-you change its input parameters or docstring, re-run
-```console
-python src/fractal_helper_tasks/dev/create_manifest.py
-git add src/fractal_helper_tasks/__FRACTAL_MANIFEST__.json
-git commit -m'Update `__FRACTAL_MANIFEST__.json`'
-git push origin main
-```
-
-5. If you add a new task, you should also add a new item to the `task_list`
-property in `src/fractal_helper_tasks/__FRACTAL_MANIFEST__.json`. A minimal example
-may look like
-```json
-    {
-      "name": "My Second Task",
-      "executable": "my_second_task.py",
-      "input_type": "zarr",
-      "output_type": "zarr",
-      "meta": {
-        "some-property": "some-value"
-      },
-    }
-```
-Notes:
-
-* After adding a task, you should also update the manifest (see point 4 above).
-* The minimal example above also includes the `meta` task property; this is optional, and you can remove it if it is not needed.
-
-6. Run the test suite (with somewhat verbose logging) through
-```console
-python -m pytest --log-cli-level info -s
-```
-7. Build the package through
-```console
-python -m build
-```
-This command will create the release distribution files in the `dist` folder.
-The wheel one (ending with `.whl`) is the one you can use to collect your tasks
-within Fractal.
+Refer to the developers-guide in the [Fractal template repo](https://github.com/fractal-analytics-platform/fractal-tasks-template/blob/main/DEVELOPERS_GUIDE.md) for more detailed instructions.

pyproject.toml

@@ -25,10 +25,9 @@ authors = [
 ]
 
 # Required Python version and dependencies
-requires-python = ">=3.10"
+requires-python = ">=3.11"
 dependencies = [
-    "fractal-tasks-core==1.4.2",
-    "ngio==0.1.6",
+    "ngio>=0.2.2,<0.3.0",
     "fractal-task-tools==0.0.12",
 ]
 

src/fractal_helper_tasks/__FRACTAL_MANIFEST__.json

@@ -10,7 +10,6 @@
         "Singleton time dimension"
       ],
       "docs_info": "### Purpose\n- Removes a **singleton time (T) dimension** from an OME-Zarr image.  \n- Creates a new OME-Zarr image with updated metadata and dimensions.\n- Optionally overwrites the input image if `overwrite_input` is set to True.\n\n### Outputs\n- A **new Zarr image** without the singleton T-dimension, stored with a configurable suffix.  \n\n### Limitations\n- Only processes OME-Zarr images where the **T-axis is the first axis**.  \n- Assumes the T-dimension is **singleton**; does not process non-singleton time axes.  \n- Does not copy associated **label images** or **ROI tables** to the new Zarr structure.  ",
-      "type": "parallel",
       "executable_parallel": "drop_t_dimension.py",
       "meta_parallel": {
         "cpus_per_task": 2,
@@ -58,7 +57,6 @@
         "2D to 3D workflows"
       ],
       "docs_info": "### Purpose\n- Converts a **2D segmentation** image into a **3D segmentation** by replicating the 2D segmentation across Z-slices.  \n- Supports OME-Zarr datasets where **2D and 3D images** share the same base name but differ by suffixes.  \n- Optionally copies associated ROI tables and adjusts them to align with the replicated Z-dimensions.  \n\n### Outputs\n- A **3D segmentation label image** saved with a new name.  \n- Updated **ROI tables** adjusted for Z-dimensions (optional).  \n\n### Limitations\n- Only supports **same-base 2D and 3D Zarr names**; full flexibility in file names is not yet implemented.  \n- Assumes **2D OME-Zarr images** and corresponding 3D images are stored in the same base folder and just differ with a suffix before the .zarr.  \n",
-      "type": "parallel",
       "executable_parallel": "convert_2D_segmentation_to_3D.py",
       "meta_parallel": {
         "cpus_per_task": 2,
@@ -78,18 +76,18 @@
             "description": "Name of the label to copy from 2D OME-Zarr to 3D OME-Zarr"
           },
           "level": {
-            "default": 0,
+            "default": "0",
             "title": "Level",
-            "type": "integer",
-            "description": "Level of the 2D OME-Zarr label to copy from"
+            "type": "string",
+            "description": "Level of the 2D OME-Zarr label to copy from. Valid choices are \"0\", \"1\", etc. (depending on which levels are available in the OME-Zarr label)."
           },
-          "ROI_tables_to_copy": {
+          "tables_to_copy": {
             "items": {
               "type": "string"
             },
-            "title": "Roi Tables To Copy",
+            "title": "Tables To Copy",
             "type": "array",
-            "description": "List of ROI table names to copy from 2D OME-Zarr to 3D OME-Zarr"
+            "description": "List of tables to copy from 2D OME-Zarr to 3D OME-Zarr"
           },
           "new_label_name": {
             "title": "New Label Name",
@@ -100,7 +98,7 @@
             "items": {},
             "title": "New Table Names",
             "type": "array",
-            "description": "Optionally overwriting the names of the ROI tables in the 3D OME-Zarr"
+            "description": "Optionally overwriting the names of the tables in the 3D OME-Zarr"
           },
           "plate_suffix": {
             "default": "_mip",
@@ -146,7 +144,6 @@
         "Many files"
       ],
       "docs_info": "### Purpose\n- Rechunks OME-Zarr to new chunking parameters: Changes whether the array is stored as many small files or few larger files.\n- Optionally applies the same rechunking to label images.\n\n### Outputs\n- A **new Zarr image** that is rechunked.\n",
-      "type": "parallel",
       "executable_parallel": "rechunk_zarr.py",
       "meta_parallel": {
         "cpus_per_task": 1,