Skip to content

Add logging infrastructure with debug output for scaled processing plugins#153

Draft
Copilot wants to merge 3 commits intomainfrom
copilot/add-logging-for-scaled-processing
Draft

Add logging infrastructure with debug output for scaled processing plugins#153
Copilot wants to merge 3 commits intomainfrom
copilot/add-logging-for-scaled-processing

Conversation

Copy link
Contributor

Copilot AI commented Nov 25, 2025

Adds library-style logging to zarrnii that can be easily controlled by calling scripts (e.g., Snakemake). Includes detailed debug output for scaled_processing plugins to help diagnose memory issues by exposing zarr chunking and dask graph details.

Changes

  • New zarrnii/logging.py module

    • get_logger(name) - hierarchical loggers under zarrnii.* namespace
    • configure_logging(level, format_string, handler, stream) - one-liner setup for calling scripts
    • Uses NullHandler by default per Python library conventions
  • Debug logging in apply_scaled_processing()

    • Zarr shapes, chunks, dtype at each step
    • Dask graph layers and task counts
    • Intermediate OME-Zarr paths
    • Lowres array memory footprint
  • Debug logging in scaled processing plugins

    • GaussianBiasFieldCorrection, N4BiasFieldCorrection, SegmentationCleaner
    • Input/output shapes, processing parameters, array statistics

Usage

import logging
from zarrnii import configure_logging, ZarrNii

# Enable debug output
configure_logging(level=logging.DEBUG)

# Or write to file
configure_logging(level=logging.DEBUG, handler=logging.FileHandler("zarrnii.log"))

# Scaled processing will now emit detailed debug logs
result = znii.apply_scaled_processing(plugin, downsample_factor=4)

Example debug output:

2025-11-25 23:48:27 - zarrnii.core - INFO - Starting scaled processing with plugin: Gaussian Bias Field Correction (downsample_factor=4)
2025-11-25 23:48:27 - zarrnii.core - DEBUG - Input zarr - shape: (1, 256, 256, 256), chunks: (1, 64, 64, 64), dtype: float32
2025-11-25 23:48:27 - zarrnii.core - DEBUG - Upsampled dask graph layers: 3, total tasks: 125
Original prompt

Can yoh add logging to zarrnii, in such a way that it is easily controlled by calling scripts (like the snakemake script logging in spimquant). Also add some debug level output that can be enabled, specifically want this adddd for the scaled_processing plugins, giving info on the zarrs being used (chunking) and the dask graphs, tryibg to debug some memroy issues


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI and others added 2 commits November 25, 2025 23:43
…ugins

Co-authored-by: akhanf <11492701+akhanf@users.noreply.github.com>
… error handling

Co-authored-by: akhanf <11492701+akhanf@users.noreply.github.com>
Copilot AI changed the title [WIP] Add logging to scaled_processing plugins for debugging Add logging infrastructure with debug output for scaled processing plugins Nov 25, 2025
Copilot AI requested a review from akhanf November 25, 2025 23:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants