Support Indicator-Specific and Threshold-Flexible Transient Detection

[pauladkisson]

Summary

The current transient detection algorithm uses a single global threshold configuration that does not generalize well across fluorescence indicators with different signal dynamics. GCaMP and dLight produce signals with different amplitude distributions, noise floors, and event kinetics, such that a threshold calibrated for one indicator may produce excessive false positives or false negatives for the other. The lab requests support for indicator-specific detection parameters and for a percentage-based dF/F threshold as an alternative detection method.

Motivation

• Rodrigo Paz (Nelson lab) explicitly noted: "we found out that actually the same method does not detect... it's not useful for both signals" when comparing GCaMP and dLight experiments within the same lab
• The Nelson lab defines transients operationally as changes in fluorescence greater than 2% dF/F — a direct threshold on the dF/F signal — rather than the z-score MAD-based approach currently implemented in src/guppy/analysis/transients.py
• The current transientsThresh and highAmpFilt parameters in GuPPyParamtersUsed.json are single global scalars shared across all channels in a session; there is no mechanism to configure them differently per storename or indicator type
• Venus Sherathiya (GuPPy) mentioned that transient detection can have false positives/negatives and that parameters can be tuned, but acknowledged the tuning is indicator-specific — currently requiring users to run separate sessions with different parameter files

Proposed Solution

• Add a transient_detection_method parameter to GuPPyParamtersUsed.json with at least two options:
  • "mad_zscore" — existing behavior using MAD-based threshold on z-scored signal
  • "percent_dff" — threshold on raw dF/F percentage change (e.g., >2%), finding local maxima exceeding the threshold
• Extend the parameter schema to support per-storename overrides: a dict keyed by storename label allowing transientsThresh, highAmpFilt, movingWindow, and transient_detection_method to be set independently per channel
• Refactor the threshold logic out of the monolithic processChunks function in src/guppy/analysis/transients.py into a strategy/dispatch pattern so that new detection methods can be added without modifying the core chunking loop
• The GUI should allow selection of detection method and expose the per-channel override mechanism (likely as an advanced/expandable section in the Step 5 panel)
• Maintain the existing defaults for backward compatibility — sessions that do not set these new parameters should behave identically to the current behavior

Open Questions

• Should per-channel overrides be stored as a flat list (ordered by channel index) or as a dict keyed by storename label? A dict is more robust to channel reordering but requires changes to how inputParameters is consumed throughout the pipeline
• When using "percent_dff", should the artifact-removal chunking logic (createChunks) still be applied, or is a whole-recording pass preferable given that dF/F threshold detection doesn't have the same sensitivity to slow drift as MAD estimation?
• Does the 2% threshold apply to the filtered/smoothed dF/F output from deltaFF, or to the raw dF/F before filtering?
• Should the GUI enforce that the selected detection method is compatible with the selected signal type (selectForTransientsComputation already supports "z_score" vs. "dff")? For example, "percent_dff" method should require "dff" signal type selection
• Is there interest in supporting derivative-based detection (e.g., using the first derivative of the signal to detect rising edges) as a third method, given that Alexandra asked about this during the meeting?