Skip to content

README.md

Latest commit

 

History

History
233 lines (183 loc) · 10.5 KB

README.md

File metadata and controls

233 lines (183 loc) · 10.5 KB
Raw

demixs — engine, GUI and CLI

The analysis extracted from the numbered pipeline folders into an importable package, so a GUI, a command line and a script can all drive the same code.

demixs/
├── engine/       pure analysis — numpy, cv2, skimage, scipy. NO Qt, no print,
│   │             no file writes, no sys.exit
│   ├── params.py     parameter dataclasses, the schema, and param_set hashing
│   ├── imageio.py    loading and calibration
│   ├── background.py noise scale and the signal image
│   ├── cells.py      cell segmentation
│   ├── detect.py     LoG, watershed, half-max, measure, filters
│   ├── overlay.py    renders to an ndarray, never saves
│   ├── ratio.py      IN/OUT signal fraction, independent of detection
│   ├── convert.py    .oir -> TIFF via Bio-Formats through JPype
│   └── pipeline.py   Session (staged) and run_detection (one-shot)
├── io/writers.py CSV and JSON output
├── gui/settings.py  the one place persistent "Recent" state is opened;
│                 DEMIXS_SETTINGS_DIR redirects it
├── report.py     progress and log callbacks
├── gui/          PySide6 shell + modules. Imports engine; engine never
│                 imports this
└── cli/main.py   demixs <subcommand>

Install and run

conda activate demixs
pip install -e .            # from the repository root

demixs gui                  # the explorer (Convert, Detection, IN/OUT ratio)
demixs convert oir_folder/  # .oir -> lossless TIFF, no Fiji
demixs detect image.tif     # one field
demixs ratio  image.tif     # IN/OUT signal fraction
demixs batch  "dir/*.tif"   # many, under one frozen param set
demixs describe             # what can be tuned

The idea that makes the GUI interactive

The pipeline is a cascade, and the stages have wildly different costs. Measured on a 1024×1024 field with 120 candidates:

tier stages cost re-run when
A load, top-hat, background, LoG, watershed, half-max, measure 1460 ms generator params change
B cell segmentation 42 ms cell params change
C filters, summaries, CSVs 0.02 ms filter params change
D overlay redraw ~100 ms anything visual

Tier C is ~78,000× cheaper than tier A, so filter sliders update live. That only works because measure() returns every candidate with a reject_reason, rather than discarding rejects as the original script did.

Tier B depends only on img and bg_sigma, so cell settings never force detection to repeat.

Parameter tiers

The tier is the user-facing property, and it is encoded in the schema, the GUI group colour and the CLI's describe output from one definition.

  • live — min_peak_snr, min_solidity, r_*_keep_um. Free.
  • cells — segmentation. Cheap. Deliberately outside the hash, so a run with cells on and one with cells off stay poolable.
  • advanced — sigma estimator, LoG, top-hat, overlap, boundary fraction, detection radii. Changes param_set, which invalidates hand labels.
  • hidden — correct as set; not worth exposing.

The radius split

v4 used R_MIN_UM/R_MAX_UM both to set the LoG scale range and as a post-hoc size filter, and listed them in both the generator and filter blocks. So a "filter" change silently altered detection and forked the hash. They are now r_*_det_um (hashed) and r_*_keep_um (free), with equal defaults. They are not the same quantity: the LoG value is a blob radius at peak response, the filter is sqrt(area/pi) of the trimmed region.

param_set

Eight hex characters over the generator block, reproducing v4's hash exactly — the reference field hashes to d8b83243 under the defaults, asserted in the tests. Every CSV row carries it.

Int-valued fields are coerced on construction, because 12 and 12.0 serialise differently and a Qt spin box returns floats; without that the hash would fork the first time anyone touched a slider.

Deliberate differences from v4

  1. A zero is a result. v4 called sys.exit(0) mid-flow when nothing survived, so the field wrote no per-object file, no per-cell file, and vanished from aggregates. The engine returns normally with n_kept == 0 and writes the zero rows.

  2. Output is one folder per image, under results_detection/ (ratio uses results_ratio/). v4 wrote to results_v3/, silently overwriting v3 output, and grouped files by TYPE - csv/, summary/, params/ - so the five files describing one field were scattered across five directories:

    <image folder>/results_detection/<image name>/
    <image name>.csv              one row per condensate
    <image name>_summary.csv      one row per image
    <image name>_percell.csv      one row per cell
    <image name>_params.json      parameters + param_set
    <image name>_overlay.png

  3. Calibration is not a global. v4 rebound PIXEL_SIZE_UM from the TIFF tag, so in a long-lived process the next image inherited the previous one's calibration whenever its own tag was missing.

  4. Directories are created by writers, not by computing a path.

Everything else is byte-identical, enforced by tests/test_golden.py over ten fields spanning three datasets.

Conversion

demixs convert <folder|glob|file> turns .oir into lossless 16-bit TIFF without launching Fiji. It reuses the Bio-Formats jars and bundled JDK already inside the Fiji install — nothing is downloaded — and starts one JVM for the whole run rather than one per file.

Verified bit-identical to the Fiji macro's output: 10/10 fields, pixelwise difference min = max = 0, on both 1-channel and 2-channel data.

Two details it has to get right, both covered by tests:

  • The calibration tag. imageio.read_pixel_size_um needs XResolution and the word "micron" in ImageDescription. tifffile appends its own {"shape": ...} description by default, which is written second and wins — so metadata=None is required or the calibration silently disappears and every micron figure reverts to a hard-coded constant.
  • Channel order. Plane 0 → _green (C1), plane 1 → _pink (C2).

Output mirrors the Fiji macro's layout, and for the same reason:

<dataset>/converted/
    RAW/<base>_green.tif    16-bit, unannotated   <- QUANTIFY THIS
    FIG/<base>_green.tif     8-bit, scale bar     <- FIGURES ONLY

Quantising FIG loses real condensates (47 vs 49 on a test field) and the burned-in bar is ~4.9 µm², compact and high-solidity, so a detector counts it as a condensate and it drags the background estimate up. Separate folders are what stop that happening by accident. --no-fig skips the copies; --scalebar 0 writes them without a bar.

The root defaults to <dataset>/converted/, beside the oir/ folder rather than inside it. Existing files are skipped so a run can be resumed; --overwrite to force. --verify re-reads each written file and compares it to the plane in memory.

FIJI_HOME overrides where Bio-Formats is found. The JVM cannot be restarted in a process, so changing it needs a restart — the engine says so rather than silently ignoring the change.

Where results go

Anchored on the image, never the working directory, one folder per image:

module tree
detection <image folder>/results_detection/<image name>/
ratio <image folder>/results_ratio/<image name>/

Separate trees because they are independent measurements; mixing them makes it impossible to tell which analysis produced what. --out overrides the root.

In the GUI, auto-write is on by default - the common case is analysing a folder and wanting the results. Untick "Write automatically after each image" to browse without writing.

GUI: the shared widgets

Every module is assembled from demixs/gui/components/, not from hand-rolled Qt. A module chooses which widget it needs; it never decides what one looks like. Adding a per-module variant is how four panels end up with four kinds of Reset.

widget what it settles
buttons.PrimaryAction the one filled action of a screen, plus the one-line reason it is disabled (tooltip = same text)
buttons.secondary_button everything else, Cancel included
collapsible.CollapsibleGroup chevron + name + speed tag left, Reset right, in the header row
sliders.LabeledSlider label / printed min / track / printed max / spin / unit, on one fixed grid
sliders.RangeRow a two-ended parameter as one dual-handle control
image_canvas.ImageCanvas view tabs, Fit / 1:1 / zoom, auto-contrast, overlay fade, zoom readout, hint
results_header.ResultsHeader count + current file + Copy / Export, identical in both result panels
empty_state canvas: two lines and an action; side panel: one muted line
scroll.StickyScroll one scroll column that pins the current group's name
segmented.SegmentedControl Basic / All parameters, as one exclusive row

Four button roles and no fifth: filled accent = the one primary action, outlined = everything else, grey = disabled only, red = destructive only. Cancel is outlined - it discards nothing - and exists only while a run is in progress.

Units are styles.unit_text / decimals_for: the schema keeps ASCII (um, um^2, sigma) because those strings go into CSV headers, and the GUI never prints them - it prints µm, µm² and σ, with one decimal rule per unit, in a static label beside the spin box rather than inside the editable field. The rule is widened only where a parameter's own step is finer than it, so no default is rounded away by its own display.

The status bar is two labels: module · file · field n of m on the left, from the module's context_text(), and the last outcome plus a timestamp on the right, from its status signal.

For agents

demixs describe --json gives the full parameter schema with tiers and ranges. --json on any command gives one parseable object with counts, paths, param_set, qc_flags and warnings. Exit codes: 0 ok, 2 bad input, 3 nothing usable (including a genuine zero, which is worth noticing).

qc_flags is the field to branch on: no_objects, saturation_ceiling, objects_outside_cells, merged_cells, calibration_fallback.

Or import it: from demixs.engine.pipeline import run_detection.

Testing

pytest                      # 139 tests

tests/golden/ holds reference output captured from the original v4 script before any refactoring. Regenerate it only when a change to the measurement is intended and understood.