add support for anisotropic values of the flow3D_smooth

[yuriyzubov]

Currently flow3D_smooth only accepts a single integer, applying the same Gaussian smoothing sigma uniformly across all spatial axes (Z, Y, X).

If voxel spacing differs across axes, passing a vector value to the flow3D_smooth parameter (anisotropic smoothing) makes more sense.

Changes

• flow3D_smooth now accepts int or list[int] (length 3)
• If an int is passed, it is broadcast to [val, val, val] for backwards compatibility
• Updated docstring to reflect the new accepted types

Example

# Uniform smoothing
model.eval(img, do_3D=True, flow3D_smooth=2)

# Anisotropic smoothing
model.eval(img, do_3D=True, flow3D_smooth=[3, 0, 0])  

[mrariden]

@yuriyzubov I think it makes more sense to scale the flow3D_smooth's first term by anisotropy inside of CellposeModel.eval(). That way anisotropy of 1 will have the same effect as it does now.

I'm testing this now

[mrariden]

Hi @yuriyzubov please merge my ring_artifacts-refactor and this can be completed.

[codecov]

Codecov Report

❌ Patch coverage is 33.33333% with 22 lines in your changes missing coverage. Please review.
✅ Project coverage is 41.99%. Comparing base (8b3930e) to head (ef481bf).
⚠️ Report is 49 commits behind head on main.

Files with missing lines	Patch %	Lines
cellpose/io.py	27.77%	13 Missing ⚠️
cellpose/models.py	20.00%	8 Missing ⚠️
cellpose/transforms.py	80.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1408      +/-   ##
==========================================
- Coverage   42.39%   41.99%   -0.41%     
==========================================
  Files          16       16              
  Lines        3781     3777       -4     
==========================================
- Hits         1603     1586      -17     
- Misses       2178     2191      +13     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[mrariden]

Coverage appears to have decreased, but the code is actually tested through the CLI tests which also catches CLI argument parsing.

Also, anisotropy does not affect the flow smoothing in this implementation. The two are kept separate for consistency with the past code.

Resolves #1398

[carsen-stringer]

@yuriyzubov @mrariden the main reason to use flow3D_smooth is for the more cross-hatch like artifacts, I think those ring artifacts are rare - did you see them @yuriyzubov ?

[carsen-stringer]

(this could go after the first sentence on flow3D_smooth)
"If you are seeing increased fragmentation along the Z axis, or ring-artifacts, you can specify increased smoothing in the z-axis by providing a list, e.g. flow3D_smooth = [2, 1, 1]."

[carsen-stringer]

Just a comment here, if resample=True (default), then the flows are run at the original volume pixel size. For example with anisotropy 5, and size 100x300x300, the flows are computed at size 500x300x300, and the dynamics are run at 100x300x300. So actually to keep the smoothing size consistent you would want to technically decrease the smoothing in Z

[derekthirstrup]

Thanks @yuriyzubov, @mrariden, and @carsen-stringer for picking this up and getting it merged so quickly.

@carsen-stringer -- good point about resample=True running dynamics at the original volume pixel size. That's an important subtlety for users tuning the smoothing sigma.

For anyone landing on this issue, here's my practical guidance after working with this extensively on 3D confocal nuclei data (~6:1 Z anisotropy):

Recommendation: use anisotropy=1 as the default

Setting anisotropy=1 avoids the ring artifacts entirely and has an additional benefit that's easy to overlook: it eliminates a significant runtime penalty. With correct anisotropy (e.g., 6.15), a 20-slice stack gets upsampled to ~123 synthetic Z-planes before network inference. That's ~6x more 2D predictions to run through the network and a correspondingly larger 3D flow field for dynamics. For large datasets this adds up fast. Setting anisotropy=1 skips all of that.

The tradeoff is that Z-flow resolution is coarser with anisotropy=1. In practice, this manifests as segmentation boundaries that can be off by a few pixels in the upper and lower Z-planes of each object -- where the cross-section is changing most rapidly between slices and the sparse Z-sampling provides less information for the Euler integration to converge accurately. For most downstream analyses (counting, tracking, volume estimation), this is a much more acceptable tradeoff than ring artifacts.

If you want to tighten up those peripheral-plane boundaries without reintroducing artifacts, the new anisotropic smoothing can help even with anisotropy=1:

masks, flows, styles, diams = model.eval(
    image,
    do_3D=True,
    anisotropy=1,
    flow3D_smooth=[1, 0, 0],  # Light Z-only smoothing to regularize coarse Z-flows
    channels=[0, 0]
)