Missing output channel description for instance segmentation

[danifranco]

Hello,

Currently, instance segmentation models ouput various channels to be used to reconstruct the final instances. As this post-processing part needs to be done by the consumer software, as currently no post-processing is provided with the models, we need to know where to place/read what is representing each channel. This information is crucial in order to create the final instances afterwards using a post-processing such as marker controlled watershed, multicut, agglomeration etc. For example: foreground + contour, foreground + contour + distances etc.

We can use the name convention we are following in BiaPy, but it can be other (we just need to stablish one). Somewhere in the model description the following information should be contained:

• Type of method, e.g., "bottom-up" or "top-down"
• For the "bottom-up" approaches, each channel representation: "BC", "BCD", "C", "D" ...

Where each letter corresponds to a channel and where the order matches model's output. In BiaPy we are currently using these ones:

• 'B' stands for 'Binary segmentation', containing each instance region without the contour.
• 'C' stands for 'Contour', containing each instance contour.
• 'D' stands for 'Distance', each pixel containing the distance of it to the instance contour.
• 'M' stands for 'Mask', contains the B and the C channels, i.e. the foreground mask. Is simply achieved by binarizing input instance masks.
• 'Dv2' stands for 'Distance V2', which is an updated version of D channel calculating background distance as well.
• 'P' stands for 'Points' and contains the central points of an instance (as in Detection workflow)
• 'A' stands for 'Affinities" and contains the affinity values for each dimension.

[FynnBe]

relates to #530

[danifranco]

Hi,

Until this is decided, where can I put a description for each channel? In the RDF maybe somewhere? Following the proposed taxonomy above, right now, and as far as I know, only B+C or A representations are in present in the zoo, but if we are about to upload different representations we would need to describe what is represented on each channel. If not, it will be super difficult for a com. partner to consume models automatically, as each case is threated differently when creating the final instances.