Valid COCO JSON files for frames and clips

[niksirbi]

• Check that ethology's ValidCOCO validator work on poseinterface-converted COCO JSON files.
• Think about the COCO JSON file for clips and the corresponding validator. Compared to the COCO JSON file for frames, this may need additional constraints, e.g. that the included frames are consecutively increasing by 1, and match the range specified in the clip's filename.

[lochhh]

afaik we've only converted labelled data from a single video/clip.
do we already know if sleap-io outputs COCO JSON file for clips, if the labelled data spans multiple videos?

[niksirbi]

if the labelled data spans multiple videos?

I think in the end we discussed saving each clip's labels to a separate JSON file, right?

[lochhh]

yup, but based on the description of this issue, I assume we will eventually want to cover both cases, but prioritise COCO JSON file for frames

[niksirbi]

The way I see it, the COCO json files for those two cases will have the same format, with the one for labelled "clips" being a special case of the one for labelled frames. The things that make the clip case special:

• All frames of the clip are labelled (as opposed to a few frames scattered across a video)
• We need to check that the frame indices in the clip's COCO JSON agree with the start-{FrameID}_dur-{NumFrames} part of the clip's file name

But yes, priority is the general-purpose COCO JSON validator. The additional constraints for clips can be added on later.

[lochhh]

Ah ok I misunderstood the clip case as what's described in #pose-benchmarks > Data formats @ 💬

[niksirbi]

Yeah, some of that discussion is now outdated.

[lochhh]

But it is still a valid case we want to support in the future, i.e. generating COCO json file that contains labels for multiple videos; in addition to COCO json file for a single video (fully or partially labelled). For instance, a single sleap labels object can contain labels from multiple videos - for this we want to know how sleap-io outputs the COCO json file.

[niksirbi]

Ah yes, that is definitely in scope for poseinterface as a project, just not immediately required in the next months.