You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/source/project_structure.md
+23-15Lines changed: 23 additions & 15 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -104,9 +104,9 @@ A session *may* include a `Clips` folder containing short video segments and the
104
104
* Clip filenames *must* follow the pattern: `sub-<subjectID>_ses-<sessionID>_cam-<camID>_start-<frameID>_dur-<nFrames>.mp4`.
105
105
*`<frameID>` in the `start` field *must* be the 0-based index of the first frame of the clip in the session video, padded to a consistent width (e.g. `0500`, `1000`).
106
106
*`<nFrames>` in the `dur` field *must* be the duration of the clip in number of frames (e.g. `5`, `30`).
107
-
* A single label file *must* be provided per clip file:
108
-
* In the `Train` split it's named `sub-<subjectID>_ses-<sessionID>_cam-<camID>_start-<frameID>_dur-<nFrames>_cliplabels.json` and provides keypoint annotations for every frame in the clip. See [Clip labels](target-cliplabels) for details.
109
-
* In the `Test` split it's named `sub-<subjectID>_ses-<sessionID>_cam-<camID>_start-<frameID>_dur-<nFrames>_startlabels.json` and provides keypoint annotations only for the first frame of each clip. See [Clip start labels](target-startlabels) for details.
107
+
* A single label file *must* be provided per clip:
108
+
* In the `Train` split, the file is named `sub-<subjectID>_ses-<sessionID>_cam-<camID>_start-<frameID>_dur-<nFrames>_cliplabels.json` and contains keypoint annotations for every frame in the clip. See [Clip labels](target-cliplabels) for details.
109
+
* In the `Test` split, the file is named `sub-<subjectID>_ses-<sessionID>_cam-<camID>_start-<frameID>_dur-<nFrames>_startlabels.json` and contains keypoint annotations only for the first frame of the clip. See [Clip start labels](target-startlabels) for details.
110
110
111
111
## File naming
112
112
@@ -117,7 +117,12 @@ All filenames follow a key-value pair convention, similar to the [BIDS standard]
117
117
<key>-<value>_<key>-<value>.<extension>
118
118
<key>-<value>_<key>-<value>_<suffix>.<extension>
119
119
```
120
-
The recognised suffixes are `framelabels` (for frame label files), `cliplabels` (for clip label files), and `startlabels` (for clip start label files).
120
+
The recognised suffixes are:
121
+
122
+
*`framelabels` for [frame label files](target-framelabels).
123
+
*`cliplabels` for [clip label files](target-cliplabels).
124
+
*`startlabels` for [clip start label files](target-startlabels).
125
+
121
126
* The following keys are used:
122
127
123
128
| Key | Description | Examples |
@@ -137,22 +142,23 @@ All filenames follow a key-value pair convention, similar to the [BIDS standard]
137
142
## Label format
138
143
139
144
* The `Train` split includes ground-truth keypoint annotations both for the sampled frames (`framelabels.json`) and for entire clips (`cliplabels.json`), if present.
140
-
* The `Test` split only includes keypoint annotations for the first frame of each clip (`startlabels.json`), if clips are present. Labels for frames and entire clips are withheld to support evaluation of pose estimation and point tracking methods.
145
+
* The `Test` split includes keypoint annotations only for the first frame of each clip (`startlabels.json`), if clips are present. Labels for frames and entire clips are withheld to support evaluation of pose estimation and point tracking methods.
141
146
* Labels *must* be stored in the same folder as the corresponding frames or clips.
142
-
* Labels *must* be stored in [COCO keypoints format](https://cocodataset.org/), with some additional requirements described below. Each label file is a JSON file with `images`, `annotations`, and `categories` arrays. Image, annotation and category `id` values *must* be unique integers within a label file.
147
+
* Labels *must* be stored in [COCO keypoints format](https://cocodataset.org/#format-data), with additional requirements described below. Each label file is a JSON file with `images`, `annotations`, and `categories` arrays. Image, annotation and category `id` values *must* be unique integers within a label file.
143
148
144
149
:::{note}
145
150
Annotation and category `id` values *should* be 1-indexed. This convention follows sleap-io's [`save_coco`](https://io.sleap.ai/latest/reference/sleap_io/io/coco/) function and avoids conflicts with models that treat category `0` as background.
146
151
147
-
Image `id` values are always 0-indexed. However, the indexing origin differs between frame labels and clip labels; clip start labels follow the same conventions as clip labels — see below for details.
152
+
Image `id` values are always 0-indexed. The indexing origin differs for frame labels and clip labels, and clip start labels follow the same conventions as clip labels. Details are provided below.
148
153
:::
149
154
150
155
(target-framelabels)=
151
156
### Frame labels (`framelabels.json`)
152
157
153
-
* In the `Train` split, there *must* be one `framelabels.json` per camera view within the `Frames` folder.
158
+
* Frame labels *must* only exist in the `Train` split.
159
+
* Within the `Frames` folder, there *must* be one frame label file per camera view, named `sub-<subjectID>_ses-<sessionID>_cam-<camID>_framelabels.json`.
154
160
* Each entry in the `images` array *must* have an `id` equal to the 0-based frame index in the session video (matching the `<frameID>` in the corresponding image filename).
155
-
* Each entry in the `images` array *must* have a `file_name` that matches the full filename (including extension) of an existing frame image in the `Frames` folder.
161
+
* Each entry in the `images` array *must* have a `file_name` that exactly matches the name of an existing [frame image](#frames) in the `Frames` folder (including the extension).
156
162
157
163
:::{admonition} Example
158
164
:class: tip
@@ -175,11 +181,12 @@ Here each `id` is the 0-based frame index in the session video (matching the `<f
175
181
(target-cliplabels)=
176
182
### Clip labels (`cliplabels.json`)
177
183
178
-
* In the `Train` split, there *must* be one `cliplabels.json` per clip.
184
+
* Clip labels *must* only exist in the `Train` split.
185
+
* If a `Clips` folder is present, there *must* be one clip label file per clip, named `sub-<subjectID>_ses-<sessionID>_cam-<camID>_start-<frameID>_dur-<nFrames>_cliplabels.json`.
179
186
* The `images` array *must* contain an entry for every frame in the clip, in consecutive, monotonically increasing order (covering the entire clip duration).
180
187
* Clip labels follow the same COCO keypoints format as frame labels, but with different conventions for image `id` and `file_name` values:
181
188
* Each image `id`*must* be the **0-based index of the frame within the clip** (i.e. `0`, `1`, `2`, ...), not the index in the session video.
182
-
* Each `file_name`*must* follow the same pattern as frame image filenames, but **without the extension**. The `frame` field in the `file_name`*must*hold the index of that frame in the **session video**.
189
+
* Each `file_name`*must* follow the same pattern as [frame image filenames](#frames), but **without the extension**. The `frame` field in the `file_name`*must*correspond to the index of that frame in the **session video**.
183
190
184
191
This means that each entry in the `images` array encodes two pieces of information: the `id` gives the local position within the clip, while the `frame` field in `file_name` gives the global position in the session video. Note that in both cases the indices are 0-based.
185
192
@@ -204,14 +211,15 @@ Here `id: 0` through `id: 4` are the local clip indices, while `frame-1000` thro
204
211
(target-startlabels)=
205
212
### Clip start labels (`startlabels.json`)
206
213
207
-
* Clip start labels are included in the `Test` split only. If a `Clips` folder is present, each clip *must* have a corresponding clip start label file named `sub-<subjectID>_ses-<sessionID>_cam-<camID>_start-<frameID>_dur-<nFrames>_startlabels.json`.
208
-
* Clip start labels provide keypoint annotations for the **first frame of the clip only**, and are intended to support point tracker evaluation (i.e. providing the initial point positions from which a tracker should propagate).
209
-
* Clip start labels follow the same COCO keypoints format and conventions as clip labels, with one difference: the `images` array *must* contain exactly one entry, for the first frame of the clip (`id: 0`).
214
+
* Clip start labels *must* only exist in the `Test` split.
215
+
* If a `Clips` folder is present, there *must* be one clip start label file per clip, named `sub-<subjectID>_ses-<sessionID>_cam-<camID>_start-<frameID>_dur-<nFrames>_startlabels.json`.
216
+
* Clip start labels provide keypoint annotations for the **first frame of the clip only**. They are intended for point-tracker evaluation, where the annotated points serve as the initial positions from which a tracker should propagate.
217
+
* Clip start labels are identical to [Clip labels](target-cliplabels), except that the `images` array *must* contain exactly one entry corresponding to the first frame of the clip, and therefore must have `id: 0`.
210
218
211
219
:::{admonition} Example
212
220
:class: tip
213
221
214
-
For a clip starting at frame 1000, the `images` array in the `startlabels.json` would be:
222
+
For a clip starting at frame 1000 with a duration of 5 frames, the `images` array would be:
0 commit comments