Multiple bounding boxes

[Irenetema]

Hi,

I am working on the Snapshot Serengeti camera trap dataset for animal detection. The images have anywhere between 0 and more than 50 bounding boxes identifying animal locations. Is there a way in croissant to represent multiple bounding boxes within a single image?

The COCO example with bounding box on the croissant repo only handles 1 bounding box per image. When I try to follow similar logic, where I create a dictionary enumerating the bounding boxes, I get a read error with message "ValueError: All arrays must be of the same length".

Here's a snippet of my json annotation file:

{
  "0": {
    "image_path": "S6/P07/P07_R2/S6_P07_R2_IMAG0077.JPG", 
    "width": 2048, 
    "height": 1536, 
    "animal_name": "eland",  
    "animal_count": 1, 
    "bboxes": {
      "0": [0.5068, 0.008463, 0.4931, 0.9388]}
  }, 
  "1": {
    "image_path": "S2/U10/U10_R2/S2_U10_R2_IMAG0661.JPG", 
    "width": 2048, 
    "height": 1536, 
    "animal_name": "eland", 
    "animal_count": 2,  
    "bboxes": {
      "0": [0.7353, 0.2858, 0.2646, 0.608], 
      "1": [0.3205, 0.0633, 0.3531, 0.8798]
    }
  }
}

Here's what my jsonl looks like:

"recordSet": [
    {
      "@type": "cr:RecordSet",
      "@id": "images_and_bbox",
      "name": "images_and_bbox",
      "field": [
        {
          "@type": "cr:Field",
          "@id": "images_and_bbox/image_path",
          "name": "image_path",
          "dataType": "sc:Text",
          "source": {
            "fileObject": {
              "@id": "train_annotations"
            },
            "extract": {
              "jsonPath": "$..image_path"
            }
          }
        },
        {
          "@type": "cr:Field",
          "@id": "images_and_bbox/image_bboxes",
          "name": "image_bboxes",
          "dataType": "cr:BoundingBox",
          "source": {
            "fileObject": {
              "@id": "train_annotations"
            },
            "extract": {
              "jsonPath": "$..bboxes.*"
            }
          }
        }
      ]
    }
  ]

The fact that image "0" has one bbox and image "1" has two is why I'm getting the error and I don't known if there's any fix. I initially tried to but all the bounding boxes into a list (i.e., a list of lists) since the order does matter but obviously the resulting list wasn't of data type BOUNDING_BOX.

Any idea how to overcome this?

[benjelloun]

Discussed in #673