Dataset size of 0

[alexeidinca-ro]

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Question

Hello! Hope you are well!

I am trying to run a Supervision Benchmark (confusion matrix) on a custom dataset and everytime it gives me the dataset size of zero and a confusion matrix of zero. What am i doing wrong in this approach? The data set is a test one, the format is a YOLO one with images and annotations in txt format. The YAML file is the classic YOLO one.

I am attaching the code and content of the YAML file.

data.yaml below:

train: ../ita-my-patch/train/images
val: ../ita-my-patch/valid/images
test: ../ita-my-patch/test/images

nc: 1
names: ["HH"]

Notebook code below:

import supervision as sv
from ultralytics import YOLO
import numpy as np 

dataset = sv.DetectionDataset.from_yolo(
    images_directory_path='ita-my-patch/test/images',
    annotations_directory_path='ita-my-patch/test/labels',
    data_yaml_path='data.yaml'
)

model = YOLO("runs/detect/train27/weights/best.pt")

print('dataset classes:', dataset.classes)
print('dataset size:', len(dataset))

def callback(image: np.ndarray) -> sv.Detections:
    result = model(image)[0]
    return sv.Detections.from_ultralytics(result)


confusion_matrix = sv.ConfusionMatrix.benchmark(
   dataset = dataset,
   callback = callback
)

confusion_matrix.matrix

Additional

No response

[SkalskiP]

Hi @alexeidinca-ro! 👋🏻 Let me convert that question into the discussion and put it into the Q&A section.

[SkalskiP]

@alexeidinca-ro

1. What was the output of those logs?

print('dataset classes:', dataset.classes)
print('dataset size:', len(dataset))

2. Are you sure there are files in 'ita-my-patch/test/images' and 'ita-my-patch/test/labels'?

[alexeidinca-ro]

Hello @SkalskiP
The output is:

dataset classes: ['HH']
dataset size: 0

There are like 2500 files in each folder (images and annotations).

[SkalskiP]

Are you running this in Google Colab or locally?

[alexeidinca-ro]

I am running this locally.

[SkalskiP]

Any chance we could work on Colab to figure out what's wrong?

[alexeidinca-ro]

The dataset is private and I can't make it public by uploading the data to a Collab. I tried changing the dataset to a smaller one (some 80-90 images in total) and the above code works fine. From the first observations, it seems that the large number of test images seems to be the problem.