Class names are not the same as the class which the detector is trained on

[Abhishek0075]

Search before asking

• I have searched the BoxMOT issues and found no similar bug report.

Question

I trained my yolov8 on the custom dataset of single class "drone" and then I am tracking the drones using deepocsort. The results are bad and the class name which it is showing in the eval result is person.
The command used is :
boxmot eval --yolo-model /Data3/Abhishek/IIT_Drone/yoloe_runs/yoloe3/weights/yolov8_best.pt --tracking-method deepocsort --save --source /Data3/Abhishek/IIT_Drone/test

This is the result after the evaluation : 
INFO     | ==========================================================================================
INFO     | 📊 Results Summary
INFO     | ==========================================================================================
INFO     | Class           HOTA     MOTA     IDF1     AssA     AssRe    IDSW     IDs
INFO     | ------------------------------------------------------------------------------------------
INFO     | person          12.77%   29.30%   13.79%   5.59%    5.63%    18       13
INFO     | ==========================================================================================

I had a warning saying to use the config so set it config as this :

benchmark:
  source: "/Data3/Abhishek/IIT_Drone"
  split: "test"
  eval_classes:
    0: drone

Also I need to ask is it really looking for person or it is just the class name different but the tracking is really for the drone in the test set

[mikel-brostrom]

Once you create the config under boxmot/configs/datasets. For example boxmot/configs/datasets/IIT_Drone you can instead run:

boxmot eval --yolo-model /Data3/Abhishek/IIT_Drone/yoloe_runs/yoloe3/weights/yolov8_best.pt --tracking-method deepocsort --save --source IIT_Drone

This will load the classes under that config file. In your run, benchmark is inferred from parent folder name (IIT_Drone), so your custom YAML is ignored unless it exists as a dataset config file in the repo with that name. Making the classes fall back to the default, which is person

boxmot/boxmot/engine/evaluator.py

Lines 647 to 656 in fb66088

	classes_to_eval = ["person"]
	class_ids = [1]
	if isinstance(eval_classes_cfg, dict) and len(eval_classes_cfg) > 0:
	ordered = sorted(((int(k), v) for k, v in eval_classes_cfg.items()), key=lambda kv: kv[0])
	class_ids = [k for k, _ in ordered]
	classes_to_eval = [v for _, v in ordered]
	elif hasattr(args, "classes") and args.classes is not None:
	class_indices = args.classes if isinstance(args.classes, list) else [args.classes]
	classes_to_eval = [COCO_CLASSES[int(i)] for i in class_indices]
	class_ids = [int(i) + 1 for i in class_indices]

[mikel-brostrom]

The bad results are more easily understood by the video(s) you are evaluating on. If you could upload them here that would help

[Abhishek0075]

What is the range of each metrics in the evaluation? Is it from [0,100]

[Abhishek0075]

Also why is the tracking : 550 frames being shown even though there is 2334 frames in the test folder with one video called IIT_video_01. The detections are done properly for 2334 frames.

INFO     | ReID embedding dims per model: n/a (no detections)
INFO     | YOLO batch frames=1 | dets/frame=[0] | total_dets=0
INFO     | ReID embedding dims per model: n/a (no detections)
Batched YOLO+ReID (yolov8_best.pt, bs=16): 100%|███████████████████████████████████████████████████████████████████████| 2334/2334 [00:01<00:00, 2158.24frame/s]
ReID embeddings: 0det [00:01, ?det/s]
INFO     | YOLO batch frames=1 | dets/frame=[0] | total_dets=0
INFO     | ReID embedding dims per model: n/a (no detections)
INFO     | [3/4] Running tracker...
Frames IIT_video_01: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████| 550/550 [00:24<00:00, 22.76it/s]
INFO     | Tracking: 550 frames, total: 6554.0ms, avg: 11.92ms/frame
INFO     | [4/4] Evaluating results...

[mikel-brostrom]

The evaluation is comprised of 3 sequential and independent steps:

1. Generation (detections and embeddings for all sequences)

It works in a round-robin fashion. A frame from each sequences is moved into the batch till it is full, so all the frames from all sequences are taken into consideration for the generation phase. This is possible as the detections and ReID embeddings generations are completely independent of the tracking phase.

2. Tracking (mot results generation for each sequence)

Each sequence is run separately by feeding its associated detections and embeddings to the tracker to be evaluated.

3. Evaluation by using trackeval

The MOT results from 2. are fed to trackeval for metrics generation

[Abhishek0075]

I am only having 1 sequence (i.e video01) in the test split.
So what I was asking is as you can see the below the tqdm bar at [3/4] Running tracker..... section., it is shown that
INFO | Tracking: 550 frames, total: 6554.0ms, avg: 11.92ms/frame

There is 2334 frames in the video. In previous tqdm you can see it shows 2334 but not in tracking part.
What does the tracking 550 frames means ? It should be running track on the whole frames right

So sorry if I am wrong, I am learning, so just clarifying.

[Abhishek0075]

When I ran another test on same dataset but with different YOLOv8 it shows :

| [3/4] Running tracker...
Frames IIT_video_01: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████| 2184/2184 [01:44<00:00, 20.80it/s]
INFO     | Tracking: 2184 frames, total: 27728.8ms, avg: 12.70ms/frame

Here it is running tracking on 2184 frames. I think it is only doing the tracking on the frames that is mentioned there because the time taken i showing drastic difference (167.8 vs 13615.4, for 550 vs 2184)

[github-actions]

👋 Hello, this issue has been automatically marked as stale because it has not had recent activity. Please note it will be closed if no further activity occurs.
Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!