FastTracker-Benchmark-MOT

[Fleyderer]

1. Changed data storage path: trackeval/data instead of directly trackeval folder
2. Changed dets_n_embs storage path: now it uses dets_n_embs/<benchmark_name> to separate different benchmarks
3. Replaced boxmot.utils.dataloaders.MOT17.MOT17DetEmbDataset with boxmot.utils.dataloaders.dataset.MOTDataset. Almost nothing has changed, I just generalized and refactored it.
4. Made fixes to original FastTracker dataset (changed gt.txt files, added seqinfo.ini) and uploaded to HuggingFace
5. Added HuggingFace direct datasets downloading functionality
6. Changed logic of evaluation with --classes argument: for example, if --classes 0 is passed, only this class will be shown in final metrics.
7. Removed the ambiguity in the names: there were both opt and args with the same meaning, now only args.

[Fleyderer]

We also need to update runs.zip with moving currently existing files into dets_n_embs/MOT17-ablation folder. (And maybe remove unneeded __MACOSX folder 😄

I didn't want to upload without review

[mikel-brostrom]

Nice! So

runs/
├── dets_n_embs/
│   ├── <benchmark>/            
│   │   └── <yolo_model>/
│   │       ├── dets/<sequence>.txt
│   │       └── embs/<reid_model>/<sequence>.txt
...

?

[Fleyderer]

Exactly: benchmark, then model, then files.

[mikel-brostrom]

You can try the new structure with: https://github.com/mikel-brostrom/boxmot/releases/download/v16.0.11/runs.zip

Instead of

boxmot/boxmot/configs/datasets/MOT17-ablation.yaml

Line 3 in fb66088

runs_url: "https://github.com/mikel-brostrom/boxmot/releases/download/v12.0.7/runs.zip"

You could just commit this change as well 😄

[mikel-brostrom]

HuggingFace is a good move 🚀

[Fleyderer]

Downloaded runs.zip: there is still old structure, we need dets_n_embs/MOT17-ablation/...

[mikel-brostrom]

I see this when downloading the runs.zip from (https://github.com/mikel-brostrom/boxmot/releases/download/v16.0.11/runs.zip)

[Fleyderer]

Ah, sorry. Problem is not in folder structure: there should be yolox_x_MOT17_ablation instead of yolox_x because this is a detector trained on MOT17-half.

Re-upload please

[mikel-brostrom]

You are completely right, my bad. Fixed.

[Fleyderer]

Thanks, now everything is good, ready to merge

[mikel-brostrom]

We need to adjust the rest of the dataset sources under:

https://github.com/mikel-brostrom/boxmot/tree/master/boxmot/configs/datasets

to be:

boxmot/engine/trackeval/data/...

[mikel-brostrom]

Nice dataset addition @Fleyderer! Next some sports dataset? For trying some global tracklet associations algorithms?

[Fleyderer]

Nice dataset addition @Fleyderer! Next some sports dataset? For trying some global tracklet associations algorithms?

Haha, I don't have any plans for now (at least it is easier now)

I need to work on this benchmark first; it might even become a blog post material. My next task here is Kalman and hyperparameter tuning – perhaps using existing detections and a more mathematically accurate approach, or perhaps in conjunction with parameter tuning using Ray.

[mikel-brostrom]

I think I might integrate either SoccerNet or SportsMOT