Error when using README example for HOTA when using multiple files

[mikel-brostrom]

I am trying to run the following:

import os
import numpy as np
import motmetrics as mm
import pandas as pd

def compute_motchallenge(gt_files, test_files):
    
    accs = []
    for gt_file, test_file in zip(gt_files, test_files):

        df_gt = mm.io.loadtxt(gt_file)
        df_test = mm.io.loadtxt(test_file)

        # HOTA requires different thresholds for matching
        th_list = np.arange(0.05, 0.99, 0.05)
        # Compare ground truth and test data
        accs.append(mm.utils.compare_to_groundtruth_reweighting(df_gt, df_test, "iou", distth=th_list))

    return accs


gt_files = ['/Users/mikel.brostrom/boxmot/assets/MOT17-mini/train/MOT17-04-FRCNN/gt/gt.txt',
            '/Users/mikel.brostrom/boxmot/assets/MOT17-mini/train/MOT17-02-FRCNN/gt/gt.txt']
test_files = ['/Users/mikel.brostrom/boxmot/runs/mot/yolov8n_osnet_x0_25_msmt17_botsort/MOT17-04-FRCNN.txt',
              '/Users/mikel.brostrom/boxmot/runs/mot/yolov8n_osnet_x0_25_msmt17_botsort/MOT17-02-FRCNN.txt']

accs = compute_motchallenge(gt_files, test_files)
mh = mm.metrics.create()

summary = mh.compute_many(
    accs,

    metrics=[
        "deta_alpha",
        "assa_alpha",
        "hota_alpha",
    ],
    generate_overall=True,  # `Overall` is the average we need only
)
strsummary = mm.io.render_summary(
    summary,  # Use list to preserve `DataFrame` type
    formatters=mh.formatters,
    namemap={"hota_alpha": "HOTA", "assa_alpha": "ASSA", "deta_alpha": "DETA"},
)
print(strsummary)

But I get the following error:

  File "/Users/mikel.brostrom/Library/Caches/pypoetry/virtualenvs/boxmot-YDNZdsaB-py3.11/lib/python3.11/site-packages/motmetrics/metrics.py", line 681, in events_to_df_map
    raw = df[df.Type == "RAW"]
             ^^^^^^^
AttributeError: 'list' object has no attribute 'Type'

When flattening the 2D list obtained from compare_to_groundtruth_reweighting , by:

def compute_motchallenge(gt_files, test_files):
    
    accs = []
    for gt_file, test_file in zip(gt_files, test_files):

        df_gt = mm.io.loadtxt(gt_file)
        df_test = mm.io.loadtxt(test_file)

        # HOTA requires different thresholds for matching
        th_list = np.arange(0.05, 0.99, 0.05)
        # Compare ground truth and test data
        accs.append(mm.utils.compare_to_groundtruth_reweighting(df_gt, df_test, "iou", distth=th_list))
        #accs.append(mm.utils.compare_to_groundtruth(df_gt, df_test, "iou", distth=0.5))
    
    # flatten as compare_to_groundtruth_reweighting generates a 2D list
    print(len(accs))
    print(len(accs[0]))
    if all(isinstance(sublist, list) for sublist in accs):
        accs = [j for sub in accs for j in sub]
    print(len(accs))
    return accs

I get:

(boxmot-py3.11) ➜  boxmot git:(pymotmetrics) ✗ python3 try_pymotmetrics.py
2
19
38
         DETA  ASSA  HOTA
0       10.7% 83.4% 29.9%
1       10.7% 83.4% 29.9%
2       10.7% 83.4% 29.9%
3       10.7% 83.4% 29.9%
4       10.7% 83.4% 29.9%
5       10.7% 83.4% 29.9%
6       10.7% 83.4% 29.9%
7       10.7% 83.4% 29.9%
8        9.8% 83.0% 28.5%
9        9.8% 83.0% 28.5%
10       9.8% 83.0% 28.5%
11       9.4% 80.1% 27.4%
12       8.8% 82.6% 27.0%
13       7.9% 81.6% 25.3%
14       7.6% 81.9% 25.0%
15       7.3% 83.2% 24.7%
16       5.3% 77.2% 20.2%
17       2.6% 75.3% 13.9%
18       1.3% 46.1%  7.7%
19       9.6% 75.0% 26.9%
20       9.6% 75.0% 26.9%
21       9.6% 75.0% 26.9%
22       9.6% 75.0% 26.9%
23       9.6% 75.0% 26.9%
24       9.6% 75.0% 26.9%
25       9.6% 75.0% 26.9%
26       9.6% 75.0% 26.9%
27       9.6% 75.0% 26.9%
28       8.9% 70.0% 25.0%
29       7.5% 75.0% 23.8%
30       7.5% 75.0% 23.8%
31       7.5% 75.0% 23.8%
32       7.5% 75.0% 23.8%
33       6.2% 69.2% 20.7%
34       5.6% 75.0% 20.4%
35       5.6% 75.0% 20.4%
36       4.3% 66.7% 16.9%
37       0.0%  0.0%  0.0%
OVERALL  8.2% 75.1% 24.6%

How should we use HOTA when working with multiple files @Justin900429 ? Are we supposed to run mh.compute_many for each sequence as accumulating with HOTA does not seem to work?

[mikel-brostrom]

I just saw this comment in the PR @Justin900429 :

1. Users should create a list of accumulator and compute the average manually.

I suppose this implies that mh.compute_many should be called for each sequence