Some relabeled data not blacklisted

[HzEdward]

I have noticed that some of the mislabeled frames in the data/data.csv file are represented as "2" instead of "1" for the blacklist column, which prevents these frames from being excluded from the blacklist when the re-labeling parameter is "False". Could I know what is the reason for this?

[TheoPis]

Thanks for noting this.
blacklist = 2 in data.csv corresponds to originally problematic but relabelled frames.
As you mention, when you set blacklist=True and use_relabelled=False (in config) the relabelled frames are kept in the dataset. This is propably not the ideal behaviour for those flags. The reason this was never an issue for us, is that whenever we set blacklist=True we also set use_relabelled=True. Setting both flags to true gives us the filtered and relabelled version of the dataset, which is used for most of the experiments of the paper. Setting both flags to false gives you the originally released CADIS dataset.

[HzEdward]

Also, in addition to the mislabeled frames you commented on, I still have found a number of mislabeled frames (but not commented in the csv file), especially for instruments. May I know what technique you used to identify the mislabeled data?

[TheoPis]

It was done by simply looking at frames with labels overlayed. It is entirely possible we did not find all of the mislabeled frames.

[HzEdward]

Thanks for noting this. blacklist = 2 in data.csv corresponds to originally problematic but relabelled frames. As you mention, when you set blacklist=True and use_relabelled=False (in config) the relabelled frames are kept in the dataset. This is propably not the ideal behaviour for those flags. The reason this was never an issue for us, is that whenever we set blacklist=True we also set use_relabelled=True. Setting both flags to true gives us the filtered and relabelled version of the dataset, which is used for most of the experiments of the paper. Setting both flags to false gives you the originally released CADIS dataset.

Hi, I think I'm still a bit confused because I've noticed that some of the relabeled data is annotated as blacklisted = 2, while others are still annotated as blacklisted = 1. Could you explain the reason behind this?