question about customized dataset

[go-ahead-maker]

Hi @Mountchicken, Sorry for bother you again

I'm trying to derive a customized dataset from your provide script -> add_candidate_by_grounding_dino.py. But I found that the best matched bbox between candidate_boxes and ans are still NOT match exactly

e.g., ans": [150.88, 264.89, 250.96, 407.53] | The best matched one is: "bbox": [149.92767333984375, 264.1410827636719, 249.41592407226562, 409.3323059082031]

But I noticed that your official provided refcocog dataset from https://hf-mirror.com/datasets/Mountchicken/RefCOCOg-RexThinker-20k seems 100% matched. So have you make some sort of post-process to make them aligning 100%?

Thank you

[Mountchicken]

Hi @go-ahead-maker
I think I replaced the original GT box in RefCOCOg with the best-matched box from Grounding DINO.

[go-ahead-maker]

Yes I think so, but It seems that the logic here: https://github.com/IDEA-Research/Rex-Thinker/blob/master/rexthinker/tools/dataset_tools/add_candidates_by_grounding_dino.py#L199

is that only best_match=None will add GT box into Candidate Box. So if best_box found, it will do nothing -> maybe caused a mismatch issue. But It can be fixed by some postprocess :)

[Mountchicken]

You are right. I think it's okay to use either the original GT box or the box detected by Grounding DINO.

[go-ahead-maker]

Also hope this great work can get a spotlight-level at NeurIPS ^_^