How to reproduce the results in the paper?

[Candy-Crusher]

Hi! May I ask how to reproduce the results shown inthe paper?

For example, on davis-17, follow the instructions
"cd data; python download_prepare_davis.py; cd ..
CUDA_VISIBLE_DEVICES=4,5,6,7 torchrun --nproc_per_node=4 --master_port=29604 launch.py
--mode=eval_pose --n_iter 0
--pretrained="checkpoints/MonST3R_PO-TA-S-W_ViTLarge_BaseDecoder_512_dpt.pth"
--eval_dataset=davis --output_dir="results/davis/easi3r_monst3r_sam"
--use_atten_mask --sam2_mask_refine
python mask_metric.py --results_path results/davis/easi3r_monst3r_sam"

The results I got is:

While the results shown in the paper is:

[Candy-Crusher]

And for DyCheck:

[rover-xingyu]

Hi @Candy-Crusher, the results for the DAVIS dataset seem to correspond to the DAVIS-all evaluation. You can check here to confirm if you are evaluating the 2016/2017/all subsets. If it's for all, JM and JR results look quite good, as they outperform our testing results.

As for the DyCheck results, it appears to be based on a subset. Please evaluate the complete dataset and apply a multiplier of 10 to both Accuracy and Completeness scores.