PSNR on TransLab

[Zero-4869]

Hi author, Thanks for your work. I reproduced your results on TransLab, but the PSNR on the test split is much lower than the one published in your paper. Is the PSNR in your paper on train split or on test split or without split? I used your script without any modifications.
The PSNR I got for scene01 to 08 are: 39.41 35.17 38.34 36.69 34.41 35.48 42.67 36.69 on test split

[longxiang-ai]

Hi author, Thanks for your work. I reproduced your results on TransLab, but the PSNR on the test split is much lower than the one published in your paper. Is the PSNR in your paper on train split or on test split or without split? I used your script without any modifications. The PSNR I got for scene01 to 08 are: 39.41 35.17 38.34 36.69 34.41 35.48 42.67 36.69 on test split

On test split (like all other baselines, each 8 image 1 test). Which device are you running on? It might be a little difference on difference devices, but it should not be so much, could you please tell me more details?

[Zero-4869]

Thanks for your quick reply. I re-clone and re-run just now but it doesn't change much. I am running it on A100, using python 3.8.20, torch 2.4.1 and CUDA 12.1.1(different from your implementation but you mentioned that other version should also work). The PSNR for test split is stored in results/test/result_30000.json. The dataset is TransLab
Actually the result in the train split is quite close to what you published so I posed the question.

[longxiang-ai]

Thanks for your quick reply. I re-clone and re-run just now but it doesn't change much. I am running it on A100, using python 3.8.20, torch 2.4.1 and CUDA 12.1.1(different from your implementation but you mentioned that other version should also work). The PSNR for test split is stored in results/test/result_30000.json. The dataset is TransLab Actually the result in the train split is quite close to what you published so I posed the question.

Thank you for the follow-up. I've double-checked my original data and confirmed that I'm using the correct test split as provided. And the results of training split are about 0.8 db even higher than the results reported in the paper. I suspect this discrepancy could be due to one of the following: 1. Device or environmental differences (e.g., different GPU types or CUDA versions). 2. A minor inconsistency of hyper-parameters between the released code and the exact training code used for the paper results, which might have been introduced during the code cleanup and organization process.

I will investigate this further on my end. Any insights from your side would be greatly appreciated.

[Zero-4869]

I found that in your script, you disable the 2dgs_normal_loss and you disable the depth_normal_detach loss, I think that's the problem. Could you check in your code whether you use them or not?

[longxiang-ai]

Thanks a lot, I'll check it later