evaluation

[betterze]

Dear PnPInversion authors,

Thank you for sharing this great work.

I am trying to use PIE-Bench for evaluation. There are image masks in the dataset; are we allowed to use the masks for image editing? Or the masks are only used in evaluation?

The evaluation_result.csv I obtained a lot of NaN (in psnr, lpips, mse and ssim of the unedit part), is this expected? How to get the final scores? By averaging over columns?

Thank you for your help.

Best Wishes,
Zongze

[Ed-ivan]

in evaluate.y line 41:

if metric=="psnr_unedit_part":
if (1-src_mask).sum()==0 or (1-tgt_mask).sum()==0:
return "nan"

So , I think the item which is nan is can be ignored. If you have some opinion , you can share it .