reproduce the results of the paper

[jia-975]

Hi, I want to know how to reproduce the results of the paper. I downloaded the branch pretrain and checkpoints, and then I ran the generation without any modification. However, the results I got are different from the paper. Here are my results.

In addition, I would like to ask you how many seeds did you use for averaging, and what are they? Thank you.

[MinkaiXu]

Hi Jia,

I didn't clearly remember the seeds, but even the same seed would have different results on different machines. And I feel like your reproduced number is roughly the same already?

[RylanSchaeffer]

@jia-975 as a random passerby using this repo, I think those numbers (Geodiff vs paper) are basically the same. I've seen other projects with much higher variance