I can't get the same results as the paper.

[lrt366]

I ran the codes which In the path（\LeakGAN-master\Image COCO\Main.py）（LeakGAN-master\Image COCO\eval_bleu.py）. but I can't get the same results as the paper. I just changed the python's version(3.6) and tensorflow-gpu 's version(1.8.0).
this is the result i got :
BLEU2: 0.902
BLEU3: 0.7837
BLEU4 :0.6206
BLEU5: 0.4208
Please tell me how I can modify it.thanks a lot！

[lianyixin]

I got the same result with you,
BLEU2: 0.9027
BLEU3: 0.7619
BLEU4: 0.5890

Hope the author can give us a clear explanation.

[lianyixin]

Besides, it's worth pointing out that the bleu score in the adversarial training just bounces around the final value of pre-train stage, meaning that the adversarial training has no help for enhancing the performance of the generator.

[CR-Gjx]

Thanks for your remind, the decline of results may be caused by the update of NLTK nltk/nltk#1844. The latest NLTK version updates the algorithm of BLEU, which may cause the decline of BLEU.
I will rerun the experiment and discuss with you.

[lianyixin]

Thanks for your remind, the decline of results may be caused by the update of NLTK nltk/nltk#1844. The latest NLTK version updates the algorithm of BLEU, which may cause the decline of BLEU.
I will rerun the experiment and discuss with you.

Thanks for your quick response. It's very helpful. Looking for your further messages.

[bearcatt]

Hi @CR-Gjx , in Table 3 of your paper, how do you obtain the results of SeqGAN and RankGAN? I can't find any result about image captioning in the paper of SeqGAN. I do find some results about image captioning in RankGAN, but their results are not consistent with those reported in Table 3. Do you reimplement SeqGAN and RankGAN to get those results? If so, what's the detailed architecture of these image captioning models? If not, do the results of SeqGAN and RankGAN also suffer from the BUG in NLTK? I would be really appreciated if you can answer my questions :)

[CR-Gjx]

Hi, @bearcatt , the experiments implemented in our paper is open domain text generation rather than conditional text generation (image caption, translation...). We just use the texts in Coco dataset as open domain datasets.
In addition, the update of NLTK also causes the decline of BLEU.