Unrealistic perplexity #17
Description
I'm trying to evaluate 5-gram model on a Vietnamese corpus but the perplexity
doesn't seem to be right...
What steps will reproduce the problem?
1. Download and extract problem.zip
2. Follow the README file
What is the expected output? What do you see instead?
The result from BerkeleyLM and SRILM should be comparable but in fact
BerkeleyLM return an unrealistic perplexity of around 1.
What version of the product are you using? On what operating system?
1.1.5 on Ubuntu.
Please provide any additional information below.
Original issue reported on code.google.com by ngocminh...@gmail.com on 12 Feb 2014 at 3:27
Attachments: