Inspec dataset evaluation

[laurita]

Hey,
I see in the code that for evaluating INSPEC dataset, only uncontrolled keyphrases are used, but in this comment it says ".contr and .uncontr". What is the source of truth for achieving 0.422 F1@10 score that is mentioned in the Experimental Results section of the paper Bidirectional LSTM Recurrent Neural Network for Keyphrase Extraction?
Thanks in advance!
Laura

[basaldella]

Hello,

that paper is based on a different codebase. Anyway, the comment is a mistake; as you can see a few lines below, only .uncontr files are loaded.

Marco

[laurita]

Thanks for a quick reply! Could you give some more info about how the 0.422 was achieved? It seems quite a high number compared to other state-of-the art methods. Also, the paper that improves BLSTM by adding attention mechanism reports lower numbers, namely 0.394 F1@10. Is there a difference how these numbers are computed?