stats for ideal_answer in task B

[fulaibaowang]

Hi, it seems evaluation metices for ideal_answer in task B is missing?
In official page i notice there are stats like R-2 (Rec), R-2 (F1), R-SU4 (Rec), R-SU4 (F1).

cheers

[tasosnent]

Hi, it seems evaluation metices for ideal_answer in task B is missing? In official page i notice there are stats like R-2 (Rec), R-2 (F1), R-SU4 (Rec), R-SU4 (F1).

cheers

Hi,
The official evaluation for ideal_answers is based on manual scores. The automated ROUGE metrics, like R-2 (Rec), R-2 (F1), R-SU4 (Rec), R-SU4 (F1), are provided for completeness. These ROUGE metrics are calculated with ROUGE-1.5.5.pl
Thank you for your participation.