Skip to content

Commit 4554b47

Browse files
committed
Update a comment explaining our poor scores on this dataset
1 parent 8c92951 commit 4554b47

File tree

1 file changed

+5
-4
lines changed

1 file changed

+5
-4
lines changed

stanza/utils/datasets/coref/convert_hebrew_iahlt.py

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,4 @@
1-
"""
2-
Convert the coref annotation of IAHLT to the Stanza coref format
1+
"""Convert the coref annotation of IAHLT to the Stanza coref format
32
43
This dataset is available at
54
@@ -16,8 +15,10 @@
1615
1716
python3 stanza/utils/datasets/coref/convert_hebrew_iahlt.py
1817
19-
TODO: the scores from this model are horrible, only 30 F1.
20-
Need to either verify the usage elsewhere or double check the outputs of the conversion
18+
The scores for models built from the dataset are pretty lousy in
19+
general, but seem to be in line with the scores obtained by other
20+
people working on this data. For example, the authors said they had a
21+
52 F1, whereas if we use roberta-xlm, we get 50.
2122
"""
2223

2324
from collections import defaultdict, namedtuple

0 commit comments

Comments
 (0)