Coreference with incompletely annotated data?

[KennethEnevoldsen]

So want to perform coreference resolution with an incomplete dataset where only some of the documents are annotated.

I know that for some components (e.g. NER), it is possible to annotate using an incomplete dataset. Is that also the case for the coreference component?

If it is how does the model differentiate between "not annotated" vs. "annotated but no clusters"?

[adrianeboyd]

So this isn't true for coref (yet, we should take a look at this), but in general all components should be able to support training from misaligned tokenization. The get_loss methods should basically ignore instances where the tokens can't be aligned. (But be aware that the alignment code is full of special cases depending on the attribute to try to keep as much annotation as possible, like only the token start char matters for SENT_START and it will align AB/TAG1 to A/TAG2 B/TAG2 as long as the tags are the same for the whole sequence.)

Not all components can train from partial annotation, though, mainly because there's not always a way to mark partial annotation, like for spancat training from annotation on doc.spans there's no way to mark unannotated sections of a text. I think this will be similar for coref, at least initially.

Also be aware that we've tried to train transformer components with partial annotation as in en_core_web_trf and it just doesn't work well and we're not sure what's going on. See the discussion (well, it's just me) starting here: #7493 (comment)

[KennethEnevoldsen]

Thanks exactly the info I was looking for. Thanks! I will then start out with training the coref component using frozen weights.

Will try some experiments with the partial annotation as well (for NER+POS without dep.). Will add to the thread if I find something which is promising.