Transfer learning for Relation Extraction using Spacy v3

[harshpreetskychub]

Hi,
I am implementing Relation Extraction with the help of this video. I already have a working NER model which I trained on a corpus using spacy transformers. Output format is output/model-best I want to take the same NER model and re-train it, but now on Relation Extraction data. How can I achieve that? I am using the default cfg file for cpu and transformers which was there in the spacy project. It can be found here

[kadarakos]

There is a related discussion about continuing training: #8176. Let me know if you have further, more specific questions not answered in the thread!

[kadarakos]

This one https://github.com/explosion/projects/blob/v3/tutorials/rel_component/configs/rel_trf.cfg?

[harshpreetskychub]

Yeah, this one!
https://github.com/explosion/projects/blob/v3/tutorials/rel_component/configs/rel_trf.cfg

[kadarakos]

To make it clear for me a bit: when you train the REL component you would like the entities to be the ones predicted by your already trained ner component right?

[harshpreetskychub]

Yes,
I have a trained NER model and I want my new relation extraction model to use that NER component for the NER task and focus just on Relation Extraction task

[kadarakos]

Alright I think I more or less get it now.

A ner component needs to be added before the relation_extractor and sourced from output/model-best: https://spacy.io/usage/processing-pipelines#sourced-components.

The ner component needs to be added also to annotating_components: https://spacy.io/usage/training#annotating-components.
If I understand it correctly you should also freeze it, because you are not going to train it further: https://spacy.io/usage/training#config-components.

If you are freezing the ner component its important that the transformer does not get updated either. This is because training the relation-extractor will modify the transformer making it incompatible with the ner you trained earlier. You can set:

grad_factor = 0.0

to prevent the gradients being propagated into the transformer.