Sharing a single embedding accross all components in a pipeline #10482
-
|
Hi, Requesting you to provide an example how to share a single TOK - TO - VEC / transformer accross multiple NER components in the training and prediction pipelines. I am not able to fine an example for this. Awaiting your reply. Bala |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
|
The double NER example project shows how to use two NER components at once and explains various details of that. It doesn't cover training different NER components at once. That's not possible in a straightforward way, since entity annotations go in You should be able to do this with the spancat if you use different span keys for the components. I am not sure if we have tried that before, but it shouldn't have issues. Also note that sharing tok2vec between components is the default for spaCy pipelines. You can confirm this is happening by checking that your component is using a Listener in the config. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you polm for your prompt reply. Thank you once again. Bala |
Beta Was this translation helpful? Give feedback.
The double NER example project shows how to use two NER components at once and explains various details of that.
It doesn't cover training different NER components at once. That's not possible in a straightforward way, since entity annotations go in
doc.entsthere's no way to have some annotations for one NER component and some for a different one. (I guess you could use a custom component to juggle annotations but I'm not sure if that would cause other issues.)You should be able to do this with the spancat if you use different span keys for the components. I am not sure if we have tried that before, but it shouldn't have issues.
Also note that sharing tok2vec between components is the d…