Is there any Multi-task example? #9677
-
|
I wish to train an NLP model with two heads, for similar tasks. So it will embedd the text once, and then train two MLP heads for single-label prediction. Is there any example in SpaCy on how to do this? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
Multi-task learning is the default in spaCy pipelines, see the docs on shared embeddings. It sounds like your pipeline would be a tok2vec (or transformer) with two textcat components, but I need to look at how that works - the issue is that textcat components use the What you can do out of the box is train a multilabel textcat with non-exclusive categories. If you label your data consistently the classifier shouldn't violate your constraints even if it's technically possible; you can try that to get started with the spaCy training process at least. |
Beta Was this translation helpful? Give feedback.
Multi-task learning is the default in spaCy pipelines, see the docs on shared embeddings.
It sounds like your pipeline would be a tok2vec (or transformer) with two textcat components, but I need to look at how that works - the issue is that textcat components use the
.catsattribute, which is unique, so there's no place to separate your annotations.What you can do out of the box is train a multilabel textcat with non-exclusive categories. If you label your data consistently the classifier shouldn't violate your constraints even if it's technically possible; you can try that to get started with the spaCy training process at least.