Looking for a Multi-task example of two multi label text classifiers

[timpal0l]

I wish to train a shared encoder that performs text classification with two outputs. Basically a nn with two heads, and each head is a multi-label classifier.

My dataframe:

text, category_a, category_b
"Today the weather is great, but the economy is not!", "great_weather", "bad_economy"

So I dont wish this to be a multi-label classifier since the two categories can be very different, I would like it to be a multi-task network. I cannot find any python examples on the docs on how to this with two multi-label text classifiers, but it states that spacy uses multi-task learning.

Looking for a code example of how to use two textcat_multilabel in the same training loop over the same dataset.

[polm]

We don't have an example for this. spaCy uses multi-task learning, but having multiple classifier heads is a case we hadn't considered. We usually use the multi-task functionality for things like tagger+parser.

That said, it should work fine if you add two textcat components and explicitly specify their labels (rather than letting them be inferred from the data). If you try that and have issues let us know.

Going to link your previous question, since it's closely related: #9677

[info2000]

Would be possible train the 2 task standalone, and next add the task2 into the task1 pipeline?
There's an example on prodi.gy forum (sorry I don't found it now)

[adrianeboyd]

Yes, you can have two textcat components in the same pipeline. If they write to the same category label in doc.cats, the second component will clobber the values from the first component, and the scores in doc.cats in the end won't add up to 1, but each textcat component just writes to its own labels about in doc.cats and doesn't modify other keys.