score_weights for individual labels in SpanCategorizer training

[phlobo]

As far as I understand, the score_spans method in the Scorer returns P/R/F1 for individual entities as well as a dict.

Is it possible to assign weights to theses individual scores through the score_weights section of the training config?
If so, what is the syntax?

I want to assign higher weight to a rare class, where otherwise the model produces 0 recall.

[adrianeboyd]

No, you can only assign weights to the top-level (float) entries in the scores dict. Even if you could do this, it would only affect the eval step (primarily used for early stopping) and wouldn't influence the actual model training at all.

If you have a rare class, you can try other options like augmenting your training data or experimenting with having the rare class in its own spancat component with a lower threshold (obviously the whole pipeline is slower overall, but if recall is important the tradeoff might be worth it for your task).

[phlobo]

Thanks for the clarification, also regarding the role of score_weights during training. I ended up creating a custom scorer that unwraps the dict for each label, so I can at least monitor the performance for these classes during training.