Additional training of pre-trained model does not work on BILUO

[xim0g]

Hello!

I am trying to update Russian ru_core_news_md model, adding recognition of date and time. I use internal API for training.

It successfully learns to predict time (like 23:59) as well as dates (like "tomorrow"), which are marked with U- (U-TIME and U-DATE)

However, entities with B- and L- tags (like "24 May" date) are not recognised after training (even those that were used).

For instance, it was trained at ['15', 'июль', 'поездка', 'в', 'питер'] (15 July trip to Saint-Petersburg) annotated as ['B-DATE', 'L-DATE', 'O', 'O', 'U-LOC'] and it does not recognise the date in that particular case (as well as on unseen samples).

Could you please help me understand how to solve this issue? I would appreciate your help very much.

I use "example training loop" from here: https://spacy.io/usage/training#api

[polm]

Can you clarify how you're converting your training data? spaCy can convert BILOU data but internally it only stores BIO tags.

Also note that while you can train a model for this, for dates like "24 May" rule-based matching should be sufficient unless you need to catch imaginary months or something.