What is the suggested amount of training per entity?

[Atomic-man007]

Hi Team,
actually i wanted to know how much training data i need to accurately train the Spacy model
RasaHQ/rasa#164 in this discussion he says [ For the spacy backend a suggested amount of training data per entity is around 5000 samples ] is it true, if i have 150 labels then multiply by 5000 then its 750000 in total
it would be impossible to use Spacy the right !
can anyone clarify this please please

[koaning]

"It depends".

If you can share more about the task that you're trying to tackle then I might be able to guide you more, but the number of examples that you'll "need" really depends on the task and the definition of success.

I'd also want to point out that you might not need machine learning. In this series on YouTube I use spaCy to detect programming languages in text. I was able to get very reasonable results with 1000 examples, but I was also able to get reasonable results with 0 labels by just implementing some rules using rule-based matching.

[koaning]

It should also be said that the quoted "5000 examples" is out of date. The original source seems to be here and it's about spaCy v1.x. The same comment also mentions:

Things will improve with spaCy 2.0 a little bit. You might be able to get a useful model with as little as 500 or 1,000 sentences annotated with a new NER class. Maybe.