Handling long text

[nishchay47b]

How is spacy able to handle long text with BERT like models. BERT like models have a limitation of 512 tokens, I understand there is window operation going, but with windows and strides running over the text you will get multiple prediction for the same tokens, how is that handled then?

[polm]

This is covered at the top of the documentation for Transformers - the sum of embeddings is used for tokens. Please see the docs for more detailed explanation, including how to change the way it works.

https://spacy.io/api/transformer