Document Classfication

[KMayank29]

Hi spaCy team,

I want to classify text on the document level. I have news articles and the task is to classify them into three mutually exclusive categories- positive, negative and neutral. It is not wise to classify each sentence of the article and aggregate the scores to categorize the article. Also feeding the whole article into model is not possible due to limitation of the input size (512/1024).

I, therefore, request you to guide me or point the reference for the mentioned task.
One more query if I could use Prodigy to annotate the data on document level.

Thanks

[polm]

Sounds like you should just be able to use a textcat model. Take a look at the example projects for reference.

Also feeding the whole article into model is not possible due to limitation of the input size (512/1024).

Not entirely sure what you're referring to here, but spaCy can handle automatically splitting and merging documents to get a useful representation. There are issues with very long documents, like whole books, but newspaper articles should be fine.

Prodigy also supports annotations like this, it's a basic annotation mode.

[KMayank29]

Thank you for the reply @polm