Incorrect tokenization:

[sadeghjafari5528]

Spacy code to tokenize an english text.

import spacy

nlp = spacy.load('en_core_web_sm')

doc = nlp('Natural language processing(NLP)')
print([token.text for token in doc])

output of code is ['Natural', 'language', 'processing(NLP', ')'].

why spacy can not tokenize it correctly ?

[adrianeboyd]

The default English tokenizer settings don't split on ( as an infix, just as a prefix. You can modify the tokenizer settings for your task: https://spacy.io/usage/linguistic-features#tokenization

But be aware that the performance for a trained model like en_core_web_sm might degrade a bit if you modify its tokenizer settings because it wasn't trained on this exact tokenization.