Spacy's Sentencizer is Giving Different Sentence boundaries

[shamilsaeed]

Hi All,

I am getting an issue with Spacy Sentencizer giving different sentence boundaries on specific sentences.
Current Spacy version: 3.0.0
Current Spacy model: en_core_web_sm (3.0.0)

It works fine on jupyter but gets messed up on docker container.

The problem is happening right here. Given

text = "Not all all the way done, but let's see here."

doc = nlp(text)

for sentence in doc.sents:
    print(sentence)

This prints:

Not all all the way done
,
but let's see here.

Not sure why it splits at comma (,), but does not happen in my jupyter.

Does anyone know why this may be happening?

[polm]

Can you provide the output of spacy info in Jupyter and in your Docker environment? With the same spaCy version I would expect those to give the same results, and for the sentence tokenization results to be the same.

Note that you appear to be using the parser for sentence boundaries and not the sentencizer, which is a separate component.

Also, if you don't have a particular reason for using 3.0.0, I would recommend upgrading, as there have been several bugfixes since then., though I can't think of any that would cause a difference between environments like this.