Preserve whitespace in sentence segmentation

[saraswat40]

How to reproduce the behaviour

nlp = spacy.load('en_core_web_lg')
s = "Cool. Glad I was helpful"
p_chunk = nlp(s)
for s in p_chunk.sents:
	print('|' + s.text + '|')

Output:

|Cool.|
|Glad I was helpful|

Notice that the space between two sentences is missing.

if s = "Good morning. How are you? I'm fine, thank you."

Output:

|Good morning.|
| How are you?|
| I'm fine, thank you.|

Notice that one whitespace between the first and second sentence is preserved but the two whitespaces between 2nd and 3rd sentences is converted to one whitespace.

Is there a way to consistently preserve the whitespace?

My understanding from #1707 is that the whitespace will be preserved. But this is not happening.

Your Environment

• Operating System: Ubuntu 21.04
• Python Version Used: 3.8
• spaCy Version Used: 3.2.3
• Environment Information:

[adrianeboyd]

If you also want to get the trailing space for sentence spans, use sent.text_with_ws instead of sent.text. See the Span attributes here: https://spacy.io/api/span#attributes

It looks like the only change in spaCy since Ines's answer in the linked issue is that it's text_with_ws and not text_with_ws_.

[saraswat40]

Hello,

Thanks for responding. This issue is now resolved.