Problem in retraining spacy for sentence splitting

[nikhilajoshy]

I created training data using

for _, rw in tqdm(data.iterrows()): 
    doc = nlp.make_doc(rw.text)
    for token_id, token in enumerate(nlp(rw.text)):
        if token.idx in rw['starts']:
            doc[token_id].is_sent_start = True

        else:
            doc[token_id].is_sent_start = False
    for sent in doc.sents:
        print("--", sent, "---")
    db.add(doc)

db.to_disk("./train.spacy")

But when I debug config file, I am getting this

why is training data split as one word per sentence??

[adrianeboyd]

For data where you have set sentence boundaries with token.is_sent_start, you want to train a senter component instead of a parser component.

(The parser uses dependency trees to identify sentence boundaries, so in data without dependency annotation, each word looks like its own separate tree and own separate sentence.)