Multiple roots in a dependency tree issue

[kanayer]

I have a Thai dataset consisting of 13 000 sentences and I have trained the POS tagger & parser pipeline using it. Sometimes the parser would split the original sentence into two at the inference step, and I was wondering if there is a way to force the parser to give a dependency tree with exactly one root?

The documentation mentions Token.is_sent_start that saves sentence boundaries and I was wondering if adding a block of code like below will help?

sentence = "เอา ตัว นี้ แทน ได้ ค่ะ 4 ขวด"
nlp = spacy.load('th_custom_model")

def change_sentence_boundary(doc):
    doc[0].is_sent_start = True
    for token in doc[1:]: 
        token.is_sent_start = False
    return doc

If yes, would there be any side effects from enforcing sentence boundaries like this?

[adrianeboyd]

Yes, you can add a custom component if you want to add sentence boundaries so that the parser treats each doc as a single sentence: https://spacy.io/usage/processing-pipelines#custom-components

The parses might be odd if the texts are very different from the kinds of sentences seen during training, but it should work technically.