German Entity Recognition incorrect star_char/end_char

[lsmith77]

How to reproduce the behaviour

model = spacy.load("de_core_news_lg")
# "Freundliche Grüße Nadia" (start_char: 0, end_char: 22, label_: "PER")
tokens = model("Freundliche Grüße Nadia")

# "Freundliche Grüße" (start_char: 0, end_char: 16, label_: "PER")
# "Nadia" (start_char: 32, end_char: 36, label_: MISC)
tokens = model("Freundliche Grüße meine liebste Nadia") 

# "Hallo Herr Müller" (start_char: 0, end_char: 16, label_: "PER")
tokens = model("Hallo Herr Müller.")

# no entities recognized
tokens = model("Hallo Herr Müller")

In the comments above we are examining tokens.ents.

Note for English we were in a position to reproduce the documented behavior, where it would accurately identify the entity.

That being said, in English we noticed some issues with typos, however using en_core_web_trf solved those.

Unfortunately de_dep_news_trf does not support entities.

Your Environment

• Operating System: OSX
• Python Version Used: 3.11
• spaCy Version Used: 3.5.1 (de-core-news-lg)
• Environment Information:

[lsmith77]

@DuyguA maybe this ticket interests you?

[DuyguA]

@DuyguA maybe this ticket interests you?

I had a quick look, it seems to me that the issue is greetings doesn't exist in NER datasets usually. As a result model is clueless in such lines. Also, having rest of the paragraph would help, this expressions are not full sentences anyway just a greeting and a name.

I see 2 options here:

• Make a small corpus of such cases with greeting + person names, update the NER component so make a custom model.
• Second option is to play some tricks. In the second example, I put a comma after Hallo and a period at the sentence like this: Hallo, Herr Mueller. , it worked. In the first sentence, I cut the greeting totally and the expressions is only Nadia, then recognized correctly by the model.
  This option should be quite easy, there are only a countable set of greetings is German language ... so you can implement a little preprocessor to cut the greetings maybe?
• Third option is to use the EntityRuler component, if you see a greeting, and if the next words are in capitals; most probably they're parts of a person name.

[svlandeg]

Hi all,

as this is not an actual bug but rather relates to incorrect predictions of the pretrained models (and what to do about that), I'll move this to discussions.