ValueError('[E1010] Unable to set entity information for token 3 which is included in more than one span in entities, blocked, missing or outside.')

[Yukari-Tryhard]

How to reproduce the behaviour

First here is the code:

ents_data = json["ents_data"]
for ents in ents_data:
                # print(ents, ":")
                doc = nlp(ents["text"])
                for ent in ents["ents"]:
                    start = ent["start"]
                    end = ent["end"]
                    label = ent["type"]
                    span = doc.char_span(int(start), int(end), label=label)
                    if not _temp_ents.count(span.text):
                        _ents.append(span)
                        _temp_ents.append(span.text)
                doc.ents = _ents
                db.add(doc)

And here is my json:

"ents_data": 
  [
    {
      "text": "I have a fever and stomach hurt", 
      "ents": [
      {
        "start": "9",
        "end": "14",
        "type": "SYMPTOM"
      },
      {
        "start": "19",
        "end": "31",
        "type": "SYMPTOM"
      }
      ]
    },
    {
      "text": "I'm experiencing head pain and a fever", 
      "ents": [
      {
        "start": "17",
        "end": "26",
        "type": "SYMPTOM"
      }
      ]
    }
  ]

When i run the above code there is an error: ValueError('[E1010] Unable to set entity information for token 3 which is included in more than one span in entities, blocked, missing or outside.')

But i checked there are only three spans in the entities array which are "Fever", "Stomach hurt" and "Head pain". Noway Head pain overlapped with Stomach hurt right guys?. I just can't figure why?

My Environment

• Operating System: Windows 11
• Python Version Used: 3.10.10
• spaCy Version Used: v3.5

[thomashacker]

Hey,
Thanks for the report, however, this isn't a spaCy issue.

The problem is the _ents list, every iteration, you're adding more entities to the list from different doc objects and trying to set them as .ents per individual doc. That's why you're getting the ValueError.

Here's a suggestion on how to fix your problem, you initialize the _ents list inside the loop, so that it doesn't collect all the entities from different doc objects.

for ents in ents_data:
                doc = nlp(ents["text"])
                _ents = [] # Add the list inside the loop
                for ent in ents["ents"]:
                    start = ent["start"]
                    end = ent["end"]
                    label = ent["type"]
                    span = doc.char_span(int(start), int(end), label=label)
                doc.ents = _ents

Hope this helps!