Extracting output of displacy.serve

[EvelynGriffith]

I've been able to run displacy.serve so that it opens a localhost browser displaying this: "Jerry PERSON walked past the School ORG, down to the lake where Martin PERSON was fishing.", but I was wondering if it's possible to extract the words that have labels from this output? Is there a way that I could filter those words into lists based on the label that displacy gave them?

[kadarakos]

Hey EvelynGriffith,

I'm not sure if I understand everything correctly -- due to not having a code snippet -- but to me it seems like you are running displacy.serve(doc, style="dep").

The named entity annotations you are looking for are stored on the doc which is an instance of the Doc: https://spacy.io/api/doc.
You can find the entity spans in the document like this:

import spacy

nlp = spacy.load("en_core_web_sm")
doc = nlp("Jerry walked past the School, down to the lake where Martin was fishing.")
print(doc.ents)

This should print (Jerry, School, Martin). The elements of doc.ents are Spans: https://spacy.io/api/span

for ent in doc.ents:
    print(doc[ent.start:ent.end])

This prints:

Jerry
School
Martin

If you'd like to just extract the strings of various entity types you can maybe do:

from collections import defaultdict

ents_dict = defaultdict(list)
for ent in doc.ents:
    ents_dict[ent.label_].append(ent.text)

print(ents_dict)

This prints:

defaultdict(list, {'PERSON': ['Jerry', 'Martin'], 'ORG': ['School']})

You can extract the various types of entities from the dictionary with ents_dict["PERSON"], ents_dict["PERSON"] and so forth.

[EvelynGriffith]

Okay yes that’s what I was hoping to do! Thank you!

…

On Fri, Jun 16, 2023 at 5:31 AM kadarakos ***@***.***> wrote: Hey EvelynGriffith, I'm not sure if I understand everything correctly -- due to not having a code snippet -- but to me it seems like you are running displacy.serve(doc, style="dep"). The named entity annotations you are looking for are stored on the doc which is an instance of the Doc: https://spacy.io/api/doc. You can find the entity spans in the document like this: import spacy nlp = spacy.load("en_core_web_sm") doc = nlp("Jerry walked past the School, down to the lake where Martin was fishing.") print(doc.ents) This should print (Jerry, School, Martin). The elements of doc.ents are Spans: https://spacy.io/api/span for ent in doc.ents: print(doc[ent.start:ent.end]) This prints: Jerry School Martin — Reply to this email directly, view it on GitHub <#12726 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQ2UKMZJN2WW3CMG5WXGDHDXLRG3DANCNFSM6AAAAAAZGZAKDY> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[EvelynGriffith]

Is there a way to edit the labels that "spacy.displacy.serve(piano_class_doc, style="ent")" gives? I have it saying that the name "Daniel" is an ORG. Right now I have code that matches the name given in the text to a name in my external list, but once I have that it says that the names aren't tagged as "PERSON" I was hoping that I could change the way that DisplaCy is tagging so that it will recognize the names in my list and change the tag to "PERSON" whenever a it matches the name in the list.

with open("C:\Users\gforc\MITACs_research_workspace\src\using\names_list.py") as file:
nlp = spacy.load("en_core_web_sm")
piano_class_text = input("Add your text: ")
piano_class_doc = nlp(piano_class_text)
spacy.displacy.serve(piano_class_doc, style="ent")
# print(names)
for ent in piano_class_doc.ents:
non_person_ents = []
new_list = []
person_ents = []
entities = list(piano_class_doc.ents)
for i in entities:
label = i.label_
if label == "PERSON":
person_ents.append(i)
# print(person_ents)
if label != "PERSON":
# print(label)
non_person_ents.append(i)
# print(non_person_ents)
# print(person_ents)
# print(non_person_ents)
for name in names:
# print(name)
for i in entities:
# print(i.label_)
if name == (str(i)):
i.label_ == "PERSON" #<-- this is the line that won't work. I was hoping that I could change the label to "PERSON"
print(i, " should have a", i.label_, "label, and is now labeled with", i.label_)

[kadarakos]

May I ask you to share your code using formatting and a code environment with syntax highlighting?

[EvelynGriffith]

I tried getting it to highlight the way it does in VSCode, but it wasnt working, and I couldn't get the file to

from names_list import names
from spacy.tokens import Span
import spacy
import en_core_web_sm
nlp = en_core_web_sm.load()
from spacy import displacy

with open("C:\\Users\\gforc\\MITACs_research_workspace\\src\\using\\names_list.py") as file:
    nlp = spacy.load("en_core_web_sm")
    piano_class_text = input("Add the Biblical verse you would like to animate: ")
    piano_class_doc = nlp(piano_class_text)
    spacy.displacy.serve(piano_class_doc, style="ent")
    # print(names)
    for ent in piano_class_doc.ents:
        non_person_ents = []
        new_list = []
        person_ents = []
        entities = list(piano_class_doc.ents)
        for i in entities:
            label = i.label_
            if label == "PERSON":
                person_ents.append(i)
            # print(person_ents)
            if label != "PERSON":
                # print(label)
                non_person_ents.append(i)
                old_ent = entities[0]
                new_label = Span(piano_class_doc, old_ent.start, old_ent.end, label="PERSON")
            # print(non_person_ents)
        # print(person_ents)
        # print(non_person_ents)
        for name in names:
            # print(name)
            for i in entities:
                print(i.label_)
                if name == (str(i)):
                    i.label_ == "PERSON"
                    print(i, " should have a", i.label_, "label, and is now labeled with", i.label_)

[kadarakos]

If you'd like to change the labels and it being reflected on doc.ents then its better to create a new list of entities and assign it back to doc.ents. So you can loop through doc.ents and for each entity you can decide whether to just add it to new_ents or first create a new Span and then add it to new_ents. Finally, at the end you can assign doc.ents = new_ents.

[EvelynGriffith]

Okay, that makes sense, is there any way that you could give me an example (even if it doesn't pertain to my code) just so that I can get a general idea of how to implement that?

[kadarakos]

We have lots of tutorials in the documentation of spaCy. Please checkout this section for more information on the topic: https://spacy.io/usage/linguistic-features#setting-entities

[EvelynGriffith]

okay, thank you very much for your help!