Using spaCyOpenTapioca with spacy_streamlit

[aouriri]

Forgive me if I should ask this elsewhere...I'll be happy to do that, just point me in the right direction!

I'm working on an app in Streamlit and wanted to use spaCy, thankfully, there's spacy-streamlit! I've adapted one of the examples for my project but I also wanted to incorporate spaCyOpenTapioca. I've been able to run the examples in Jupyter Notebook, but I'm unsure of how to fold OpenTapioca within spacy-streamlit. I've been working on this for some time, so I may be too deep to see a solution. Any suggestions (if this is even possible) are appreciated. I'm willing to manually do this outside of spacy-streamlit (again, if possible).

[adrianeboyd]

Hi, can you provide some more details about what you're trying to do and what problems you're running into? It's easiest to help if you can provide some sample code that we can run on our end to see more details.

[aouriri]

Hi Adriane,

My apologies for the late reply, I had been trying to work it out and figure out a better way of explaining this. Thank you for sharing the link. What I'm wanting to do and why I wanted to incorporate OpenTapioca into my spaCy-Streamlit project is to automatically populate whatever is entered into the Streamlit text box with the appropriate QID/link (if one exists) for the highlighted entity. Going back to the example image, the QIDs weren't manually entered or added. I'm adding an image of OpenTapioca in a Jupyter Notebook. The code is similar to the example code but I've changed the "doc". Now, the entity "Cesar Chavez" is identified and the QID is properly populated. Now, I see it works on its own via the Notebook example but I would like it to work within the Streamlit app.

Please let me know if this does/doesn't make sense, won't work, or if you can see the example image.

[adrianeboyd]

Can you show your streamlit example?

[aouriri]

This is what I currently have and I don't get an error with this:

st.title('Named Entity Recognition')

	DEFAULT_TEXT = """Google was founded in September 1998 by Larry Page and Sergey Brin while they were Ph.D. students at Stanford University in California. Together they own about 14 percent of its shares and control 56 percent of the stockholder voting power through supervoting stock. They incorporated Google as a California privately held company on September 4, 1998, in California. Google was then reincorporated in Delaware on October 22, 2002."""
	
	spacy_model = "en_core_web_sm"
	
	text = st.text_area("Text to analyze (Default text can be used, but I'm okay with change.)", DEFAULT_TEXT, height=200)
	doc = spacy_streamlit.process_text(spacy_model, text)

	spacy_streamlit.visualize_ner(
		doc,
		labels=["PERSON", "DATE", "GPE", "ORG", "NORP", "LAW", "LOC"],
		show_table=False,
		title="Person, Places and Other Things",
	)

[adrianeboyd]

Ah, is your question just how to add the pipe so it can be loaded from the model argument that you pass in?

Here is a quick, hacky version. You can also load from a path instead of a name, so this adds the component and saves the pipeline to a temporary directory. In a better version, you'd want to save the model to a more suitable path (which you can do in advance outside of this script) or even better, you could package it as a named spacy model with spacy package, install it, and then load from the name instead of the path.

import streamlit as st
import spacy
import spacy_streamlit

st.title('Named Entity Recognition')

DEFAULT_TEXT = """Google was founded in September 1998 by Larry Page and Sergey Brin while they were Ph.D. students at Stanford University in California. Together they own about 14 percent of its shares and control 56 percent of the stockholder voting power through supervoting stock. They incorporated Google as a California privately held company on September 4, 1998, in California. Google was then reincorporated in Delaware on October 22, 2002."""

spacy_model = "en_core_web_sm"
nlp = spacy.load(spacy_model)
nlp.add_pipe("opentapioca")
nlp.to_disk("/tmp/en_core_web_sm_ot")

text = st.text_area("Text to analyze (Default text can be used, but I'm okay with change.)", DEFAULT_TEXT, height=200)
doc = spacy_streamlit.process_text("/tmp/en_core_web_sm_ot", text)

spacy_streamlit.visualize_ner(
        doc,
        labels=["PERSON", "DATE", "GPE", "ORG", "NORP", "LAW", "LOC"],
        show_table=False,
        title="Person, Places and Other Things",
)

The links to the knowledge base entries aren't set up to be linked to anything in this version. I think that would require some additional arguments/setting in the ner visualizer (I think this PR is related: explosion/spacy-streamlit#31).

This is what it looks like on my end:

[aouriri]

Yes! I knew we'd get here at some point, thank you for putting it so eloquently. I'll take a look at the PR and play around with things to get it linked. I just wanted to, at least, get to the point where I could add the IDs. Thank you, thank you!