Loading 'en_core_web_sm' pipeline on a deployed web app?

[oozr]

Hi all! I've deployed a web app using Flask, via Railway.app. Using Github.
Repository: https://github.com/oozr/globalscaleoflanguages
Website: www.globalscaleoflanguages.com

The web app is a text anlyser. Asks for input text from the user, then analyses the text using the Python library Textstat.
All good so far. I'm now trying to make it more advanced by integrating Spacy library to lemmatize the text (to then lookup the lemmatized words in a sql table of words that I have with rated difficulty). It works in my offline environment because I have the pipeline downloaded on my local host. My issue is in loading the pipeline for the deployed app: error with the code: nlp = spacy.load("en_core_web_sm").

Spacy is in my requirements.txt file. But I'm not sure how to load/add the pipeline for visitors to the web app. Is this something that needs to be loaded every time a user comes to the web app/makes a request? If so how do I go about loading the pipeline. Or would it be better to go via an API route?

This is the best help I could find on StackOverflow, but to be honest I don't really understand it as I'm fairly new to CS.
https://stackoverflow.com/questions/46990884/how-to-use-spacy-nlp-from-within-a-website - answer from Ines Montahy.

Thanks guys, any advice appreciated.

[danieldk]

Hi, I am not sure if I fully understand your question. Is the issue the deployment of the model with the web app? If so, you could add the model to requirements.txt. The models are distributed as Python wheels. For example:

spacy==3.5.3
https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.5.0/en_core_web_sm-3.5.0-py3-none-any.whl
# Your other dependencies.

The spacy dependency is not really needed here, because the model depends on the spacy package, so it will be downloaded as a transitive dependency.

[oozr]

That's great, that works! Thank you.
I'm told I can download the spacy profile on Railway by adding the following to the Procfile:
web: python -m spacy download en_core_web_sm && gunicorn main:app

Both actually work! I'm not sure why either work, or if they're different solutions. But thank you all the same!