Conflicts with Spacy language models and Streamlit?

[robertpfaff]

I am running into some issues with Spacy language models, specifically en_core_web_lg and _sm, now that I have tried to introduce Streamlit as an interface for my app. It does not recognize that either model exists.

My earnest efforts have not produced any progress. Can anyone suggest some ways to resolve this problem? I have attached some images of the error messages I received. It also does not recognize that I have installed "pip install spacy-streamlit" when I try "import spacy_streamlit." I cannot figure out what I am doing wrong. The code here on the Spacy-streamlit support page produces the same error message. I have displayed it in its entirety, though I fear its length will confuse matters further:

(venv) C:\Users\rober\food_ner>streamlit run spst.py

  You can now view your Streamlit app in your browser.

  Local URL: http://localhost:8501
  Network URL: http://192.168.4.59:8501

C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy\util.py:841: UserWarning: [W094] Model 'en_core_web_sm' (2.2.0) specifies an under-constrained spaCy version requirement: >=2.2.0. This can lead to compatibility problems with older versions, or as new spaCy versions are released, because the model may say it's compatible when it's not. Consider changing the "spacy_version" in your meta.json to a version range, with a lower and upper pin. For example: >=3.2.0,<3.3.0
  warnings.warn(warn_msg)
2021-11-21 17:09:15.348 Traceback (most recent call last):
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\legacy_caching\caching.py", line 509, in get_or_create_cached_value
    return_value = _read_from_cache(
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\legacy_caching\caching.py", line 308, in _read_from_cache
    raise e
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\legacy_caching\caching.py", line 293, in _read_from_cache
    return _read_from_mem_cache(
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\legacy_caching\caching.py", line 212, in _read_from_mem_cache
    raise CacheKeyNotFoundError("Key not found in mem cache")
streamlit.legacy_caching.caching.CacheKeyNotFoundError: Key not found in mem cache

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\script_runner.py", line 354, in _run_script
    exec(code, module.__dict__)
  File "C:\Users\rober\food_ner\spst.py", line 7, in <module>
    spacy_streamlit.visualize(models, default_text)
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy_streamlit\visualizer.py", line 71, in visualize
    nlp = load_model(spacy_model)
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\legacy_caching\caching.py", line 543, in wrapped_func
    return get_or_create_cached_value()
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\legacy_caching\caching.py", line 525, in get_or_create_cached_value
    return_value = func(*args, **kwargs)
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy_streamlit\util.py", line 9, in load_model
    return spacy.load(name)
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy\__init__.py", line 51, in load
    return util.load_model(
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy\util.py", line 420, in load_model
    return load_model_from_package(name, **kwargs)  # type: ignore[arg-type]
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy\util.py", line 453, in load_model_from_package
    return cls.load(vocab=vocab, disable=disable, exclude=exclude, config=config)  # type: ignore[attr-defined]
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\en_core_web_sm\__init__.py", line 12, in load
    return load_model_from_init_py(__file__, **overrides)
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy\util.py", line 615, in load_model_from_init_py
    return load_model_from_path(
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy\util.py", line 487, in load_model_from_path
    config = load_config(config_path, overrides=overrides)
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy\util.py", line 646, in load_config
    raise IOError(Errors.E053.format(path=config_path, name="config.cfg"))
OSError: [E053] Could not read config.cfg from C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\en_core_web_sm\en_core_web_sm-2.2.0\config.cfg

2021-11-21 17:09:15.833 Traceback (most recent call last):
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\legacy_caching\caching.py", line 509, in get_or_create_cached_value
    return_value = _read_from_cache(
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\legacy_caching\caching.py", line 308, in _read_from_cache
    raise e
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\legacy_caching\caching.py", line 293, in _read_from_cache
    return _read_from_mem_cache(
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\legacy_caching\caching.py", line 212, in _read_from_mem_cache
    raise CacheKeyNotFoundError("Key not found in mem cache")
streamlit.legacy_caching.caching.CacheKeyNotFoundError: Key not found in mem cache

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\script_runner.py", line 354, in _run_script
    exec(code, module.__dict__)
  File "C:\Users\rober\food_ner\spst.py", line 7, in <module>
    spacy_streamlit.visualize(models, default_text)
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy_streamlit\visualizer.py", line 71, in visualize
    nlp = load_model(spacy_model)
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\legacy_caching\caching.py", line 543, in wrapped_func
    return get_or_create_cached_value()
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\legacy_caching\caching.py", line 525, in get_or_create_cached_value
    return_value = func(*args, **kwargs)
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy_streamlit\util.py", line 9, in load_model
    return spacy.load(name)
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy\__init__.py", line 51, in load
    return util.load_model(
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy\util.py", line 420, in load_model
    return load_model_from_package(name, **kwargs)  # type: ignore[arg-type]
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy\util.py", line 453, in load_model_from_package
    return cls.load(vocab=vocab, disable=disable, exclude=exclude, config=config)  # type: ignore[attr-defined]
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\en_core_web_sm\__init__.py", line 12, in load
    return load_model_from_init_py(__file__, **overrides)
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy\util.py", line 615, in load_model_from_init_py
    return load_model_from_path(
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy\util.py", line 487, in load_model_from_path
    config = load_config(config_path, overrides=overrides)
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy\util.py", line 646, in load_config
    raise IOError(Errors.E053.format(path=config_path, name="config.cfg"))
OSError: [E053] Could not read config.cfg from C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\en_core_web_sm\en_core_web_sm-2.2.0\config.cfg

2021-11-21 17:09:16.252 Traceback (most recent call last):
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\legacy_caching\caching.py", line 509, in get_or_create_cached_value
    return_value = _read_from_cache(
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\legacy_caching\caching.py", line 308, in _read_from_cache
    raise e
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\legacy_caching\caching.py", line 293, in _read_from_cache
    return _read_from_mem_cache(
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\legacy_caching\caching.py", line 212, in _read_from_mem_cache
    raise CacheKeyNotFoundError("Key not found in mem cache")
streamlit.legacy_caching.caching.CacheKeyNotFoundError: Key not found in mem cache

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\script_runner.py", line 354, in _run_script
    exec(code, module.__dict__)
  File "C:\Users\rober\food_ner\spst.py", line 7, in <module>
    spacy_streamlit.visualize(models, default_text)
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy_streamlit\visualizer.py", line 71, in visualize
    nlp = load_model(spacy_model)
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\legacy_caching\caching.py", line 543, in wrapped_func
    return get_or_create_cached_value()
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\legacy_caching\caching.py", line 525, in get_or_create_cached_value
    return_value = func(*args, **kwargs)
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy_streamlit\util.py", line 9, in load_model
    return spacy.load(name)
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy\__init__.py", line 51, in load
    return util.load_model(
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy\util.py", line 420, in load_model
    return load_model_from_package(name, **kwargs)  # type: ignore[arg-type]
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy\util.py", line 453, in load_model_from_package
    return cls.load(vocab=vocab, disable=disable, exclude=exclude, config=config)  # type: ignore[attr-defined]
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\en_core_web_sm\__init__.py", line 12, in load
    return load_model_from_init_py(__file__, **overrides)
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy\util.py", line 615, in load_model_from_init_py
    return load_model_from_path(
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy\util.py", line 487, in load_model_from_path
    config = load_config(config_path, overrides=overrides)
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy\util.py", line 646, in load_config
    raise IOError(Errors.E053.format(path=config_path, name="config.cfg"))
OSError: [E053] Could not read config.cfg from C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\en_core_web_sm\en_core_web_sm-2.2.0\config.cfg

2021-11-21 17:09:48.738 Traceback (most recent call last):
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\legacy_caching\caching.py", line 509, in get_or_create_cached_value
    return_value = _read_from_cache(
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\legacy_caching\caching.py", line 308, in _read_from_cache
    raise e
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\legacy_caching\caching.py", line 293, in _read_from_cache
    return _read_from_mem_cache(
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\legacy_caching\caching.py", line 212, in _read_from_mem_cache
    raise CacheKeyNotFoundError("Key not found in mem cache")
streamlit.legacy_caching.caching.CacheKeyNotFoundError: Key not found in mem cache

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\script_runner.py", line 354, in _run_script
    exec(code, module.__dict__)
  File "C:\Users\rober\food_ner\spst.py", line 7, in <module>
    spacy_streamlit.visualize(models, default_text)
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy_streamlit\visualizer.py", line 71, in visualize
    nlp = load_model(spacy_model)
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\legacy_caching\caching.py", line 543, in wrapped_func
    return get_or_create_cached_value()
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\legacy_caching\caching.py", line 525, in get_or_create_cached_value
    return_value = func(*args, **kwargs)
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy_streamlit\util.py", line 9, in load_model
    return spacy.load(name)
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy\__init__.py", line 51, in load
    return util.load_model(
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy\util.py", line 420, in load_model
    return load_model_from_package(name, **kwargs)  # type: ignore[arg-type]
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy\util.py", line 453, in load_model_from_package
    return cls.load(vocab=vocab, disable=disable, exclude=exclude, config=config)  # type: ignore[attr-defined]
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\en_core_web_sm\__init__.py", line 12, in load
    return load_model_from_init_py(__file__, **overrides)
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy\util.py", line 615, in load_model_from_init_py
    return load_model_from_path(
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy\util.py", line 487, in load_model_from_path
    config = load_config(config_path, overrides=overrides)
  File "C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\spacy\util.py", line 646, in load_config
    raise IOError(Errors.E053.format(path=config_path, name="config.cfg"))
OSError: [E053] Could not read config.cfg from C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\en_core_web_sm\en_core_web_sm-2.2.0\config.cfg

  Stopping...

(venv) C:\Users\rober\food_ner>python -m spacy init fill-config config-v3.1.cfg config-v3.2.cfgpython -m spacy init fill-config config-v3.1.cfg config-v3.2.cfg
Usage: python -m spacy init fill-config [OPTIONS] BASE_PATH [OUTPUT_FILE]
Try 'python -m spacy init fill-config --help' for help.

Error: no such option: -m

(venv) C:\Users\rober\food_ner>python spacy init fill-config config-v3.1.cfg config-v3.2.cfg
python: can't open file 'C:\Users\rober\food_ner\spacy': [Errno 2] No such file or directory

(venv) C:\Users\rober\food_ner>

When I search for answers online, most suggest re-installing the components.

But that is not working for me.

1. The first image validates that I have installed both language models correctly.

2. The second image is the error message I receive, similar to the one above.

I simply don't know what it means.

.
3) The third image shows the complete error message as it appears in Streamlit in the browser.

Special thanks to anyone who can provide me with sincere feedback. I have not been feeling well this past week, and I am struggling to make heads or tails of this problem. I have tried all the solutions I can find online without much success.

Thank you again.

[ljvmiranda921]

Hi @robertpfaff , it seems that there's a conflict between (1) the model version Streamlit is looking for and (2) the one installed in the environment.

The likely explanation is that it's an environment problem. For example, this error message looks for en_core_web_sm-2.2.0, but we have en_core_web_sm-3.1.0 installed as shown in the validate command (based from your first image):

OSError: [E053] 
Could not read config.cfg from C:\Users\rober\AppData\Local\Programs\Python\Python310\lib\site-packages\en_core_web_sm\en_core_web_sm-2.2.0\config.cfg

The paths also look different. In the validate command, we have C:\Users\rober\AppData\Local\Programs\Python\Python39 but in the traceback, we have C:\Users\rober\AppData\Local\Programs\Python\Python310 (3.9 vs. 3.10). Different python environments are being run.

Some options you have for a fix

1. Install the updated en_core_web_sm-3.1.0 model in your Python 3.10 environment:

$ C:\Users\rober\AppData\Local\Programs\Python\Python310\python.exe -m download en_core_web_sm

This should install the updated en_core_web_sm file in your environment.
In some installations, I observed that using py gives you the correct python environment in Windows. So you can also try:

$ py -m download en_core_web_sm

2. Run the streamlit app in your 3.9 environment, not in the 3.10. The streamlit app is running in Python 3.10, which doesn't have the correct spaCy model version.

3. As a more permanent measure, you might want to configure Python to just use a single environment. Better yet, ensure that the virtual environment is configured correctly. It seems that even though we have the virtualenv active in the screenshots, the python path isn't coming from one of those venvs.

[robertpfaff]

Well, thank you! Having two versions of Python on the machine was the source of the problem.

I am fairly new to programming and Python (about six months) and I remember upgrading to Python 10. But I did not think that would lead to these kinds of conflicts. I tried all the options above, but the first one seemed to work the best. I just downloaded "en_core_web_sm" into the path (C:\Users\xxxx\AppData\Local\Programs\Python\Python310) and it started to respond.

The other options did not seem to work as well because Streamlit just did not like the idea of looking for "en_core_web_sm" anywhere other than the Python 10 directory. I could not trick it into looking in the Python 3.9 directory. I also had to settle for "en_core_web_sm" - not "en_core_web_lg - as I received time-out errors downloading the larger file.

I am recording the details mostly in case someone needs this information in the future. I don't want to drag you into a long discussion, so please feel free not to respond. Its a minor issue at this point. Its just that I see no reason to keep two versions on my machine, and I am wondering if I should uninstall and reinstall Python to prevent the issue from popping back up?

You saved me hours of frustration while I am trying to recover from a bad cold.

So thank you again.

[polm]

Its just that I see no reason to keep two versions on my machine, and I am wondering if I should uninstall and reinstall Python to prevent the issue from popping back up?

Yes, there's no good reason to have two different versions on your machine like you do now, so if you can figure out how to get rid of both of them and then install just one of them that should make things easier and prevent future problems.

Also, using virtualenvs can help you avoid version-related problems like that, if not perfectly, so I'd also recommend that.