TextCategorizer.from_disk unable to load trained model

[1danjordan]

How to reproduce the behaviour

I have a spaCy project that trains two separate text classifiers. I'm trying to load both models into a spaCy Language object, but it appears TextCategorizer.from_disk doesn't work because it's expecting to find a strings.json file in the wrong directory. I'm following this thread on Prodigy support. First I train a model with spacy train:

python -m spacy train configs/config.cfg --output training/ --paths.train corpus/train.spacy --paths.dev corpus/dev.spacy --nlp.lang en

Then I try to load the textcat model like so

import spacy
nlp = spacy.load('blank:en')
textcat = nlp.create_pipe('textcat')
textcat.from_disk('training/model-best/textcat/')

Resulting in this traceback:

ValueError                                Traceback (most recent call last)
<ipython-input-4-99891af671ac> in <module>
----> 1 textcat.from_disk('training/debit-model/model-best/textcat/')

~/opt/miniconda3/envs/txn-tagger/lib/python3.9/site-packages/spacy/pipeline/trainable_pipe.pyx in spacy.pipeline.trainable_pipe.TrainablePipe.from_disk()

~/opt/miniconda3/envs/txn-tagger/lib/python3.9/site-packages/spacy/util.py in from_disk(path, readers, exclude)
   1195         # Split to support file names like meta.json
   1196         if key.split(".")[0] not in exclude:
-> 1197             reader(path / key)
   1198     return path
   1199

~/opt/miniconda3/envs/txn-tagger/lib/python3.9/site-packages/spacy/pipeline/trainable_pipe.pyx in spacy.pipeline.trainable_pipe.TrainablePipe.from_disk.lambda8()

~/opt/miniconda3/envs/txn-tagger/lib/python3.9/site-packages/spacy/vocab.pyx in spacy.vocab.Vocab.from_disk()

~/opt/miniconda3/envs/txn-tagger/lib/python3.9/site-packages/spacy/strings.pyx in spacy.strings.StringStore.from_disk()

~/opt/miniconda3/envs/txn-tagger/lib/python3.9/site-packages/srsly/_json_api.py in read_json(path)
     49         data = sys.stdin.read()
     50         return ujson.loads(data)
---> 51     file_path = force_path(path)
     52     with file_path.open("r", encoding="utf8") as f:
     53         return ujson.load(f)

~/opt/miniconda3/envs/txn-tagger/lib/python3.9/site-packages/srsly/util.py in force_path(location, require_exists)
     22         location = Path(location)
     23     if require_exists and not location.exists():
---> 24         raise ValueError(f"Can't read file: {location}")
     25     return location
     26

ValueError: Can't read file: training/model-best/textcat/vocab/strings.json

I've been looking at the docs and the source code and am surprised because this seems to be the exact what that this is documented but maybe I've overlooked something here?

Info about spaCy

• spaCy version: 3.1.1
• Platform: macOS-11.5.2-x86_64-i386-64bit
• Python version: 3.9.6
• Pipelines: en_core_web_sm (3.1.0)

[adrianeboyd]

You want to load it like this:

import spacy
nlp = spacy.load("training/model-best")

# usually you just want to run the whole pipeline
doc = nlp("This is a text.")

# if you want to inspect the component for some reason
textcat = nlp.get_pipe("textcat")

You don't want to load individual components like this because you're missing the configuration from config.cfg and the shared vocab from the top-level directory in the saved pipeline (training/model-best). For more details, have a look at: https://spacy.io/usage/processing-pipelines#pipelines

[1danjordan]

Hey @adrianeboyd,

Thanks a million for getting back to me. I'm slightly confused, was loading individual components like this possible in spaCy 2? I was working off of this example @honnibal wrote in Prodigy support:

nlp = spacy.load("path/to/model1")
textcat2 = nlp.create_pipe("textcat")
textcat2.from_disk("path/to/model2/textcat")
nlp.add_pipe(textcat2, name="textcat2")
nlp.to_disk("some/output_directory")

and the TrainablePipe class has from_disk method as documented here.

If wanted to get two textcat models into the same spaCy model, would I have to manually update config.cfg with the components and place them both in the same model directory with the shared vocab e.g.

├── config.cfg
├── meta.json
├── textcat1
│   ├── cfg
│   └── model
├── textcat2
│   ├── cfg
│   └── model
├── tokenizer
└── vocab
    ├── key2row
    ├── lookups.bin
    ├── strings.json
    └── vectors

And update config.cfg like this?

[nlp]
lang = "en"
pipeline = ["textcat1", "textcat2"]
disabled = []
before_creation = null
after_creation = null
after_pipeline_creation = null
batch_size = 1000
tokenizer = {"@tokenizers":"spacy.Tokenizer.v1"}

[components]

[components.textcat1]
factory = "textcat"
threshold = 0.5

[components.textcat1.model]
# etc. 

[components.textcat2]
factory = "textcat"
threshold = 0.5

[components.textcat2.model]
# etc. 

I guess I'm really trying to understand how others correctly customise their spaCy models when they are adding custom components and so on when using spacy train and spacy package.

Thanks,
Dan

[adrianeboyd]

What you want to do is to source the components from your existing pipelines and use spacy assemble to create the final pipeline from this config:

[components]

[components.textcat1]
source = "my_custom_model1"
component = "textcat"

[components.textcat2]
source = "my_custom_model2"
component = "textcat"

There's no other config for the sourced components because their individual model configs are read from the custom pipeline's config.cfg. Things weren't as customizable in spacy v2, so it was possible to load a component without referencing a config, but now you need the config to know what kind of model was trained underneath.

You can also combine source with training or source-but-not-train using [training.frozen_components], but in general it's easiest to train each separately and then assemble as a final step.

[1danjordan]

Thanks Adriane! You've totally unblocked me 😄