Pipeline Instanitation using from_confg

[Fourthought]

Could you provide some assistance with how to instantiate a pipeline using from_config please?

• https://spacy.io/api/language

You will see in the code below that I've created a pipeline using spacy.load(en_core_web_sm) and added a custom component.

The code then saves the pipeline config file to disk and instantiates a new pipeline using the same config file.

Rather than using config.to_disk and 'config.from_disk' I would like to do it this way and then add patterns following instantiation. As far as I can tell, the to_disk and from_disk methods means saving and loading a default set of patterns, but I have different patterns for different use cases.

... continues below code

`

from hamoc_ta.pipeline.custom_tokeniser import CustomTokeniser

import spacy
from pathlib import Path 
import srsly
from spacy.lang.en import English

if __name__ == "__main__":

    text = "hello world!"

    nlp = spacy.load("en_core_web_sm")

    patterns = [{"label": "PHRASE", "pattern": [{"LOWER": "hello"}, {"LOWER": "world"}]}]
    nlp.add_pipe("CustomTokeniser", name="custom_matcher")
    nlp.get_pipe("custom_matcher").add_patterns(patterns)

    print(nlp.pipe_names)
    doc = nlp(text)
    print([tok for tok in doc])
    print([f"{ent} => {ent.label_}" for ent in doc.ents])

    config_path = Path(nlp.path) / "hamoc.cfg"
    srsly.write_json(config_path, nlp.config)

    config = srsly.read_json(config_path)

    new_nlp = English.from_config(config)
    print(new_nlp.pipe_names)

    print(nlp.meta["name"], '+>', new_nlp.meta["name"])

    print(new_nlp.pipe_names)
    doc = new_nlp(text)
    print([tok for tok in doc])
    print([f"{ent} => {ent.label_}" for ent in doc.ents])

`

You will see in this output that the original pipeline with the custom component works, and the new pipeline contains the custom component.

However, you will also see how the pipeline names are different and that attempting to run the pipeline throws up an error.

Do you know what the problem is and how it may be solved?

`

['tok2vec', 'tagger', 'parser', 'attribute_ruler', 'lemmatizer', 'ner', 'custom_matcher']
[hello world, !]
['hello world => PHRASE']
['tok2vec', 'tagger', 'parser', 'attribute_ruler', 'lemmatizer', 'ner', 'custom_matcher']
core_web_sm +> pipeline

File "path/to/file", line 32, in <module>
doc = new_nlp(text)
File "path/to/file", line 1002, in __call__
raise ValueError(Errors.E109.format(name=name)) from e
ValueError: [E109] Component 'tok2vec' could not be run. Did you forget to call `initialize()`?

`

[polm]

The config contains state about the pipeline components, but it doesn't contain all the state, so when you make a new pipeline and initialize it from the old config you get all the same components but they aren't ready to go. You don't get patterns, and components aren't initialized, which is why you get that error. You can read more about this process in "serializing the pipeline".

As far as I can tell, the to_disk and from_disk methods means saving and loading a default set of patterns, but I have different patterns for different use cases.

I am not sure what you mean by "a default set of patterns", but those methods should save all your patterns. You could save one Language per use case if you need to differentiate patterns. That said, loading serialized patterns is basically the same as just loading patterns normally (there are no speed benefits or anything), so if it's easier you can also just save your patterns somewhere and load them at runtime.