Save and Load a Custom NER model

[vrunm]

I've trained a custom NER model in spaCy with a custom tokenizer. I'd like to save the NER model without the tokenizer. I tried the following code with I found in the spaCy support forum:

import spacy

nlp = spacy.load("en")
nlp.tokenizer = some_custom_tokenizer

nlp.tokenizer = None
nlp.to_disk('my_custom_ner', disable=['tokenizer'])

When I try to load it, the pipeline is empty, and surprisingly, is has the default spaCy tokenizer.

nlp = spacy.blank('en').from_disk('my_custom_ner', disable=['tokenizer'])

Any idea how can I load the model without the tokenizer, but get the full pipeline?
Also is their a way to save the model locally and then be able to load the model in a .py file for inference?

[danieldk]

nlp = spacy.blank('en').from_disk('my_custom_ner', disable=['tokenizer'])

Language.from_disk will only deserialize pipeline components that are defined at the start of deserialization, spacy.blank pipelines do not have any components. spacy.load constructs the pipeline from the configuration first, allowing deserialization deserializes of the pipeline components. So, instead you could try:

nlp = spacy.load('my_custom_ner')

The tokenizer is not a normal pipeline component, so you cannot disable it when loading a model.

I wonder thought, why do you use a custom tokenizer and why only during training?

[vrunm]

I have data where I want tokenizer only during training and during inference I just need the NER part. Is there any way to do that?
Also is their a way to save the model locally and then be able to load the model in a .py file for inference?

[danieldk]

I am not sure I understand yet, you need a tokenizer during inference. Otherwise a Doc cannot be split up into tokens for the NER pipeline component to work on.

[vrunm]

@danieldk I agree I had tried the code below will that approach work?

[vrunm]

This is something I tried:

import ast
def load_data_spacy(training_data):
    """Load the training data from the JSON file and return as tuple"""
    with open(training_data, "r", encoding="utf-8") as fp:
        data = json.load(fp)

    texts = []
    annotations = []
    for example in data:
        text = example["text"]
        entities = example["entities"]
        entities = [(entity[0], entity[1], entity[2]) for entity in entities]
        texts.append(text)
        annotations.append({"entities": entities})

    return list(zip(texts, annotations))

TRAIN_DATA = load_data_spacy("train_data.json")

# Instantiate the nlp object
nlp = spacy.blank("en")

# Create the entity recognizer and add it to the pipeline
ner = nlp.create_pipe("ner")
nlp.add_pipe(ner)

# Add the labels to the entity recognizer
for _, annotations in TRAIN_DATA:
    for entity in annotations.get("entities"):
        ner.add_label(entity[2])

# Disable other pipelines in SpaCy to only train NER
pipe_exceptions = ["ner", "trf_wordpiecer", "trf_tok2vec"]
unaffected_pipes = [pipe for pipe in nlp.pipe_names if pipe not in pipe_exceptions]

# Train the model
with nlp.disable_pipes(*unaffected_pipes):
    optimizer = nlp.begin_training()
    for epoch in range(10):
        random.shuffle(TRAIN_DATA)
        losses = {}
        batches = minibatch(TRAIN_DATA, size=COMPONENT_CONFIGS["ner"]["batch_size"])
        for batch in batches:
            texts, annotations = zip(*batch)
            nlp.update(texts, annotations, sgd=optimizer, drop=COMPONENT_CONFIGS["ner"]["dropout"], losses=losses)
        print(f"Epoch {epoch + 1} Losses:", losses)

But ran into this issue:
Can't find table(s) lexeme_norm for language 'en' in spacy-lookups-data. Make sure you have the package installed or provide your own lookup tables if no default lookups are available for your language.

[kadarakos]

Hey vrunm,

To use lookups for lemmatization and normalization you can install the spacy-lookups-data: https://pypi.org/project/spacy-lookups-data/. You can install it as an extra for spacy like:

pip install -U 'spacy[lookups]'

[vrunm]

@kadarakos I tried this command but still ran into this issue:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/tmp/ipykernel_27/3522050298.py in <module>
     10 # Train the model
     11 with nlp.disable_pipes(*unaffected_pipes):
---> 12     optimizer = nlp.begin_training()
     13     for epoch in range(10):
     14         random.shuffle(TRAIN_DATA)

/opt/conda/lib/python3.7/site-packages/spacy/language.py in begin_training(self, get_examples, sgd)
   1247     ) -> Optimizer:
   1248         warnings.warn(Warnings.W089, DeprecationWarning)
-> 1249         return self.initialize(get_examples, sgd=sgd)
   1250 
   1251     def initialize(

/opt/conda/lib/python3.7/site-packages/spacy/language.py in initialize(self, get_examples, sgd)
   1279         config = self.config.interpolate()
   1280         # These are the settings provided in the [initialize] block in the config
-> 1281         I = registry.resolve(config["initialize"], schema=ConfigSchemaInit)
   1282         before_init = I["before_init"]
   1283         if before_init is not None:

/opt/conda/lib/python3.7/site-packages/confection/__init__.py in resolve(cls, config, schema, overrides, validate)
    727     ) -> Dict[str, Any]:
    728         resolved, _ = cls._make(
--> 729             config, schema=schema, overrides=overrides, validate=validate, resolve=True
    730         )
    731         return resolved

/opt/conda/lib/python3.7/site-packages/confection/__init__.py in _make(cls, config, schema, overrides, resolve, validate)
    776             config = Config(orig_config).interpolate()
    777         filled, _, resolved = cls._fill(
--> 778             config, schema, validate=validate, overrides=overrides, resolve=resolve
    779         )
    780         filled = Config(filled, section_order=section_order)

/opt/conda/lib/python3.7/site-packages/confection/__init__.py in _fill(cls, config, schema, validate, resolve, parent, overrides)
    847                     # We don't want to try/except this and raise our own error
    848                     # here, because we want the traceback if the function fails.
--> 849                     getter_result = getter(*args, **kwargs)
    850                 else:
    851                     # We're not resolving and calling the function, so replace

/opt/conda/lib/python3.7/site-packages/spacy/language.py in load_lookups_data(lang, tables)
    106 def load_lookups_data(lang, tables):
    107     util.logger.debug("Loading lookups from spacy-lookups-data: %s", tables)
--> 108     lookups = load_lookups(lang=lang, tables=tables)
    109     return lookups
    110 

/opt/conda/lib/python3.7/site-packages/spacy/lookups.py in load_lookups(lang, tables, strict)
     28     if lang not in registry.lookups:
     29         if strict and len(tables) > 0:
---> 30             raise ValueError(Errors.E955.format(table=", ".join(tables), lang=lang))
     31         return lookups
     32     data = registry.lookups.get(lang)

ValueError: [E955] Can't find table(s) lexeme_norm for language 'en' in spacy-lookups-data. Make sure you have the package installed or provide your own lookup tables if no default lookups are available for your language.

[kadarakos]

If you have installed the spacy-lookups-data data you should not get this error. My guess would be that maybe you need to restart your ipython kernel?

[vrunm]

@kadarakos
I restarted and tried but got this issue:
I ran into this issue:

[2023-03-20 15:33:42,911] [INFO] Added vocab lookups: lexeme_norm
[2023-03-20 15:33:42,914] [INFO] Created vocabulary
[2023-03-20 15:33:42,917] [INFO] Finished initializing nlp object

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/tmp/ipykernel_27/3522050298.py in <module>
     17         for batch in batches:
     18             texts, annotations = zip(*batch)
---> 19             nlp.update(texts, annotations, sgd=optimizer, drop=COMPONENT_CONFIGS["ner"]["dropout"], losses=losses)
     20         print(f"Epoch {epoch + 1} Losses:", losses)

/opt/conda/lib/python3.7/site-packages/spacy/language.py in update(self, examples, _, drop, sgd, losses, component_cfg, exclude, annotates)
   1131         """
   1132         if _ is not None:
-> 1133             raise ValueError(Errors.E989)
   1134         if losses is None:
   1135             losses = {}

ValueError: [E989] `nlp.update()` was called with two positional arguments. This may be due to a backwards-incompatible change to the format of the training data in spaCy 3.0 onwards. The 'update' function should now be called with a batch of Example objects, instead of `(text, annotation)` tuples. 

[kadarakos]

Hey vrunm,

Did you try to follow the error message? We spend quite a bit of time trying to make sure that the error messages in spaCy are informative. Let us know if you think the message should be improved.

[vrunm]

@kadarakos That's how I have passed the training data:

TRAIN_DATA = [("What is the revenue of Apple in Q1 2016?", {'entities': [(23, 28, 'quarter'), (29, 33, 'year'), (35, 40, 'ticker')]}),
    ("What is the net income of Apple in Q2 2017?", {'entities': [(24, 28, 'quarter'), (29, 33, 'year'), (35, 40, 'ticker')]}),
    ("What is the revenue of AMD in Q3 2018?", {'entities': [(22, 28, 'quarter'), (29, 33, 'year'), (35, 38, 'ticker')]}),
    ("What is the net income of AMD in Q4 2019?", {'entities': [(24, 28, 'quarter'), (29, 33, 'year'), (35, 38, 'ticker')]}),
    ("What is the revenue trend of Amazon from Q1 2016 to Q4 2020?", {'entities': [(37, 41, 'year'), (43, 44, 'quarter'), (56, 62, 'year'), (63, 64, 'quarter'), (75, 81, 'ticker')]}),
    ("What is the outlook for AWS of Amazon in Q2 2021?", {'entities': [(37, 41, 'year'), (42, 44, 'quarter'), (52, 58, 'ticker')]}),
    ("What is the revenue of Cisco in Q3 2017?", {'entities': [(22, 28, 'quarter'), (29, 33, 'year'), (35, 40, 'ticker')]}),
    ("What is the net income of Cisco in Q4 2018?", {'entities': [(24, 28, 'quarter'), (29, 33, 'year'), (35, 40, 'ticker')]}),
    ("What is the revenue trend of Google from Q1 2016 to Q4 2020?", {'entities': [(38, 42, 'year'), (43, 44, 'quarter'), (57, 63, 'year'), (64, 65, 'quarter'), (77, 83, 'ticker')]}),
    ("What is the outlook for Google's ad revenue in Q2 2021?", {'entities': [(38, 42, 'year'), (43, 44, 'quarter'), (51, 57, 'ticker')]}),
    ("What is the revenue of Microsoft in Q2 2018?", {'entities': [(22, 28, 'quarter'), (29, 33, 'year'), (35, 42, 'ticker')]}),
    ("What is the net income of Microsoft in Q3 2019?", {'entities': [(24, 28, 'quarter'), (29, 33, 'year'), (35, 42, 'ticker')]}),
    ("What is the revenue trend of Nvidia from Q1 2016 to Q4 2020?", {'entities': [(37, 41, 'year'), (43, 44, 'quarter'), (56, 62, 'year'), (63, 64, 'quarter'), (76, 82, 'ticker')]}),
    ("What is the outlook for Nvidia's gaming revenue in Q2 2021?", {'entities': [(37, 41, 'year'), (42, 44, 'quarter'), (50, 56, 'ticker')]}),
]

[kadarakos]

ValueError: [E989] `nlp.update()` was called with two positional arguments. This may be due to a backwards-incompatible change to the format of the training data in spaCy 3.0 onwards. The 'update' function should now be called with a batch of Example objects, instead of `(text, annotation)` tuples. 

Your TRAIN_DATA is not an Iterable[Example]. Please checkout the docs: https://spacy.io/api/language#update.

[vrunm]

@kadarakos Coming back to the original question what I meant originally to ask was I wanted the spacy tokenizer part only during training and during inference I just need the NER part. Is there any way to do that? Also I did go through the docs thoroughly and have got the training part to work.
Thanks

[kadarakos]

Hey vrunn,

It seems to me that you would benefit from familiarizing yourself a bit more with spaCy first before asking specific questions here. We made available for free a full course on spaCy: https://course.spacy.io/en/. We also have a guide to training pipelines that I think would be very informative for you: https://spacy.io/usage/training.

[vrunm]

@kadarakos Thanks I read the docs in detail but could not find any specific code where you first use a tokenizer for the text and then disable it during inference and only use the NER part. Also I do understand the tokenizer is necessary for the NER pipeline. So I believe you can just disable all the other components and use the NER part during training? But it seems that approach does not work.