Problem implementing custom pipeline component as independent class

[megamattc]

Hello,

I am trying to implement a custom pipeline component as a class FSTChecker within my pipeline which sets morphological features on tokens according to an internal lookup table. I am using a custom Language class Akkadian which contains the code defining the factory to create the component, and I have put the Python file defining the class (fst_checker.py) within the folder defining the language. Current the pipeline component is not trainable but I want to include it in the training cycle to observe the quantitative performance under evaluate, among other reasons.

When I try to train my model I get the following error:

File "/Users/matthewong/miniforge3/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Users/matthewong/miniforge3/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/Users/matthewong/miniforge3/lib/python3.10/site-packages/spacy/__main__.py", line 4, in <module>
    setup_cli()
  File "/Users/matthewong/miniforge3/lib/python3.10/site-packages/spacy/cli/_util.py", line 71, in setup_cli
    command(prog_name=COMMAND)
  File "/Users/matthewong/miniforge3/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/matthewong/miniforge3/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/matthewong/miniforge3/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/matthewong/miniforge3/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/matthewong/miniforge3/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/matthewong/miniforge3/lib/python3.10/site-packages/typer/main.py", line 532, in wrapper
    return callback(**use_params)  # type: ignore
  File "/Users/matthewong/miniforge3/lib/python3.10/site-packages/spacy/cli/train.py", line 45, in train_cli
    train(config_path, output_path, use_gpu=use_gpu, overrides=overrides)
  File "/Users/matthewong/miniforge3/lib/python3.10/site-packages/spacy/cli/train.py", line 75, in train
    train_nlp(nlp, output_path, use_gpu=use_gpu, stdout=sys.stdout, stderr=sys.stderr)
  File "/Users/matthewong/miniforge3/lib/python3.10/site-packages/spacy/training/loop.py", line 122, in train
    raise e
  File "/Users/matthewong/miniforge3/lib/python3.10/site-packages/spacy/training/loop.py", line 105, in train
    for batch, info, is_best_checkpoint in training_step_iterator:
  File "/Users/matthewong/miniforge3/lib/python3.10/site-packages/spacy/training/loop.py", line 226, in train_while_improving
    score, other_scores = evaluate()
  File "/Users/matthewong/miniforge3/lib/python3.10/site-packages/spacy/training/loop.py", line 281, in evaluate
    scores = nlp.evaluate(dev_corpus(nlp))
  File "/Users/matthewong/miniforge3/lib/python3.10/site-packages/spacy/language.py", line 1431, in evaluate
    eg.predicted = doc
  File "spacy/training/example.pyx", line 86, in spacy.training.example.Example.predicted.__set__
TypeError: Cannot convert FSTChecker to spacy.tokens.doc.Doc

I don't understand why this is happening. Relevant sections of my code which define the component include:

From the __init__.py defining the Akkadian language class:

@Akkadian.factory(
    "fst_checker",
    requires=["token.text","token.pos"]
)
def make_fst_checker(nlp: Language, name: str):
    return FSTChecker

and sections of the code defining FSTChecker:

class FSTChecker:

    file = open("eval_dict_general.json", 'r')
    lookup_dict = json.load(file)

    dialect = "oldbab"

    def __init__(self, name: str = "fst_checker") -> None:


        self.name = name
     

    def __call__(self, doc: Doc) -> Doc:
        result_doc = self.apply_fst_checker(doc)
        return result_doc


        # Define a function that compares a parse dictionary to a list of possible parse dictionaries of the same form and returns
        # the one that is 'closest'

    def apply_fst_checker(self, doc: Doc)-> Doc:

The function apply_fst_checker(self, doc: Doc) is the key method as it takes in a Doc object and modifies the tokens in it according to the internal lookup table. My intent was that the pipeline component sets the token attributes when __call__ is executed, as that returns a Doc. Should I be doing this some other way?

---

UPDATE

I recently changed my code slightly and was able to integrate by component into the pipeline. The changes were:

For fst_checker.py (which defines the component class):

    def __init__(self, nlp, name: str = "fst_checker") -> None:


        self.name = name
        self.nlp = nlp

For the __init__.py file defining my Akkadian language class:

@Akkadian.factory(
    "fst_checker",
    requires=["token.text","token.pos"]
)
def make_fst_checker(nlp: Language, name: str):
    return FSTChecker(nlp, name)

I am still curious if anyone knows why the error above was caused by the lack of including nlp as an explicit initialization argument, since I do nothing with it in the class. Moreover, will including my component at the very end of my pipeline change the way spacy compares predicted values versus reference values in training data or during evaluation? I don't think so, but I did not code the predict function etc. in my component that is found in many of the standard components.

[thomashacker]

Hello,

Yes, you're correct, you need to add the nlp argument to the factory initialization to make it work; it's simply the architecture template to adhere to when creating a new factory. If you want to evaluate your custom component you'd also need to write your own scoring function which compares the reference values with the predicted values. Every component should be evaluated individually, the order only matters if components depend on annotations of other components.

[megamattc]

Ok. Thank you!