Generating label data for custom component

[thatbudakguy]

I'm working on a very basic trainable custom component that assigns a single custom attribute. It's intended to be almost identical to the built-in Tagger (for right now, anyway). At the moment, I'm seeing two problems:

1. When using spacy init labels, no label data is found for the component.
2. When training, the loss goes up but the other metrics stay at zero.

I imagine this is the same problem and it's related to correctly informing spaCy that the component annotates/initializes from a custom extension attribute, but I'm not sure where I'm going wrong.

The script I have that generates the training data seems to be working the way I want, and I can see that it produces DocBins with the right content that I then save with store_user_data=True, so it doesn't seem like it's a problem with the training data.

The repository with the spaCy project is here: https://github.com/direct-phonology/och-g2p; the custom component is defined in scripts/phonologizer.py.

Any help/advice is appreciated!

[adrianeboyd]

If spacy init labels doesn't find labels, it usually means that something is wrong with the training data, or possibly with how initialize reads the labels from the training data.

In get_loss, you need to use the alignment in the Example to allow for differences between the reference tokenization and the predicted tokenization.

To take advantage of the default serialization in TrainablePipe, you want to store the labels in self.cfg as in Tagger. Otherwise the labels aren't saved when you use to_disk or to_bytes.

[thatbudakguy]

thanks for responding! i think i initially didn't use Example.get_aligned() because I didn't know how to retrieve a custom extension attribute via the alignment:

[E983] Invalid key(s) for 'IDS': '_'. Available keys: ['', 'IS_ALPHA', 'IS_ASCII', 'IS_DIGIT', 'IS_LOWER', 'IS_PUNCT', 'IS_SPACE', 'IS_TITLE', 'IS_UPPER', 'LIKE_URL', 'LIKE_NUM', 'LIKE_EMAIL', 'IS_STOP', 'IS_OOV_DEPRECATED', 'IS_BRACKET', 'IS_QUOTE', 'IS_LEFT_PUNCT', 'IS_RIGHT_PUNCT', 'IS_CURRENCY', 'ID', 'ORTH', 'LOWER', 'NORM', 'SHAPE', 'PREFIX', 'SUFFIX', 'LENGTH', 'LEMMA', 'POS', 'TAG', 'DEP', 'ENT_IOB', 'ENT_TYPE', 'ENT_ID', 'ENT_KB_ID', 'HEAD', 'SENT_START', 'SPACY', 'LANG', 'MORPH', 'IDX']

what's the right way to do this?

[adrianeboyd]

You can't use Example.get_aligned directly, but you can copy it to make a custom version that replaces to_array.

Unless you have a huge training corpus, I'd recommend skipping the label init in [initialize], since that may be masking bugs or always initializing it with an incorrect set of stored labels. Reading the labels from the corpus is more flexible during development, and if it turns out that this step is taking too long with your whole training corpus, you can add the initialize label block back in the future.

[thatbudakguy]

thanks for your help! I was able to implement a working version of get_aligned() for my custom attribute.