Custom model architecture with non-text inputs

[1danjordan]

I have a use case whereby I want to classify a piece of text and I have some relevant metadata (numeric and categorical) that I would like to incorporate into my model. We already use spaCy for rules based NER and text classification, which has made building our system a joy! I want to implement a custom model that allows me to include text and metadata while keeping it within the Language object. Are there any examples of this being done before? Ultimately, I need to be able to update doc.cats or a custom attribute like doc._.cats. We have our training pipeline in a spaCy project already, so I am looking for a solution played nice with that as well. I've been searching high and low for examples but haven't managed to find any.

[adrianeboyd]

This is easier in v3.2 now that you can pass in Doc objects to nlp and nlp.pipe. The simplest case would look like this:

doc = nlp.make_doc(text)
doc._.ext1 = value1
doc._.ext2 = value2
doc = nlp(doc)

Where nlp might have a custom component that uses the custom extension values.

If you want to train components using custom extensions as features, you can include custom extension values in the tok2vec component by using a custom Tok2Vec architecture. An example from a user is here: #6527 (comment). In this case, the DocBin files (.spacy) you provide as training data would typically already include these features, unless you have another custom component that adds them, in which case you'd want to put that component in annotating_components while training.

[1danjordan]

@adrianeboyd thanks so much for this - I really really appreciate it! 😄