Loading and using a HuggingFace model in code

[bpben]

Wasn't sure if this was worth attaching to #7027 or discussing here: I was having issues trying to load a non-default Transformer model in code and manipulate the attributes (e.g. use .similarity). The docs' examples aren't functional as far as I can tell. The current example (https://spacy.io/api/transformer) doesn't work:

doc = nlp("This is a sentence.")
trf = nlp.add_pipe("transformer")
# This usually happens under the hood
processed = transformer(doc)

An example based on the workaround (my edit)

trf = nlp.add_pipe("transformer")
nlp.initialize()
nlp("This is a sentence.")

Does work. (Though it seems to behave strangely compared to when you just load something like en_core_web_trf)

Would it make sense to update the docs? Is there something I'm missing here? This all works like a dream using the config+CLI workflow, but I'm specifically doing some experimentation in code, so it'd be easier if I can just load up a HF model.

[adrianeboyd]

You can assemble a transformer pipeline with spacy assemble just from a config, or programmatically it's equivalent to what you posted:

nlp = spacy.blank("en")
nlp.add_pipe("transformer")  # set your transformer config in kwarg `config`
nlp.initialize()

Then doc._.trf_data should contain the transformer output similar to the trf pipelines, which you can use with user hooks for things like similarity. (Although at the doc level in many cases other alternatives like sentence-transformers can be better.)

We can think about how to update the examples here, because I can see how it's a bit confusing. It's intended to be more of a sketch to show how that individual method is used internally rather than a complete example, which would always require initialization.

[bpben]

Didn't know about spacy assemble, thanks! I guess I was looking for an in-code implementation.

doc._.trf_data seems like it needs a Listener layer to be aligned to play nice with similarity, am I understanding that correctly?

I'm happy to help out with updating the examples, if that's useful.

[adrianeboyd]

I think these kinds of examples with vector hooks should work fine with just a transformer component in the pipeline:

https://applied-language-technology.mooc.fi/html/notebooks/part_iii/05_embeddings_continued.html#contextual-word-embeddings-from-transformers

If there aren't any user hooks for vectors, the similarity method uses the static vectors by default, so for the default pipeline config from spacy.blank you get empty vectors for everything because no vectors are loaded by default.

Again, I don't think the results are going to be very good for the .similarity method using the transformer output, but you can certainly try it out to see what how it works for your task.

[bpben]

Nice, thanks for the example. It seems like using Tok2VecTransformer also works, or at least the .vector is aligned appropriately.

[polm]

There is an FAQ post about the right way to do this now: #10768