How to install transformer from source code

[vitdainhan]

Can someone give me a pointer to how to do this? I already have spaCy installed from source code, but I can't figure out how to install the transformers such that I can work with the installed spaCy the same way had I installed them with pip.

TIA.

[polm]

Are you referring to spacy-transformers? You should be able to just do the same thing as any library:

git clone https://blablah/spacy-transformers.git
pip install ./spacy-transformers/

If you mean HuggingFace's code then they have instructions.

How did you install spaCy from source?

Can you explain why you're installing from source rather than via pip so we can understand your situation better?

[vitdainhan]

Thanks for the prompt response.
I want to install spaCy from source because I want to access the code. As mentioned in #9063, I want to retrieve the best score output from the greedy parser for each token to see how high is the best. I want to see if I can use that as a 'confidence score' in my decision to accept such best action. Also, I want to see if I can use the beam parser code that is still in the code base to compute a confidence score. Hope that makes sense.

[polm]

Ah OK, that makes sense. I had forgotten about the other thread.

I would recommend you get everything you want with the parser working on its own before getting transformers involved, that should make it easier to keep track of what's going on.