How to convert transformers output to spacy Doc objects

[emiltj]

Hi all,

I have been wanting to work with the NER predictions of a model, using spaCy. However the predictions are in the format output by the transformers library - i.e. a list of dictionaries with information on the texts.

It looks like this:

tokenizer = AutoTokenizer.from_pretrained("tner/roberta-large-ontonotes5")
model = AutoModelForTokenClassification.from_pretrained("tner/roberta-large-ontonotes5")
classifier = pipeline("ner", model=model, tokenizer=tokenizer)
classifier("Languages Spanish, English and Danish.")

[{'entity': 'B-LANGUAGE', 'score': 0.99868673, 'index': 3, 'word': 'ĠSpanish', 'start': 10, 'end': 17}, {'entity': 'B-LANGUAGE', 'score': 0.9987031, 'index': 5, 'word': 'ĠEnglish', 'start': 19, 'end': 26}, {'entity': 'B-LANGUAGE', 'score': 0.9984553, 'index': 7, 'word': 'ĠDanish', 'start': 31, 'end': 37}]

My question is: Is it possible to create Doc objects using this output?

[rmitsch]

Hi @emiltj, spaCy-wrap should be able to help you with that as well (context for posterity).