Extract probabilities for custom ner model

[kalyanm2305]

I trained a custom entity recognizer to extract products from a webpage and it does well.
Now i wouId want to extract the probabilities from the softmax inorder to understand the confidence of the model.
Is there a way to extract probability for the extract entities.
I tried using the beam search code that works on spacy v2 & it failed.
CouId you please suggest a way forward

import sys
from collections import defaultdict

beam_width = 16
beam_density = 0.0001 
nlp =  spacy.load("./output/model-best")

with nlp.disable_pipes('ner'):
    docs = list(nlp.pipe(texts))
    print(docs)
beams = nlp.entity.beam_parse(docs, beam_width=beam_width, beam_density=beam_density)

for doc, beam in zip(docs, beams):
    entity_scores = defaultdict(float)
    for score, ents in nlp.entity.moves.get_beam_parses(beam):
        for start, end, label in ents:
            entity_scores[(start, end, label)] += score

print ('Entities and scores (detected with beam search)')
for key in entity_scores:
    start, end, label = key
    score = entity_scores[key]
    if ( score > 0.01):
        print ('Label: {}, Text: {}, Score: {}'.format(label, doc[start:end], score))

[polm]

This is not supported at the moment, see #5917. The upcoming SpanCategorizer linked in that issue should provide this functionality.

[XBeg9]

Do you suggest using spancat instead of NER? How about EntityLinking, is there a plan to support it for spancat?

[polm]

The NER model has stronger constraints which can help it make better predictions and lead to higher accuracy. But if you need any of the features that are only in the spancat, like overlapping spans or confidence scores, then you should use the spancat.

We haven't been working on any special support for spancat in the EntityLinker. The easiest way to combine them would be to just copy spans to entities with a simple component in between them, though that wouldn't handle overlapping spans.

[XBeg9]

@polm thanks! that's very helpful