Relying on local knowledge

[tianchiguaixia]

When doing NER. How to share local knowledge with the model to improve effectiveness

[rmitsch]

Converting this to a discussion.

[rmitsch]

Hi @tianchiguaixia! Can you elaborate on which kind of local knowledge you would like to share and how you think this would improve NER accuracy?

[tianchiguaixia]

[rmitsch]

Please state in text what this local knowledge looks like and how you think it can improve NER performance.

[tianchiguaixia]

It is a JSON structure with corresponding key value values. But there are no start and end.

[rmitsch]

I'm not sure how this JSON structure relates to the entities you want to recognize, but you could include some data as examples in the prompt.

Beyond that there is no possibility to directly incorporate this information into the LLM task. I recommend looking at the other components in spaCy to see how this can be achieve - you could use e. g. a rule-based matcher or train your own NER model.

[tianchiguaixia]

Your suggestion is very useful to me