Updating NER trained model?

[MagiCsito]

Hello all!

I trained NER model to detect malware names from news (urls) and it works pretty good, now I would like to train the same model to detect Threat Actors too but the problem is that I need to do 2 trains instead of one with both labels because I go Out of memory (nvidia 3090).

So when I train the NER threat actor model using the NER malware names model, it looks like it forget how to recognize malware names. Is there a way to update the "malware names" model with the "threat actor" model without lose the first one?

As solution I can use 1 model per NER but i would like to avoid it.

PD: I am using the commands from spacy webpage to create the config and to train it.

Thank you all! And sorry about my english!

Here is the code to generate the label where it load the previous trained model (nlp = spacy.load("output/model-last"):

import spacy
import io
from spacy.matcher import Matcher
from spacy.tokens import Span, DocBin
from spacy.matcher import PhraseMatcher
from newsplease import NewsPlease

def get_article(url):
    article = NewsPlease.from_url(url)
    return article.maintext


mi = io.open("ransom_rf.txt", "r", encoding="utf-8")
lines = mi.readlines()
nm = []

for line in lines:
    line = line.replace("\n","")
    nm.append(line)
print(nm)
mi.close()
apt = []
apf = io.open("apts.txt", "r", encoding="utf-8")
lines = apf.readlines()
for line in lines:
    line = line.replace("\n", "")
    apt.append(line)
f = io.open("urls.txt", "r", encoding="utf-8")
lines = f.readlines()
docs = []
doc_bin = DocBin(docs=docs)
n = 0
nlp = spacy.load("output/model-last")
for line in lines:
    n += 1
    line = line.replace("\n","")
    try:
        t = get_article(line)
        t = t.replace("\n","")
        t = t.lower()
        TEXTS = t
    except:
        continue

    try:
        print(n)


        matcher = PhraseMatcher(nlp.vocab, attr='LOWER')
        """nm = ['Conti', 'killnet']"""
        """mal = [nlp.make_doc(text) for text in nm]"""
        apu = mal = [nlp.make_doc(text) for text in apt]
        """pattern1 = [{"LOWER": "conti"}]"""
        """matcher.add("MALWARE", None, *mal)"""
        matcher.add("TA", None, *apu)
        docs = []
        doc = nlp(TEXTS)
        matches = matcher(doc)
        spans = [Span(doc, start, end, label=match_id) for match_id, start, end in matches]
        if spans:
            print(spans)
        doc.ents = spans
        docs.append(doc)
        doc_bin.add(doc)

        """    for doc in nlp.pipe(TEXTS):
            matches = matcher(doc)
            spans = [Span(doc, start, end, label=match_id) for match_id, start, end in matches]
            print(spans)
            doc.ents = spans
            docs.append(doc)
        doc_bin.add(doc)"""
    except:
        continue
f.close()
doc_bin.to_disk("./trainta.spacy")

Commands used for training:

python -m spacy init fill-config base_config.cfg config.cfg
python -m spacy train config.cfg --output ./output --paths.train ./trainta.spacy --paths.dev ./trainta.spacy

BTW: I know that its "catastrophic forgetting" issue but how can I do it if I can't train two labels at the same time because I dont have enough hardware?

[adrianeboyd]

Take a look at the suggestions under "I'm getting Out of Memory errors" in this FAQ: #8226

[MagiCsito]

Thank you so much ;)