Entity Linker Takes long time to initialize (transformer) #12364
Replies: 3 comments 3 replies
-
|
A little more scientifically, I've added a print statement in spacy code to check the words/sec of the evaluation loop. The first time it runs speed is: And then while training, the speed is dramatically faster: Here is the code I added to print speed around L1415 of spacy/language.py |
Beta Was this translation helpful? Give feedback.
-
|
Hi Jerry, I know you were asking about issues with the I still wonder how big your KB is (vector length 300? number of entities/aliases? Size on disk), because here in this config I also notice that you use a custom candidate generator (
Is it in fact the case that it takes long to start, or does one iteration take long? The difference is this: if it takes long to start, then the following iterations (printed lined on the console) would be faster. But if it's the case that every iteration takes long, then it'll be a while before you see the first line of results on the console, but it will take an equally long amount for the next line. It would be good to understand which of the two is happening to better identify likely culprits. Have you been able to run this long enough to get at least one or two lines of results printed, and can you share those? One other idea is whether you can try setting |
Beta Was this translation helpful? Give feedback.
-
|
@svlandeg Regarding #12324 One of the things that I've done is rather than use text from 3-10 page PDFs for each On the topic of this thread: Regarding my custom candidate generator, I don't believe it's adding significant latency due to the characteristics of my KB as well as if I go back to using the default candidate generator I still observe the slow training start behavior. I've added code below that is a sketch of my custom candidate generator. The reason I use it is because I have an entity id with alias like In the folder of my KB, this is the size of the serialized files: Script I use to encode entities: nlp = spacy.load("en_core_web_trf")
def sim(a, b):
cos_sim = np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))
return cos_sim
def calculate_vector_for_doc(doc: Doc):
vec_idx_splits = np.cumsum(doc._.trf_data.align.lengths)
# get transformer vectors and reshape them into one large continous tensor
trf_vecs = doc._.trf_data.tensors[0].reshape(-1, 768)
# calculate mapping groups from spacy tokens to transformer vector indices
vec_idxs = np.split(doc._.trf_data.align.dataXd, vec_idx_splits)
# take sum of mapped transformer vector indices for spacy vectors
vecs = np.stack([trf_vecs[idx].sum(0) for idx in vec_idxs[:-1]])
return vecs[0]
def main(save_kb_loc: Path):
umls_mds_df = pd.read_csv("./assets_vcs/umls_mds.csv")
umls_cui_dropdup = umls_mds_df[["cui", "description"]].drop_duplicates()
kb = InMemoryLookupKB(vocab=nlp.vocab, entity_vector_length=768)
for row in umls_cui_dropdup.itertuples():
desc_doc = nlp(row.description)
desc_vec = calculate_vector_for_doc(desc_doc)
kb.add_entity(row.cui, entity_vector=desc_vec, freq=1)
print(f"Added entity: {row.cui} with description: {row.description}")
kb.to_disk(save_kb_loc)Sketch of my script for fuzzy candidate matching: from rapidfuzz import fuzz, process
blank_pipeline = spacy.blank("en")
def get_fuzzy_candidates(kb, span):
orig_candidates = kb.get_candidates(span)
if orig_candidates:
return orig_candidates
candidate = process.extractOne(span.text, kb.get_alias_strings(), scorer=fuzz.ratio, score_cutoff=88)
return kb.get_candidates(blank_pipeline(candidate)) |
Beta Was this translation helpful? Give feedback.
-
The first set of evaluation metrics takes a long time to print. Subsequent evaluations are much faster. I will re-run the beginning of my training loop and print out the results of evaluation. There is an initial evaluation step that I believe happens and prints out poor results before any SGD loops have happened. This is the one I believe is problematic. It takes about 25 minutes for it to complete before the progress par starts rendering. Once the progress bar renders and completes the number of steps before evaluation, the actual evaluation runs much quicker. The subsequent evaluation step printing out "speed" on the github line below was ~4,600 vs. the ~36 for the first evaluation step. Does this clarify your question? https://github.com/explosion/spaCy/blob/master/spacy/language.py#L1420 |
Beta Was this translation helpful? Give feedback.
-
|
Setting Here is a printout of the beginning of the operation that is very slow. My custom candidate generator is printing our suggestions. Once this initial portion that takes ~25 minutes is complete, then it prints the first line of evaluation results. Next the model starts iterating. Then once it hits Further context I'm using distillbert-base-uncased on a GPU. |
Beta Was this translation helpful? Give feedback.
-
|
Hey @jkgenser, thanks for all the details, that's very helpful!
You're right that this is unexpected and unideal. If you could share your project with us, we could look into it more - you can find my email address at https://explosion.ai/about and of course we'd keep that entirely confidential and would only use the data/code for debugging. With respect to the long initialization time, I haven't yet been able to identify a likely culprit. The main thing that happens before printing any train lines is assembling the dataset, which is why I asked whether setting From the thread, I understand that your custom candidate generator is likely not to blame, because you run into the same issues with the built-in one. Additionally, if you use a One hypothesis is that the EL needs to be further optimized for GPU usage, in particular by avoiding indexing into a GPU-allocated array during a loop. When you ran the Again, if you could share your project & data so I can rerun it and do some more experiments on both CPU & GPU, I'd love to dig into this further. Thanks for your patience! |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Hello -
I am running an entity linker model with the following pipeline:
transformer --> sentencizer --> ner --> entity_linker
When I start training, for some reason it takes a very very very long time to even start training. The operation that spacy is ruining while I start training seems to be related to entity linking since it's printing out some print statements I have in my custom candidate generator. I'm using the same candidate generator on tok2vec models without this same hold up on just starting training so I'm wondering if someone can help point me in the direction on why it takes so long just to start?
Do I need to change something about my initial data? Below I've copied the output that shows the print outs during the initialization step. I've timed it and it takes 25 minutes before training even starts. Once training starts, I get 2-5 its/sec which is reasonable and expected given other transformers I've trained with spacy.
Any recommendation or questions to help diagnose why the why entity linking init takes so long and how to improve it would be much appreciated.
One note is that I am using 1/12 of my dataset as an evaluation set. I see the span candidate print outs during evaluation since I know that is when these candidate fns are called. However it's curious to me that it takes much more than the evaluation time * 12 in order to get the model initialized. i.e. It takes about 30 seconds to run an evaluation on 1/12 of my dataset but like I said earlier, about 25 minutes before the training loop actually starts.
Config file:
Beta Was this translation helpful? Give feedback.
All reactions