What is "loss" when it comes to tok2vec

[bpben]

I couldn't find quite a clear answer on this - an NER pipeline with a tok2vec component will report tok2vec loss, but I'm not clear what the objective for tok2vec is. Is it the loss from the NER objective with respect to the tok2vec component? Or is there something separate being optimized there? The FAQ describes loss as the "work remaining for the optimizer", but this seems like a way of describing loss more generally. The docs describe "tok2vec predictions", but what is tok2vec predicting?

Thanks!

[ljvmiranda921]

Hi @bpben ,

During training, the tok2vec loss is just from other components (it's not separate). You can find more information in the model architectures documentation.

[bpben]

Thanks, Lj. Not sure I understand where I should be looking in the docs for this - your link brings me to docs that explain the architectures, but not how they calculate the loss they return in the training run. Tok2vec loss is different from NER loss in my training run and I was understanding that as the loss on the NER objective attributed to the tok2vec component, but I wasn't sure.