How is the loss function computed?

[puraminy]

Could you please guide me how loss function for T5 is computed? I mean it is a seq2seq model. Suppose it must map a sequence of X tokens to Y tokens, but it generates Z tokens. How Y and Z are compared to calculate loss?

[adarob]

Cross-entropy loss.

[puraminy]

Thanks, can one use metrics like BLUE or ROUGE?

Also, I have another minor questions about T5 which could be obvious!
Does the embedding of words change during supervised fine-tuning of the model? I mean is it just the change in embedding of words what the model learns from examples, or the learning is stored in another part of the model?