Add logits_to_keep to many older CausalLM models
#41335
Conversation
|
I was wondering if I could also add this to the many seq2seq models that also inherit from |
|
[For maintainers] Suggested jobs to run (before merge) run-slow: bart, bert, bert_generation, big_bird, bigbird_pegasus, biogpt, blenderbot, blenderbot_small, blip, bloom, camembert, chameleon, codegen, cpmant, ctrl, data2vec |
There was a problem hiding this comment.
Mostly looks good, with one comment that applies to a couple of models!
This is a fairly big change that standardizes a lot of older models with the modern API so cc core maintainers @ArthurZucker @Cyrilvallez
| vocab_size=self.config.vocab_size, | ||
| **kwargs, | ||
| ) | ||
| loss = self.loss_function(logits=logits, labels=labels, vocab_size=self.config.vocab_size, **kwargs) |
There was a problem hiding this comment.
Some of these cases slightly change other behaviour (e.g. not moving labels to logits.device). Have you checked that this is equivalent?
What does this PR do?
Fixes #40984
Adds
logits_to_keepto many (older)ForCausalLMmodels that inherit fromGenerationMixin.Also consistently renames to
lossandlogits, and removes some code for float casting and mapping labels to the logits' device for models where that is already handled by the loss function (e.g.gpt_neo).Before submitting
Pull Request section?
to it if that's the case.
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR. @Rocketknight1