Gradient accumulation code implement #43
Description
Can I ask how do you implement gradient accumulation code in deit model training? Since I can not find other resources on the internet doing gradient accumulation on deit training, but I am interested in doing this in order to training deit from scratch. Thanks!