gradnorm==0

[EthenCheng]

When I was training, I found that gradnorm was always 0, and the loss kept jumping between 8.3-8.5 and could not be reduced. My batchsize was set to 256, eight images of 4090. Why did this happen?

[raytrun]

This issue might be due to a batch size that's too small — we use a batch size of 4096. If you're using a batch size of 512, you may need to adjust the hyperparameters accordingly, such as the learning rate.