Defaulting detect_anomaly to True (in Trainer) #20708
Description
Description & Motivation
Currently, if detect_anomaly is set to False (default), if losses become NaN or Inf, training continues.
Defaulting detect_anomaly to True (in Trainer) will raise an RTE if losses become NaN or Inf as further training doesn't make sense in this case
Pitch
Will save a lot of unnecessary debug and training time (fail fast)
Alternatives
No response
Additional context
No response