About optimization #13
Description
Hello. Are you quite sure that history of optimizer (e.g. moments) should be zeroed at the beginning of each epoch?
{{ message }}
Hello. Are you quite sure that history of optimizer (e.g. moments) should be zeroed at the beginning of each epoch?