[Training Log 2022-10-16] We skipped some training data last night. #205

zh-zheng announced in Training Logs 训练日志

zh-zheng
Oct 16, 2022
Maintainer

CPM-Live Training Log (October, 16)

Time: October, 16 2022 19:00

Recorder: @zh-zheng

Loss

Begin: 2.44
End: 2.71

Completed Data

$\approx$ 1,274.11GB

Average Grad Norm

2.26

Progress

3.0%

Comment

We observed that the training loss was still going up, and we tried to solve this problem by skipping some training data at 23:00 last night.

Replies: 0 comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment