Training Log 2022-06-30

[zh-zheng]

CPM-Live Training Log (June, 30)

Time: June, 30 2022 16:00

Recorder: @zh-zheng

Loss

• Begin: 2.59
• End: 2.65

Completed Data

• $\approx$ 379.34GB

Average Grad Norm

• 0.91

Progress

• 40.73%

Comment

The training progress has exceeded 40%, meaning that our model has seen about 80 billion tokens! Actually, an average human speaks 860,341,500 words in the entire life[1]. That is to say, the model has heard 23 people talk to him for a lifetime🤣.

[1] Brandreth, Gyles Daubeney. The joy of lex: How to have fun with 860,341,500 words. William Morrow & Company, 1980.