Training Log 2022-06-21

[zh-zheng]

CPM-Live Training Log (June, 21)

Time: June, 21 2022 16:00

Recorder: @zh-zheng

Loss

• Begin: 2.46
• End: 2.39

Completed Data

• $\approx$ 240.33GB

Average Grad Norm

• 0.81

Progress

• 25.81%

Comment

Today is a peaceful day! I notice a paper published by OpenAI, in which researchers use diffs and commit messages from GitHub to teach a large language model how to modify the code. I just wonder whether it is possible to teach our model to write some comments, giving it the training logs of two consecutive days. What do you think?😁

[jayzzhou-thu]

I think there is no problem finetuning a model based on CPM-Live to write comments. But the input should be more, such as the machine shutdown information and running logs. 😆

[zibuyu]

how about making this as a proposal for the next phase of cpm-live. :)

[jayzzhou-thu]

Cool, it may be interesting if there is a "model comment" section in the future.