如何让模型学会100内的加减法？ #620
Description
已经在104M的模型下sft了很多轮，数据格式如下：
{"conversations": [{"role": "user", "content": "81 + 14="}, {"role": "assistant", "content": "95"}]}
{"conversations": [{"role": "user", "content": "94 + 35="}, {"role": "assistant", "content": "129"}]}
{"conversations": [{"role": "user", "content": "28 + 17="}, {"role": "assistant", "content": "45"}]}
{"conversations": [{"role": "user", "content": "86 + 94="}, {"role": "assistant", "content": "180"}]}
{"conversations": [{"role": "user", "content": "75 + 54="}, {"role": "assistant", "content": "129"}]}
{"conversations": [{"role": "user", "content": "3 + 11="}, {"role": "assistant", "content": "14"}]}
{"conversations": [{"role": "user", "content": "29 + 64="}, {"role": "assistant", "content": "93"}]}
{"conversations": [{"role": "user", "content": "71 - 25="}, {"role": "assistant", "content": "46"}]}
{"conversations": [{"role": "user", "content": "28 - 57="}, {"role": "assistant", "content": "-29"}]}
{"conversations": [{"role": "user", "content": "0 + 97="}, {"role": "assistant", "content": "97"}]}
{"conversations": [{"role": "user", "content": "89 - 54="}, {"role": "assistant", "content": "35"}]}
{"conversations": [{"role": "user", "content": "35 + 19="}, {"role": "assistant", "content": "54"}]}
{"conversations": [{"role": "user", "content": "97 + 43="}, {"role": "assistant", "content": "140"}]}
{"conversations": [{"role": "user", "content": "11 + 48="}, {"role": "assistant", "content": "59"}]}
{"conversations": [{"role": "user", "content": "45 - 44="}, {"role": "assistant", "content": "1"}]}
{"conversations": [{"role": "user", "content": "5 - 93="}, {"role": "assistant", "content": "-88"}]}
{"conversations": [{"role": "user", "content": "68 - 15="}, {"role": "assistant", "content": "53"}]}
{"conversations": [{"role": "user", "content": "10 - 70="}, {"role": "assistant", "content": "-60"}]}
{"conversations": [{"role": "user", "content": "80 - 79="}, {"role": "assistant", "content": "1"}]}
{"conversations": [{"role": "user", "content": "73 + 24="}, {"role": "assistant", "content": "97"}]}
。。。。。
大概有10W条
效果依然不好