Provide more explanations of the models trained in the project.

[Precipitation2027]

The work is wonderful. It offers new perspectives for research in this field, which is truly inspiring. During my reading, I pay particular attention to the part about the three models trained in the project. Notably, different models are labeled with "cot_ratio" and "ratio". Could you please provide a detailed explanation regarding these? I look forward to your reply.

[Robot2050]

Thank you very much for your attention to our MoM project and for asking such detailed question!

The cot_ratio and ratio you mentioned are key parameters we use to distinguish the composition of training data when training MemReaders of different scales. As you have noticed, this is closely related to the CoM strategy mentioned in our paper.

1. cot_ratio (applied to 1.5B and 3B models): This parameter represents the inclusion, in the training data, of reasoning paths generated through the CoM strategy. We not only provide the SLM with the original documents and the final document memories as supervised data but also utilize high-quality expert thought paths derived by a guiding model. Our goal is to instill deeper, human-like reading and reasoning abilities with the help of CoM, rather than merely imitating the input-to-output mapping.

2. ratio (applied to the 7B model): This parameter indicates that the model does not use CoM during training. Given the stronger capabilities of the 7B model, we train the model to directly learn to generate structured document memories from original documents. This helps reduce the computational consumption of LLMs.

Thank you again for your question. If you have any further inquiries, feel free to communicate with us at any time.