Garbled code issue in the D2S stage

[DiracSeas]

Hello, I encountered the following problem while fine-tuning the D2S stage. No matter what I ask the model, it always talks nonsense, but there is no such problem in the D2D stage. I processed the same data format in the D2D and D2S stages, and downloaded the corresponding dataset according to the guidance of train_ eval.md, and the experimental parameters are the same as the default parameters of the project How can I solve this problem？

The following is the result of running cli_inference. sh:

Thank you！

[shufangxun]

I think the tokenizer might be used incorrectly here

[DiracSeas]

I think the tokenizer might be used incorrectly here

Hello, I carefully checked the tokenizer settings and found that they are the same as the D2D stage settings, ruling out any issues with the tokenizer.
In addition, I found that after setting moefinetune=true, there were multiple code issues that made it impossible to reproduce.
May I ask if your default training setting is moefinetune=false? This will result in MOE's routing not being updated and unable to obtain correct results.
Could you please answer my question above? Thank you!