Poor Chinese Performance in EAGLE3 Model Due to Lack of Chinese Training Data

[lmyybh]

The EAGLE3 model provided by SpecBundle is very good, but it performs poorly in Chinese-language scenarios—likely because the training dataset lacks Chinese data. Consider adding Chinese datasets to the training set in future updates?

[FrankLeeeee]

Hi, yes the training dataset does not contain Chinese, so this is expected. We so far don't have plan to include Chinese yet unless we have more compute. However, I think the released weights are good starting points for finetuning for Chinese tasks.

You can try use this dataset: https://huggingface.co/datasets/opencsg/smoltalk-chinese

[johnsbucket]

Hi, FrankLeeeee,

For models that demonstrate outstanding performance in Chinese understanding and generation, removing Chinese support would significantly reduce the user base. Therefore, if feasible in future version iterations, please consider the use of bilingual Chinese-English mixed training data (e.g., https://huggingface.co/datasets/zhaode/EagleChat). @FrankLeeeee

[FrankLeeeee]

Sure, we tried EAGLEChat, but the performance is not as good as perfectblend, thus we suggest smoltalk-chinese. If we have sufficient compute in the future, we will surely add chinese data. For the current release, we would advise users to finetune these checkpoints on the Chinese datasets.

[ggg-s]

hi @FrankLeeeee , When fine-tuning a model on a Chinese vertical-domain dataset, what should the composition of the fine-tuning data look like? For example, should it include a mix of general training data, open-source Chinese datasets, and vertical-domain–specific data?