全参数微调后，llm回复全是.....

[lulukie]

按照readme进行复现，train.py后模型不能正确输出。

[kang-qingjiu]

我的也差不多，回复都是些奇怪的符号

[CarrotSwordsman]

乱码是由于在推理的时候仍在训练模式，没有切换到评估模式导致。在源代码的predict中添加两行model.eval()和model.train()即可

def predict(messages, model, tokenizer):
    device = "cuda"

    model.eval()  # 关键：切换到评估模式
    
    text = tokenizer.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True
    )
    model_inputs = tokenizer([text], return_tensors="pt").to(device)

    generated_ids = model.generate(
        model_inputs.input_ids,
        max_new_tokens=MAX_LENGTH,
    )
    
    model.train()  # 切回训练模式
    
    generated_ids = [
        output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
    ]

    response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]

    return response```