Skip to content

显示可训练参数数量问题 #254

@xxll88

Description

@xxll88

用chatglm-6b 运行train.py,显示:
trainable params: 14680064 || all params: 3368640512 || trainable%: 0.4357860076700283
全部参数的数量不是6B,是什么原因?

微调完成后,运行 infer_lora_finetuning.py ,显示
trainable params: 6172532736 || all params: 6187212800 || trainable%: 99.7627354274933
全部参数的数量正确,但lora可训练参数 99%, 是什么原因?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions