Skip to content

Gemma-peft tutorial does not perform expected fine-tuning #3261

@dboyker

Description

@dboyker

Executing the script described in the tutorial https://github.com/huggingface/blog/blob/main/gemma-peft.md throws this warning:

You passed a dataset that is already processed (contains an input_ids field) together with a formatting function. Therefore formatting_func will be ignored. Either remove the formatting_func or pass a dataset that is not already processed.

As formatting_func is ignored, the model is not correctly fine-tuned. The script works but there is no guarantee that the model ouput the format Quote: [...] Author: [...].

Proposed solution in this PR: #3260

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions