Skip to content

Allow more granular control over which timesteps are chosen during training #2247

@codecomputation

Description

@codecomputation

Timesteps are randomly chosen during training within a specified range, defaulting to 0 to 1000. You can only adjust min and max range for timesteps chosen. However, because the min and max range apply to all training steps/epochs, it would only result in undertrained loras if anything besides the default settings are used. It would be better if there's a piecewise function parameter that allows you to choose a timestep for each step similar to the piecewise function scheduler for the learning rate. This would help with generalization and reduce overcooking since not all timesteps need to have the same emphasis.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions