Skip to content
Discussion options

You must be logged in to vote

You may use --init-model with the dp train command to training a new model from the model parameters of an old model. Not that you may reducing the learning rate so the information stored in the old model would not been forgotten, while increase the relative possibility of visiting the new data systems (by default the possibility is in proportional to the size of the data systems.)

Here is an example, please note that the start learning rate is set to 0.0001 and the possibility of using the new data systems is 30%

"learning_rate" :{
        "type":         "exp",
        "start_lr":     0.0001,  
        "stop_lr":      3.51e-8,
        "decay_steps":  5000,
},

"training" : {
"training_d…

Replies: 1 comment

Comment options

You must be logged in to vote
0 replies
Answer selected by hu-lm
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
2 participants