Skip to content

Commit b882dfb

Browse files
Catgatwinglian
andauthored
Fixed Rex Scheduler Warm Up (axolotl-ai-cloud#2535) [skip ci]
* Fixed Rex Scheduler Warm Up * chore: lint --------- Co-authored-by: Wing Lian <[email protected]>
1 parent b640db1 commit b882dfb

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

src/axolotl/utils/schedulers.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ def __init__(
4040
self.max_lr = max_lr
4141
self.total_steps = total_steps
4242
self.num_warmup_steps = num_warmup_steps
43-
self.last_step = last_step - 1
43+
self.last_step = max(last_step - 1, 0)
4444

4545
# Ensure each parameter group has an "initial_lr" key to avoid issues when resuming.
4646
for group in optimizer.param_groups:

0 commit comments

Comments
 (0)