We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent 1e43b47 commit 7328d7dCopy full SHA for 7328d7d
open_diloco/train_fsdp.py
@@ -373,6 +373,7 @@ def outer_scheduler_fn(opt):
373
model=model,
374
optimizer=optimizer.inner_optimizer,
375
scheduler=scheduler,
376
+ outer_scheduler=optimizer.outer_scheduler,
377
outer_optimizer=optimizer.state_averager.optimizer,
378
scaler=scaler,
379
data_loader=train_dataloader,
0 commit comments