We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent dfd4455 commit d84042dCopy full SHA for d84042d
open_diloco/train_fsdp.py
@@ -380,6 +380,7 @@ def outer_scheduler_fn(opt):
380
model=model,
381
optimizer=optimizer.inner_optimizer,
382
scheduler=scheduler,
383
+ outer_scheduler=optimizer.outer_scheduler,
384
outer_optimizer=optimizer.state_averager.optimizer,
385
scaler=scaler,
386
data_loader=train_dataloader,
0 commit comments