Skip to content

Commit b5e8877

Browse files
author
Seppo Enarvi
committed
More verbose description of WeightAveraging
1 parent 01161a9 commit b5e8877

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

docs/source-pytorch/advanced/training_tricks.rst

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -61,8 +61,8 @@ end up in a local minimum during optimization.
6161
Lightning provides two callbacks to facilitate weight averaging. :class:`~lightning.pytorch.callbacks.WeightAveraging`
6262
is a generic callback that wraps the
6363
`AveragedModel <https://pytorch.org/docs/stable/generated/torch.optim.swa_utils.AveragedModel.html>`__ class from
64-
PyTorch. It allows SWA, EMA, or a custom averaging strategy to be used and it can be customized to run at specific steps
65-
or epochs.
64+
PyTorch. It allows SWA, EMA, or a custom averaging strategy to be used. By default, it updates the weights after every
65+
step, but it can be customized to update at specific steps or epochs by overriding the `should_update()` method.
6666

6767
The older :class:`~lightning.pytorch.callbacks.StochasticWeightAveraging` callback is specific to SWA. It starts the SWA
6868
procedure after a certain number of epochs and always runs on every epoch. Additionally, it switches to a constant

0 commit comments

Comments
 (0)