File tree Expand file tree Collapse file tree 1 file changed +9
-7
lines changed Expand file tree Collapse file tree 1 file changed +9
-7
lines changed Original file line number Diff line number Diff line change 11"""
2- The autogram package enables the activation of Gramian-based Jacobian descent on your models (As
3- described in Section 6 of `Jacobian Descent For Multi-Objective Optimization
4- <https://arxiv.org/pdf/2406.16232>`_). It provides a convenient way to modify a model's backward
5- pass, allowing you to seamlessly integrate multi-objective optimization in your PyTorch code.
2+ The autogram package provides an engine to efficiently compute the Gramian of the Jacobian of a
3+ tensor of outputs (generally losses) with respect to some modules' parameters. This Gramian contains
4+ all the inner products between pairs of gradients, and is thus a sufficient statistic for most
5+ weighting methods. The algorithm is formally defined in Section 6 of `Jacobian Descent For
6+ Multi-Objective Optimization <https://arxiv.org/pdf/2406.16232>`_).
67
7- This method typically provides a memory improvement over the :doc:`autojac <../autojac/index>`
8- package which typically leads to time improvement.
8+ Due to computing the Gramian iteratively over the layers, without ever having to store the full
9+ Jacobian in memory, this method is much more memory-efficient than
10+ :doc:`autojac <../autojac/index>`, which makes it often much faster.
911
10- The exhaustive list of supported Weightings compatible with `autogram` is:
12+ The list of Weightings compatible with `` autogram` ` is:
1113
1214* :class:`~torchjd.aggregation.UPGradWeighting`
1315* :class:`~torchjd.aggregation.AlignedMTLWeighting`
You can’t perform that action at this time.
0 commit comments