Skip to content

Commit acea309

Browse files
authored
docs(autogram): Improve autogram package doc (#416)
1 parent 6870e90 commit acea309

File tree

1 file changed

+9
-7
lines changed

1 file changed

+9
-7
lines changed

src/torchjd/autogram/__init__.py

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,15 @@
11
"""
2-
The autogram package enables the activation of Gramian-based Jacobian descent on your models (As
3-
described in Section 6 of `Jacobian Descent For Multi-Objective Optimization
4-
<https://arxiv.org/pdf/2406.16232>`_). It provides a convenient way to modify a model's backward
5-
pass, allowing you to seamlessly integrate multi-objective optimization in your PyTorch code.
2+
The autogram package provides an engine to efficiently compute the Gramian of the Jacobian of a
3+
tensor of outputs (generally losses) with respect to some modules' parameters. This Gramian contains
4+
all the inner products between pairs of gradients, and is thus a sufficient statistic for most
5+
weighting methods. The algorithm is formally defined in Section 6 of `Jacobian Descent For
6+
Multi-Objective Optimization <https://arxiv.org/pdf/2406.16232>`_).
67
7-
This method typically provides a memory improvement over the :doc:`autojac <../autojac/index>`
8-
package which typically leads to time improvement.
8+
Due to computing the Gramian iteratively over the layers, without ever having to store the full
9+
Jacobian in memory, this method is much more memory-efficient than
10+
:doc:`autojac <../autojac/index>`, which makes it often much faster.
911
10-
The exhaustive list of supported Weightings compatible with `autogram` is:
12+
The list of Weightings compatible with ``autogram`` is:
1113
1214
* :class:`~torchjd.aggregation.UPGradWeighting`
1315
* :class:`~torchjd.aggregation.AlignedMTLWeighting`

0 commit comments

Comments
 (0)