v0.7.0

[ValerianRey]

⚡ Performance update ⚡

In this release, we updated torchjd to remove some of the unnecessary overhead in the internal code. This should lead to small but noticeable performance improvements (up to 10% speed).

We have also made torchjd more lightweight, by making optional some dependencies that were only used by CAGrad and NashMTL (the changelog explains how to keep installing these dependencies).

We have also fixed all internal type errors thanks to mypy, and we have added a py.typed file so mypy can be used downstream.

Changelog

Changed

• BREAKING: Changed the dependencies of CAGrad and NashMTL to be optional when installing
  TorchJD. Users of these aggregators will have to use pip install torchjd[cagrad], pip install torchjd[nash_mtl] or pip install torchjd[full] to install TorchJD alongside those dependencies.
  This should make TorchJD more lightweight.
• BREAKING: Made the aggregator modules and the autojac package protected. The aggregators
  must now always be imported via their package (e.g.
  from torchjd.aggregation.upgrad import UPGrad must be changed to
  from torchjd.aggregation import UPGrad). The backward and mtl_backward functions must now
  always be imported directly from the torchjd package (e.g.
  from torchjd.autojac.mtl_backward import mtl_backward must be changed to
  from torchjd import mtl_backward).
• Removed the check that the input Jacobian matrix provided to an aggregator does not contain nan,
  inf or -inf values. This check was costly in memory and in time for large matrices so this
  should improve performance. However, if the optimization diverges for some reason (for instance
  due to a too large learning rate), the resulting exceptions may come from other sources.
• Removed some runtime checks on the shapes of the internal tensors used by the autojac engine.
  This should lead to a small performance improvement.

Fixed

• Made some aggregators (CAGrad, ConFIG, DualProj, GradDrop, IMTLG, NashMTL, PCGrad
  and UPGrad) raise a NonDifferentiableError whenever one tries to differentiate through them.
  Before this change, trying to differentiate through them leaded to wrong gradients or unclear
  errors.

Added

• Added a py.typed file in the top package of torchjd to ensure compliance with
  PEP 561. This should make it possible for users to use
  mypy against the type annotations provided in torchjd.

---

This discussion was created from the release v0.7.0.