Support for dense Hessian and other improvements

Features

• dtypes are now inferred automatically instead of requiring to pass them manually #71
• n_output is also inferred #90
• adds support for Conv1d layers #76
• adds support for LayerNorm layers #80
• PVectors support pow #86
• KFAC and EKFAC support pow #87
• Support for dense Hessian matrices #91
• Support for binary classification FIMs #92
• PMat solve now supports working with PFMap objects (e.g. jacobians) #101
• Calculation of FIMs for multiclass classification is done using another formula, more numerically stable #102

Helpers:

• Gram matrix and Hessian helpers #97

Renaming (possibly breaking backward compatibility)

• Generators are renamed to Backends #96
• get_**** methods for obtaining dense pytorch versions of different objects are renamed to to_torch #100

CICD

• Updates to use uv, fixes PyPI and TestPyPI integration

Internal changes in Jacobian generator, new features, new layers

Internal changes:

• The DataLoader is not part of the generator anymore
• Full backward hooks in jacobian generator
• Migrates gradient computation outside of Jacobian class to JacobianFactory
• Adds efficient grads for convolution implementation, with switch to choose between 2 implementations
• Changes solve implementation

New features

• PVector add_to_model method
• Matrix-matrix products for KFAC, Dense, BlockDiag; Diag representations
• Solve for PMatLowRank
• Dot between 2 PVector objects
• Pickle for PMat and PVector objects
• Grad method for PVector objects
• MC Fisher helper for segmentation
• Adds get_kfe method to PMatEKFAC

New layers

• Custom layers: Cosine, Weightnorm, Affine
• Support for additional layers: ConvTranspose2d

Minor release: Jacobian generator now raises an Exception when using batch norm in training mode instead of silently failing

This minor releases makes sure that users are aware of the fact that we currently do not support batch norm in training mode. Prior releases would instead silently fail, without telling the user that the returned objects are likely wrong.

Layer Collections, support for GroupNorm, QuasiDiag representation

This release revamps NNGeometry in order to have LayerCollection at its heart, which allows:

• flexibility in parameter space definition (e.g. if you want to compute FIM or NTK for a subset of your layers)
• flexbility in some operations, e.g. in a Fv product F.mv(v) F and v do not need to be created from the same model instance, but as long as they share the same LayerCollection it is enough

We also added support for GroupNorm layers, and included the QuasiDiag parameter space representation.

This release includes some documentation (not perfect !), and supports continuous integration tools.