Add multiobjective solver and regularized training (#783)

* Add multiobjective solver and regularized training

* Add example for multiobjective training

* Add jacobian regularizer and SAM

* update license headers

* add api draft for multiobjective training

* add all necessary modules to run the complete xcebra pipeline

* add notebooks to reproduce xcebra pipeline

* add first working notebook

* add notebook with hybrid learning

* add notebook with creation of synthetic data

* add notebook with hybrid training

* add plot with R2 for different parts of the embedding

* add new API

* update api wrapper with more checks and messages

* add tests and notebook with new api

* merge xcebra into attribution

* separate xcebra dataset from cebra

* some minor refactoring of cebra dataset

* separate xcebra loader from cebra

* remove xcebra distributions from cebra

* minor refactoring with distributions

* separate xcebra criterions from cebra

* minor refactoring on criterion

* separate xcebra models/criterions/layers from cebra

* refactoring multiobjective

* more refactoring...

* separate xcebra solvers from cebra

* more refactoring

* move xcebra to its own package

* move more files into xcebra package

* more files and remove changes with the registry

* remove unncessary import

* add folder structure

* move back distributions

* add missing init

* remove wrong init

* make loader and dataset run with new imports

* making it run!

* make attribution run

* Run pre-commit

* move xcebra repo one level up

* update gitignore and add __init__ from data

* add init to distributions

* add correct init for attribution pacakge

* add correct init for model package

* fix remaining imports

* fix tests

* add examples back to xcebra repo

* update imports from graphs_xcebra

* add setup.py to create a package

* update imports of graph_xcebra

* update notebooks

* Formatting code for submission

Co-authored-by: Rodrigo Gonzalez <[email protected]>

* move test into xcebra

* Add README

* move distributions back to main package

* clean up examples

* adapt tests

* Add LICENSE

* add train/eval notebook again

* add notebook with clean results

* rm synthetic data

* change name from xcebra to regcl

* change names of modules and adapt imports

* change name from graphs_xcebra to synthetic_data

* Integrate into CEBRA

* Fix remaining imports and make notebook runnable

* Add dependencies, add version flag

* Remove synthetic data files

* reset dockerfile, move vmf

* apply pre-commit

* Update notice

* add some docstrings

* Apply license headers

* add new scd notebook

* add notebook with scd

---------

Co-authored-by: Steffen Schneider <[email protected]>

Author: gonlairo

NOTICE.yml

@@ -35,3 +35,34 @@
     - 'tests/**/*.py'
     - 'docs/**/*.py'
     - 'conda/**/*.yml'
+
+- header: |
+    Regularized contrastive learning implementation.
+
+    Not licensed yet. Distribution for review.
+    Code will be open-sourced upon publication.
+
+  include:
+    - 'cebra/solver/multiobjective.py'
+    - 'cebra/solver/regularized.py'
+    - 'cebra/solver/metrics.py'
+    - 'cebra/models/multiobjective.py'
+    - 'cebra/models/multi_criterions.py'
+    - 'cebra/data/multiobjective.py'
+    - 'cebra/attribution/*.py'
+    - 'tests/test_multiobjective.py'
+
+- header: |
+    Copyright (c) Facebook, Inc. and its affiliates.
+
+    This source code is licensed under the MIT license found in the
+    LICENSE file in the root directory of this source tree.
+
+    PyTorch implementation of Jacobian regularization described in [1].
+
+    [1] Judy Hoffman, Daniel A. Roberts, and Sho Yaida,
+        "Robust Learning with Jacobian Regularization," 2019.
+        [arxiv:1908.02729](https://arxiv.org/abs/1908.02729)
+
+  include:
+    - 'cebra/models/jacobian_regularizer.py'

cebra/__init__.py

@@ -66,7 +66,7 @@
 
 import cebra.integrations.sklearn as sklearn
 
-__version__ = "0.4.0"
+__version__ = "0.4.0+regcl"
 __all__ = ["CEBRA"]
 __allow_lazy_imports = False
 __lazy_imports = {}

cebra/attribution/__init__.py

@@ -0,0 +1,13 @@
+#
+# Regularized contrastive learning implementation.
+#
+# Not licensed yet. Distribution for review.
+# Code will be open-sourced upon publication.
+#
+import cebra.registry
+
+cebra.registry.add_helper_functions(__name__)
+
+from cebra.attribution.attribution_models import *
+from cebra.attribution.jacobian_attribution import *
+from cebra.attribution.jacobian import *