Initial open source release

Author: master

.gitignore

@@ -0,0 +1,6 @@
+.ipynb_checkpoints
+__pycache__
+*.pyc
+build
+dist
+*.egg-info

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2020 Oleg Smirnov <[email protected]>
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,93 @@
+# TensorFlow ManOpt
+
+[![License](https://img.shields.io/:license-mit-blue.svg)](https://badges.mit-license.org)
+
+A library for manifold-constrained optimization in TensorFlow.
+
+## Installation
+
+To install the latest development version from GitHub:
+
+```bash
+pip install git+https://github.com/master/tensorflow-manopt.git
+```
+
+To install a package from PyPI:
+
+```bash
+pip install tensorflow-manopt
+```
+
+## Features
+
+The core package implements concepts in differential geometry, such as
+manifolds and Riemannian metrics with associated exponential and logarithmic
+maps, geodesics, retractions, and transports. For manifolds, where closed-form
+expressions are not available, the library provides numerical approximations.
+
+<img align="right" width="400" src="https://github.com/master/tensorflow-manopt/blob/master/examples/usage.png?raw=true">
+
+```python
+S = manopt.manifolds.Sphere()
+
+x = S.projx(tf.constant([0.1, -0.1, 0.1]))
+u = S.proju(x, tf.constant([1., 1., 1.]))
+v = S.proju(x, tf.constant([-0.7, -1.4, 1.4]))
+
+y = S.exp(x, v)
+
+u_ = S.transp(x, y, u)
+v_ = S.transp(x, y, v)
+```
+
+### Manifolds
+
+ - `manifolds.Cholesky` - manifold of lower triangular matrices with positive diagonal elements
+ - `manifolds.Euclidian` - unconstrained manifold with the Euclidean metric
+ - `manifolds.Grassmannian` - manifold of `p`-dimensional linear subspaces of the `n`-dimensional space
+ - `manifolds.Hyperboloid` - manifold of `n`-dimensional hyperbolic space embedded in the `n+1`-dimensional Minkowski space
+ - `manifolds.Poincare` - the Poincaré ball model of the hyperbolic space
+ - `manifolds.Product` - Cartesian product of manifolds
+ - `manifolds.SPDAffineInvariant` - manifold of symmetric positive definite (SPD) matrices endowed with the affine-invariant metric
+ - `manifolds.SPDLogCholesky` - SPD manifold with the Log-Cholesky metric
+ - `manifolds.SPDLogEuclidean` - SPD manifold with the Log-Euclidean metric
+ - `manifolds.SpecialOrthogonal` - manifold of rotation matrices
+ - `manifolds.Sphere` - manifold of unit-normalized points
+ - `manifolds.StiefelEuclidean` - manifold of orthonormal `p`-frames in the `n`-dimensional space endowed with the Euclidean metric
+ - `manifolds.StiefelCanonical` - Stiefel manifold with the canonical metric
+ - `manifolds.StiefelCayley` - Stiefel manifold the retraction map via an iterative Cayley transform
+
+
+### Optimizers
+
+ Constrained optimization algorithms work as drop-in replacements for Keras
+optimizers for sparse and dense updates in both Eager and Graph modes.
+
+ - `optimizers.RiemannianSGD` - Riemannian Gradient Descent
+ - `optimizers.RiemannianAdam` - Riemannian Adam and AMSGrad
+ - `optimizers.ConstrainedRMSProp` - Constrained RMSProp
+
+### Layers
+
+ - `layers.ManifoldEmbedding` - constrained `keras.layers.Embedding` layer
+
+## Examples
+
+ - [SPDNet](examples/spdnet/) - Huang, Zhiwu, and Luc Van Gool. "A Riemannian network for SPD matrix learning." Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence. AAAI Press, 2017.
+ - [LieNet](examples/lienet/) - Huang, Zhiwu, et al. "Deep learning on Lie groups for skeleton-based action recognition." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
+ - [GrNet](examples/grnet/) - Huang, Zhiwu, Jiqing Wu, and Luc Van Gool. "Building Deep Networks on Grassmann Manifolds." AAAI. AAAI Press, 2018.
+ - [Hyperbolic Neural Network](examples/hyperbolic_nn/) - Ganea, Octavian, Gary Bécigneul, and Thomas Hofmann. "Hyperbolic neural networks." Advances in neural information processing systems. 2018.
+ - [Poincaré GloVe](examples/poincare_glove/) - Tifrea, Alexandru, Gary Becigneul, and Octavian-Eugen Ganea. "Poincaré Glove: Hyperbolic Word Embeddings." International Conference on Learning Representations. 2018.
+
+## Acknowledgment
+
+ TensorFlow ManOpt was inspired by many similar projects:
+
+ - [Manopt](https://www.manopt.org/), a matlab toolbox for optimization on manifolds
+ - [Pymanopt](https://www.pymanopt.org/), a Python toolbox for optimization on manifolds
+ - [Geoopt](https://geoopt.readthedocs.io/): Riemannian Optimization in PyTorch
+ - [Geomstats](https://geomstats.github.io/), an open-source Python package for computations and statistics on nonlinear manifolds
+
+## License
+
+ The code is MIT-licensed.

examples/grnet/README.md

@@ -0,0 +1,40 @@
+# GrNet in TensorFlow
+
+Implementation of GrNet [1], a deep network on Grassmann manifolds.
+
+<img align="center" width="800" src="https://github.com/master/tensorflow-manopt/blob/master/examples/grnet/grnet.png?raw=true">
+
+## Requirements
+
+ * Python 3.5+
+ * SciPy
+ * NumPy
+ * TensorFlow 2.3+
+ * TensorFlow ManOpt
+
+## Training
+
+Configure `gcloud` to use Python 3:
+
+```bash
+gcloud config set ml_engine/local_python /usr/bin/python3
+```
+
+Train GrNet locally on the Acted Facial Expression in Wild [2] dataset:
+
+```bash
+gcloud ai-platform local train \
+       --module-name grnet.task \
+       --package-path . \
+       -- \
+       --data-dir data
+       --job-dir ckpt
+```
+
+## References
+
+ 1. Huang, Zhiwu, Jiqing Wu, and Luc Van Gool. "Building Deep Networks on
+ Grassmann Manifolds." AAAI. AAAI Press, 2018.
+ 2. Dhall, Abhinav, et al. "Acted facial expressions in the wild database."
+ Australian National University, Canberra, Australia, Technical Report
+ TR-CS-11 2 (2011): 1.

examples/grnet/__init__.py

@@ -0,0 +1,148 @@
+import tensorflow as tf
+
+from tensorflow_manopt.variable import assign_to_manifold
+from tensorflow_manopt.manifolds import Grassmannian
+from tensorflow_manopt.manifolds import utils
+from tensorflow_manopt.optimizers import RiemannianSGD
+
+
+@tf.keras.utils.register_keras_serializable(name="FRMap")
+class FRMap(tf.keras.layers.Layer):
+    """Full Rank Mapping layer."""
+
+    def __init__(self, output_dim, num_proj=8, *args, **kwargs):
+        """Instantiate the FRMap layer.
+
+        Args:
+          output_dim: projection output dimension
+          num_proj: number of projections to compute
+        """
+        super().__init__(*args, **kwargs)
+        self.output_dim = output_dim
+        self.num_proj = num_proj
+
+    def build(self, input_shape):
+        grassmannian = Grassmannian()
+        self.w = self.add_weight(
+            "w",
+            shape=[self.num_proj, input_shape[-2], self.output_dim],
+            initializer=grassmannian.random,
+        )
+        assign_to_manifold(self.w, grassmannian)
+        self._expand = len(input_shape) == 3
+
+    def call(self, inputs):
+        if self._expand:
+            inputs = tf.expand_dims(inputs, -3)
+        return utils.transposem(self.w) @ inputs
+
+    def get_config(self):
+        config = {"output_dim": self.output_dim, "num_proj": self.num_proj}
+        return dict(list(super().get_config().items()) + list(config.items()))
+
+
+@tf.keras.utils.register_keras_serializable(name="ReOrth")
+class ReOrth(tf.keras.layers.Layer):
+    """Re-Orthonormalization layer."""
+
+    def call(self, inputs):
+        q, _r = tf.linalg.qr(inputs)
+        return q
+
+
+@tf.keras.utils.register_keras_serializable(name="ProjMap")
+class ProjMap(tf.keras.layers.Layer):
+    """Projection Mapping layer."""
+
+    def call(self, inputs):
+        return inputs @ utils.transposem(inputs)
+
+
+@tf.keras.utils.register_keras_serializable(name="ProjPooling")
+class ProjPooling(tf.keras.layers.Layer):
+    """Projection Pooling layer."""
+
+    def __init__(self, stride=2, *args, **kwargs):
+        """Instantiate the ProjPooling layer.
+
+        Args:
+          stride: factor by which to downscale
+        """
+        super().__init__(*args, **kwargs)
+        self.stride = stride
+
+    def call(self, inputs):
+        shape = tf.shape(inputs)
+        new_shape = [
+            shape[0],
+            shape[1],
+            self.stride,
+            shape[2] // self.stride,
+            shape[3],
+        ]
+        return tf.reduce_mean(
+            tf.reshape(inputs, new_shape), axis=-3, keepdims=False
+        )
+
+    def get_config(self):
+        config = {"stride": self.stride}
+        return dict(list(super().get_config().items()) + list(config.items()))
+
+
+@tf.keras.utils.register_keras_serializable(name="OrthMap")
+class OrthMap(tf.keras.layers.Layer):
+    """Orthonormal Mapping layer."""
+
+    def __init__(self, top_eigen, *args, **kwargs):
+        """Instantiate the OrthMap layer.
+
+        Args:
+          num_eigen: number of top eigenvectors to retain
+        """
+        super().__init__(*args, **kwargs)
+        self.top_eigen = top_eigen
+
+    def call(self, inputs):
+        _s, u, _vt = tf.linalg.svd(inputs)
+        return u[..., : self.top_eigen]
+
+    def get_config(self):
+        config = {"top_eigen": self.top_eigen}
+        return dict(list(super().get_config().items()) + list(config.items()))
+
+
+def create_model(
+    learning_rate,
+    num_classes,
+    frmap_dims=[300, 100],
+    pool_stride=2,
+    top_eigen=10,
+):
+    """Instantiate the GrNet architecture.
+
+    Huang, Zhiwu, Jiqing Wu, and Luc Van Gool. "Building Deep Networks on
+    Grassmann Manifolds." AAAI. AAAI Press, 2018.
+
+    Args:
+      learning_rate: model learning rate
+      num_classes: number of output classes
+      frmap_dims: dimensions of FrMap layers
+      pool_stride: pooling stride
+      top_eigen: number of eigenvectors to retain in OrthMap
+    """
+    model = tf.keras.Sequential()
+    for output_dim in frmap_dims:
+        model.add(FRMap(output_dim))
+        model.add(ReOrth())
+        model.add(ProjMap())
+        model.add(ProjPooling(pool_stride))
+        model.add(OrthMap(top_eigen))
+    model.add(ProjMap())
+    model.add(tf.keras.layers.Flatten())
+    model.add(tf.keras.layers.Dense(num_classes, use_bias=False))
+    model.compile(
+        optimizer=RiemannianSGD(learning_rate),
+        loss=tf.losses.SparseCategoricalCrossentropy(from_logits=True),
+        metrics=[tf.metrics.SparseCategoricalAccuracy()],
+    )
+    return model

examples/grnet/grnet.png

@@ -0,0 +1,2 @@
+scipy
+tensorflow-manopt

examples/grnet/model.py

@@ -0,0 +1,78 @@
+#!/usr/bin/env python3
+import argparse
+import os
+import tensorflow as tf
+
+from . import model
+from shared import utils
+
+DATA_URL = "https://data.vision.ee.ethz.ch/zzhiwu/ManifoldNetData/GrData/AFEW_Gr_data.zip"
+DATA_FOLDER = "grface_400_inter_histeq"
+AFEW_CLASSES = 7
+
+
+def get_args():
+    parser = argparse.ArgumentParser()
+    parser.add_argument(
+        '--job-dir', type=str, required=True, help='checkpoint dir'
+    )
+    parser.add_argument('--data-dir', type=str, required=True, help='data dir')
+    parser.add_argument(
+        '--num-epochs',
+        type=float,
+        default=50,
+        help='number of training epochs (default 50)',
+    )
+    parser.add_argument(
+        '--batch-size',
+        default=30,
+        type=int,
+        help='number of examples per batch (default 30)',
+    )
+    parser.add_argument(
+        '--shuffle-buffer',
+        default=100,
+        type=int,
+        help='shuffle buffer size (default 100)',
+    )
+    parser.add_argument(
+        '--learning-rate',
+        default=0.01,
+        type=float,
+        help='learning rate (default .01)',
+    )
+    return parser.parse_args()
+
+
+def train_and_evaluate(args):
+    utils.download_data(args.data_dir, DATA_URL, unpack=True)
+    train = utils.load_matlab_data("Y1", args.data_dir, DATA_FOLDER, "train")
+    val = utils.load_matlab_data("Y1", args.data_dir, DATA_FOLDER, "val")
+    train_dataset = (
+        tf.data.Dataset.from_tensor_slices(train)
+        .repeat(args.num_epochs)
+        .shuffle(args.shuffle_buffer)
+        .batch(args.batch_size, drop_remainder=True)
+    )
+    val_dataset = tf.data.Dataset.from_tensor_slices(val).batch(
+        args.batch_size, drop_remainder=True
+    )
+
+    grnet = model.create_model(args.learning_rate, num_classes=AFEW_CLASSES)
+    checkpoint_path = os.path.join(args.job_dir, "afew-grnet.ckpt")
+    cp_callback = tf.keras.callbacks.ModelCheckpoint(
+        filepath=checkpoint_path, save_weights_only=True, verbose=1
+    )
+    grnet.fit(
+        train_dataset,
+        epochs=args.num_epochs,
+        validation_data=val_dataset,
+        callbacks=[cp_callback],
+    )
+    _, acc = grnet.evaluate(val_dataset, verbose=2)
+    print("Final accuracy: {}%".format(acc * 100))
+
+
+if __name__ == "__main__":
+    tf.get_logger().setLevel("INFO")
+    train_and_evaluate(get_args())