Docs for dorefa, ternary weight net (#443)

* update docs for model compression

* fixed typo

Author: zsdonghao

README.md

@@ -85,7 +85,9 @@ Examples can be found [in this folder](https://github.com/zsdonghao/tensorlayer/
  - VGG 19 (ImageNet). Classification task, see [tutorial_vgg19.py](https://github.com/zsdonghao/tensorlayer/blob/master/example/tutorial_vgg19.py).
  - InceptionV3 (ImageNet). Classification task, see [tutorial\_inceptionV3_tfslim.py](https://github.com/zsdonghao/tensorlayer/blob/master/example/tutorial_inceptionV3_tfslim.py).
  - SqueezeNet (ImageNet). Classification task, see [tutorial_squeezenet.py](https://github.com/zsdonghao/tensorlayer/blob/master/example/tutorial_squeezenet.py)
- - BinaryNet (MNIST). Classification task, see [tutorial_binarynet_mnist_cnn.py](https://github.com/tensorlayer/tensorlayer/blob/master/example/tutorial_binarynet_mnist_cnn.py) 
+ - BinaryNet. Model compression, see [mnist](https://github.com/tensorlayer/tensorlayer/blob/master/example/tutorial_binarynet_mnist_cnn.py) [cifar10](https://github.com/tensorlayer/tensorlayer/blob/master/example/tutorial_binarynet_cifar10_tfrecord.py).
+ - Tenary Weight Network. Model compression, see [mnist](https://github.com/tensorlayer/tensorlayer/blob/master/example/tutorial_tenaryweight_mnist_cnn.py) [cifar10](https://github.com/tensorlayer/tensorlayer/blob/master/example/tutorial_tenaryweight_cifar10_tfrecord.py).
+ - DoReFa-Net. Model compression, see [mnist](https://github.com/tensorlayer/tensorlayer/blob/master/example/tutorial_dorefanet_mnist_cnn.py) [cifar10](https://github.com/tensorlayer/tensorlayer/blob/master/example/tutorial_dorefanet_cifar10_tfrecord.py).
  - Wide ResNet (CIFAR) by [ritchieng](https://github.com/ritchieng/wideresnet-tensorlayer).
  - More CNN implementations of [TF-Slim](https://github.com/tensorflow/models/tree/master/research/slim) can be connected to TensorLayer via SlimNetsLayer.
  - [Spatial Transformer Networks](https://arxiv.org/abs/1506.02025) by [zsdonghao](https://github.com/zsdonghao/Spatial-Transformer-Nets).

docs/modules/layers.rst

@@ -334,6 +334,10 @@ Layer list
 
    BinaryDenseLayer
    BinaryConv2d
+   TenaryDenseLayer
+   TenaryConv2d
+   DorefaDenseLayer
+   DorefaConv2d
    SignLayer
    ScaleLayer
 
@@ -834,17 +838,30 @@ Binarized Dense
 ^^^^^^^^^^^^^^^^^
 .. autoclass:: BinaryDenseLayer
 
-
 Binarized Conv2d
 ^^^^^^^^^^^^^^^^^^
 .. autoclass:: BinaryConv2d
 
+Tenary Dense
+^^^^^^^^^^^^^^^^^^
+.. autoclass:: TenaryDenseLayer
+
+Tenary Conv2d
+^^^^^^^^^^^^^^^^^^
+.. autoclass:: TenaryConv2d
+
+Dorefa Dense
+^^^^^^^^^^^^^^^^^^
+.. autoclass:: DorefaDenseLayer
+
+Dorefa Conv2d
+^^^^^^^^^^^^^^^^^^
+.. autoclass:: DorefaConv2d
 
 Sign
 ^^^^^^^^^^^^^^
 .. autoclass:: SignLayer
 
-
 Scale
 ^^^^^^^^^^^^^^
 .. autoclass:: ScaleLayer

docs/user/example.rst

@@ -22,7 +22,9 @@ Computer Vision
  - VGG 19 (ImageNet). Classification task, see `tutorial_vgg19.py <https://github.com/zsdonghao/tensorlayer/blob/master/example/tutorial_vgg19.py>`_.
  - InceptionV3 (ImageNet). Classification task, see `tutorial_inceptionV3_tfslim.py <https://github.com/zsdonghao/tensorlayer/blob/master/example/tutorial_inceptionV3_tfslim.py>`_.
  - SqueezeNet (ImageNet). Classification task, see `tutorial_squeezenet.py <https://github.com/zsdonghao/tensorlayer/blob/master/example/tutorial_squeezenet.py>`_.
- - BinaryNet (MNIST). Classification task, see `tutorial_binarynet_mnist_cnn.py <https://github.com/tensorlayer/tensorlayer/blob/master/example/tutorial_binarynet_mnist_cnn.py>`_.
+ - BinaryNet. Model compression, see ｀mnist <https://github.com/tensorlayer/tensorlayer/blob/master/example/tutorial_binarynet_mnist_cnn.py>`__ `cifar10 <https://github.com/tensorlayer/tensorlayer/blob/master/example/tutorial_binarynet_cifar10_tfrecord.py>`__.
+ - Tenary Weight Network. Model compression, see `mnist <https://github.com/tensorlayer/tensorlayer/blob/master/example/tutorial_tenaryweight_mnist_cnn.py>`__ `cifar10 <https://github.com/tensorlayer/tensorlayer/blob/master/example/tutorial_tenaryweight_cifar10_tfrecord.py>`__.
+ - DoReFa-Net. Model compression, see `mnist <https://github.com/tensorlayer/tensorlayer/blob/master/example/tutorial_dorefanet_mnist_cnn.py>`__ `cifar10 <https://github.com/tensorlayer/tensorlayer/blob/master/example/tutorial_dorefanet_cifar10_tfrecord.py>`__.
  - Wide ResNet (CIFAR) by `ritchieng <https://github.com/ritchieng/wideresnet-tensorlayer>`_.
  - More CNN implementations of `TF-Slim <https://github.com/tensorflow/models/tree/master/research/slim>`_ can be connected to TensorLayer via SlimNetsLayer.
  - `Spatial Transformer Networks <https://arxiv.org/abs/1506.02025>`_ by `zsdonghao <https://github.com/zsdonghao/Spatial-Transformer-Nets>`__.

example/tutorial_binarycifar10_tfrecord.py renamed to example/tutorial_binarynet_cifar10_tfrecord.py

@@ -350,7 +350,7 @@ def __init__(
 
         n_in = int(self.inputs.get_shape()[-1])
         self.n_units = n_units
-        logging.info("BinaryDenseLayer  %s: %d %s" % (self.name, self.n_units, act.__name__))
+        logging.info("TenaryDenseLayer  %s: %d %s" % (self.name, self.n_units, act.__name__))
         with tf.variable_scope(name):
             W = tf.get_variable(name='W', shape=(n_in, n_units), initializer=W_init, dtype=LayersConfig.tf_dtype, **W_init_args)
             # W = tl.act.sign(W)    # dont update ...
@@ -468,7 +468,7 @@ def __init__(
         self.inputs = prev_layer.outputs
         if act is None:
             act = tf.identity
-        logging.info("BinaryConv2d %s: n_filter:%d filter_size:%s strides:%s pad:%s act:%s" % (self.name, n_filter, str(filter_size), str(strides), padding,
+        logging.info("TenaryConv2d %s: n_filter:%d filter_size:%s strides:%s pad:%s act:%s" % (self.name, n_filter, str(filter_size), str(strides), padding,
                                                                                                act.__name__))
 
         if len(strides) != 2:
@@ -500,7 +500,7 @@ def __init__(
 
 
 class DorefaDenseLayer(Layer):
-    """The :class:`DorefaDenseLayer` class is a binary fully connected layer, which weights are 'bitW' bits and the output of the previous layer 
+    """The :class:`DorefaDenseLayer` class is a binary fully connected layer, which weights are 'bitW' bits and the output of the previous layer
     are 'bitA' bits while inferencing.
 
     Note that, the bias vector would not be binarized.
@@ -511,7 +511,7 @@ class DorefaDenseLayer(Layer):
         Previous layer.
     bitW : int
         The bits of this layer's parameter
-    bitA : int 
+    bitA : int
         The bits of the output of previous layer
     n_units : int
         The number of units of this layer.
@@ -561,7 +561,7 @@ def __init__(
 
         n_in = int(self.inputs.get_shape()[-1])
         self.n_units = n_units
-        logging.info("BinaryDenseLayer  %s: %d %s" % (self.name, self.n_units, act.__name__))
+        logging.info("DorefaDenseLayer  %s: %d %s" % (self.name, self.n_units, act.__name__))
         with tf.variable_scope(name):
             W = tf.get_variable(name='W', shape=(n_in, n_units), initializer=W_init, dtype=LayersConfig.tf_dtype, **W_init_args)
             # W = tl.act.sign(W)    # dont update ...
@@ -588,7 +588,7 @@ def __init__(
 
 
 class DorefaConv2d(Layer):
-    """The :class:`DorefaConv2d` class is a binary fully connected layer, which weights are 'bitW' bits and the output of the previous layer 
+    """The :class:`DorefaConv2d` class is a binary fully connected layer, which weights are 'bitW' bits and the output of the previous layer
     are 'bitA' bits while inferencing.
 
     Note that, the bias vector would not be binarized.
@@ -599,7 +599,7 @@ class DorefaConv2d(Layer):
         Previous layer.
     bitW : int
         The bits of this layer's parameter
-    bitA : int 
+    bitA : int
         The bits of the output of previous layer
     n_filter : int
         The number of filters.
@@ -684,7 +684,7 @@ def __init__(
         self.inputs = prev_layer.outputs
         if act is None:
             act = tf.identity
-        logging.info("BinaryConv2d %s: n_filter:%d filter_size:%s strides:%s pad:%s act:%s" % (self.name, n_filter, str(filter_size), str(strides), padding,
+        logging.info("DorefaConv2d %s: n_filter:%d filter_size:%s strides:%s pad:%s act:%s" % (self.name, n_filter, str(filter_size), str(strides), padding,
                                                                                                act.__name__))
 
         if len(strides) != 2: