squeezenet example (#428)

* update docs

* release squeezenet example

* release binarynet and squeezenet example

* fix typo

* fix typo

Author: zsdonghao

README.md

@@ -84,6 +84,8 @@ Examples can be found [in this folder](https://github.com/zsdonghao/tensorlayer/
  - VGG 16 (ImageNet). Classification task, see [tutorial_vgg16.py](https://github.com/zsdonghao/tensorlayer/blob/master/example/tutorial_vgg16.py).
  - VGG 19 (ImageNet). Classification task, see [tutorial_vgg19.py](https://github.com/zsdonghao/tensorlayer/blob/master/example/tutorial_vgg19.py).
  - InceptionV3 (ImageNet). Classification task, see [tutorial\_inceptionV3_tfslim.py](https://github.com/zsdonghao/tensorlayer/blob/master/example/tutorial_inceptionV3_tfslim.py).
+ - SqueezeNet (ImageNet). Classification task, see [tutorial_squeezenet.py](https://github.com/zsdonghao/tensorlayer/blob/master/example/tutorial_squeezenet.py)
+ - BinaryNet (MNIST). Classification task, see [tutorial_squeezenet.py](https://github.com/tensorlayer/tensorlayer/blob/master/example/tutorial_binarynet_mnist_cnn.py) 
  - Wide ResNet (CIFAR) by [ritchieng](https://github.com/ritchieng/wideresnet-tensorlayer).
  - More CNN implementations of [TF-Slim](https://github.com/tensorflow/models/tree/master/research/slim) can be connected to TensorLayer via SlimNetsLayer.
  - [Spatial Transformer Networks](https://arxiv.org/abs/1506.02025) by [zsdonghao](https://github.com/zsdonghao/Spatial-Transformer-Nets).

docs/modules/layers.rst

@@ -805,11 +805,9 @@ Binary Nets
 Read Me
 ^^^^^^^^^^^^^^
 
-This is an experimental API package for building Binary Nets.
-We are using matrix multiplication rather than add-minus and bit-count operation at the moment.
-Therefore, these APIs would not speed up the inferencing, for production, you can train model via TensorLayer and deploy the model into other customized C/C++ implementation (We probably provide users an extra C/C++ binary net framework that can load model from TensorLayer).
+This is an experimental API package for building Binary Nets. We are using matrix multiplication rather than add-minus and bit-count operation at the moment. Therefore, these APIs would not speed up the inferencing, for production, you can train model via TensorLayer and deploy the model into other customized C/C++ implementation (We probably provide users an extra C/C++ binary net framework that can load model from TensorLayer).
 
-Note that, these experimental APIs can be changed in anytime.
+Note that, these experimental APIs can be changed in the future
 
 Binarized Dense
 ^^^^^^^^^^^^^^^^^

docs/user/example.rst

@@ -21,6 +21,8 @@ Computer Vision
  - VGG 16 (ImageNet). Classification task, see `tutorial_vgg16.py <https://github.com/zsdonghao/tensorlayer/blob/master/example/tutorial_vgg16.py>`_.
  - VGG 19 (ImageNet). Classification task, see `tutorial_vgg19.py <https://github.com/zsdonghao/tensorlayer/blob/master/example/tutorial_vgg19.py>`_.
  - InceptionV3 (ImageNet). Classification task, see `tutorial_inceptionV3_tfslim.py <https://github.com/zsdonghao/tensorlayer/blob/master/example/tutorial_inceptionV3_tfslim.py>`_.
+ - SqueezeNet (ImageNet). Classification task, see `tutorial_squeezenet.py <https://github.com/zsdonghao/tensorlayer/blob/master/example/tutorial_squeezenet.py>`_.
+ - BinaryNet (MNIST). Classification task, see `tutorial_squeezenet.py <https://github.com/tensorlayer/tensorlayer/blob/master/example/tutorial_binarynet_mnist_cnn.py>`_.
  - Wide ResNet (CIFAR) by `ritchieng <https://github.com/ritchieng/wideresnet-tensorlayer>`_.
  - More CNN implementations of `TF-Slim <https://github.com/tensorflow/models/tree/master/research/slim>`_ can be connected to TensorLayer via SlimNetsLayer.
  - `Spatial Transformer Networks <https://arxiv.org/abs/1506.02025>`_ by `zsdonghao <https://github.com/zsdonghao/Spatial-Transformer-Nets>`__.

example/data/imagenet_class_index.json

@@ -0,0 +1,122 @@
+#! /usr/bin/python
+# -*- coding: utf-8 -*-
+
+import time, os, json
+import numpy as np
+import tensorflow as tf
+import tensorlayer as tl
+from tensorlayer.layers import InputLayer, Conv2d, MaxPool2d, ConcatLayer, DropoutLayer, GlobalMeanPool2d
+
+
+def decode_predictions(preds, top=5):  # keras.applications.resnet50
+    fpath = os.path.join("data", "imagenet_class_index.json")
+    if tl.files.file_exists(fpath) is False:
+        raise Exception("{} / download imagenet_class_index.json from: https://github.com/zsdonghao/tensorlayer/tree/master/example/data")
+    if isinstance(preds, np.ndarray) is False:
+        preds = np.asarray(preds)
+    if len(preds.shape) != 2 or preds.shape[1] != 1000:
+        raise ValueError('`decode_predictions` expects '
+                         'a batch of predictions '
+                         '(i.e. a 2D array of shape (samples, 1000)). '
+                         'Found array with shape: ' + str(preds.shape))
+    with open(fpath) as f:
+        CLASS_INDEX = json.load(f)
+    results = []
+    for pred in preds:
+        top_indices = pred.argsort()[-top:][::-1]
+        result = [tuple(CLASS_INDEX[str(i)]) + (pred[i], ) for i in top_indices]
+        result.sort(key=lambda x: x[2], reverse=True)
+        results.append(result)
+    return results
+
+
+def squeezenet(x, is_train=True, reuse=False):
+    # model from: https://github.com/wohlert/keras-squeezenet
+    #             https://github.com/DT42/squeezenet_demo/blob/master/model.py
+    with tf.variable_scope("squeezenet", reuse=reuse):
+        with tf.variable_scope("input"):
+            n = InputLayer(x)
+            # n = Conv2d(n, 96, (7,7),(2,2),tf.nn.relu,'SAME',name='conv1')
+            n = Conv2d(n, 64, (3, 3), (2, 2), tf.nn.relu, 'SAME', name='conv1')
+            n = MaxPool2d(n, (3, 3), (2, 2), 'VALID', name='max')
+
+        with tf.variable_scope("fire2"):
+            n = Conv2d(n, 16, (1, 1), (1, 1), tf.nn.relu, 'SAME', name='squeeze1x1')
+            n1 = Conv2d(n, 64, (1, 1), (1, 1), tf.nn.relu, 'SAME', name='expand1x1')
+            n2 = Conv2d(n, 64, (3, 3), (1, 1), tf.nn.relu, 'SAME', name='expand3x3')
+            n = ConcatLayer([n1, n2], -1, name='concat')
+
+        with tf.variable_scope("fire3"):
+            n = Conv2d(n, 16, (1, 1), (1, 1), tf.nn.relu, 'SAME', name='squeeze1x1')
+            n1 = Conv2d(n, 64, (1, 1), (1, 1), tf.nn.relu, 'SAME', name='expand1x1')
+            n2 = Conv2d(n, 64, (3, 3), (1, 1), tf.nn.relu, 'SAME', name='expand3x3')
+            n = ConcatLayer([n1, n2], -1, name='concat')
+            n = MaxPool2d(n, (3, 3), (2, 2), 'VALID', name='max')
+
+        with tf.variable_scope("fire4"):
+            n = Conv2d(n, 32, (1, 1), (1, 1), tf.nn.relu, 'SAME', name='squeeze1x1')
+            n1 = Conv2d(n, 128, (1, 1), (1, 1), tf.nn.relu, 'SAME', name='expand1x1')
+            n2 = Conv2d(n, 128, (3, 3), (1, 1), tf.nn.relu, 'SAME', name='expand3x3')
+            n = ConcatLayer([n1, n2], -1, name='concat')
+
+        with tf.variable_scope("fire5"):
+            n = Conv2d(n, 32, (1, 1), (1, 1), tf.nn.relu, 'SAME', name='squeeze1x1')
+            n1 = Conv2d(n, 128, (1, 1), (1, 1), tf.nn.relu, 'SAME', name='expand1x1')
+            n2 = Conv2d(n, 128, (3, 3), (1, 1), tf.nn.relu, 'SAME', name='expand3x3')
+            n = ConcatLayer([n1, n2], -1, name='concat')
+            n = MaxPool2d(n, (3, 3), (2, 2), 'VALID', name='max')
+
+        with tf.variable_scope("fire6"):
+            n = Conv2d(n, 48, (1, 1), (1, 1), tf.nn.relu, 'SAME', name='squeeze1x1')
+            n1 = Conv2d(n, 192, (1, 1), (1, 1), tf.nn.relu, 'SAME', name='expand1x1')
+            n2 = Conv2d(n, 192, (3, 3), (1, 1), tf.nn.relu, 'SAME', name='expand3x3')
+            n = ConcatLayer([n1, n2], -1, name='concat')
+
+        with tf.variable_scope("fire7"):
+            n = Conv2d(n, 48, (1, 1), (1, 1), tf.nn.relu, 'SAME', name='squeeze1x1')
+            n1 = Conv2d(n, 192, (1, 1), (1, 1), tf.nn.relu, 'SAME', name='expand1x1')
+            n2 = Conv2d(n, 192, (3, 3), (1, 1), tf.nn.relu, 'SAME', name='expand3x3')
+            n = ConcatLayer([n1, n2], -1, name='concat')
+
+        with tf.variable_scope("fire8"):
+            n = Conv2d(n, 64, (1, 1), (1, 1), tf.nn.relu, 'SAME', name='squeeze1x1')
+            n1 = Conv2d(n, 256, (1, 1), (1, 1), tf.nn.relu, 'SAME', name='expand1x1')
+            n2 = Conv2d(n, 256, (3, 3), (1, 1), tf.nn.relu, 'SAME', name='expand3x3')
+            n = ConcatLayer([n1, n2], -1, name='concat')
+
+        with tf.variable_scope("fire9"):
+            n = Conv2d(n, 64, (1, 1), (1, 1), tf.nn.relu, 'SAME', name='squeeze1x1')
+            n1 = Conv2d(n, 256, (1, 1), (1, 1), tf.nn.relu, 'SAME', name='expand1x1')
+            n2 = Conv2d(n, 256, (3, 3), (1, 1), tf.nn.relu, 'SAME', name='expand3x3')
+            n = ConcatLayer([n1, n2], -1, name='concat')
+
+        with tf.variable_scope("output"):
+            n = DropoutLayer(n, keep=0.5, is_fix=True, is_train=is_train, name='drop1')
+            n = Conv2d(n, 1000, (1, 1), (1, 1), padding='VALID', name='conv10')  # 13, 13, 1000
+            n = GlobalMeanPool2d(n)
+        return n
+
+
+x = tf.placeholder(tf.float32, (None, 224, 224, 3))
+n = squeezenet(x, False, False)
+softmax = tf.nn.softmax(n.outputs)
+n.print_layers()
+n.print_params(False)
+
+sess = tf.InteractiveSession()
+tl.layers.initialize_global_variables(sess)
+
+if tl.files.file_exists('squeezenet.npz'):
+    tl.files.load_and_assign_npz(sess=sess, name='squeezenet.npz', network=n)
+else:
+    raise Exception("please download the pre-trained squeezenet.npz from https://github.com/tensorlayer/pretrained-models")
+
+img = tl.vis.read_image('data/tiger.jpeg', '')
+img = tl.prepro.imresize(img, (224, 224))
+prob = sess.run(softmax, feed_dict={x: [img]})[0]
+start_time = time.time()
+prob = sess.run(softmax, feed_dict={x: [img]})[0]
+print("  End time : %.5ss" % (time.time() - start_time))
+
+print('Predicted:', decode_predictions([prob], top=3)[0])
+tl.files.save_npz(n.all_params, name='squeezenet.npz', sess=sess)