Merge pull request #336 from tensorlayer/fixdoc

remove unused arguments.

Author: zsdonghao

README.md

@@ -24,7 +24,7 @@ TensorLayer is a deep learning and reinforcement learning library on top of [Ten
 
 # News
 * [18 Jan] [《深度学习：一起玩转TensorLayer》](http://www.broadview.com.cn/book/5059) (Deep Learning using TensorLayer)
-* [17 Dec] Experimentally release APIs for distributed training (by [TensorPort](https://tensorport.com)). See [tiny example](https://github.com/zsdonghao/tensorlayer/blob/master/example/tutorial_mnist_distributed.py).
+* [17 Dec] Release experimental APIs for distributed training (by [TensorPort](https://tensorport.com)). See [tiny example](https://github.com/zsdonghao/tensorlayer/blob/master/example/tutorial_mnist_distributed.py).
 * [17 Nov] Release data augmentation APIs for object detection, see [tl.prepro](http://tensorlayer.readthedocs.io/en/latest/modules/prepro.html#object-detection).
 * [17 Nov] Support [Convolutional LSTM](https://arxiv.org/abs/1506.04214), see [ConvLSTMLayer](http://tensorlayer.readthedocs.io/en/latest/modules/layers.html#conv-lstm-layer).
 * [17 Nov] Support [Deformable Convolution](https://arxiv.org/abs/1703.06211), see [DeformableConv2dLayer](http://tensorlayer.readthedocs.io/en/latest/modules/layers.html#d-deformable-conv).
@@ -41,13 +41,15 @@ TensorLayer is a deep learning and reinforcement learning library on top of [Ten
 
 # Installation
 
-TensorLayer has package pre-requisites including TensorFlow, numpy, matplotlib and nltk (optional). For GPU support, CUDA and cuDNN are required. Please check [documentation](http://tensorlayer.readthedocs.io/en/latest/user/installation.html) for detailed instructions. The simplest way to install TensorLayer in your python program is: 
+TensorLayer has pre-requisites including TensorFlow, numpy, matplotlib and nltk (optional). For GPU support, CUDA and cuDNN are required. 
+The simplest way to install TensorLayer is: 
 
 ```bash
 [for master version] pip install git+https://github.com/zsdonghao/tensorlayer.git (Recommended)
 [for stable version] pip install tensorlayer
 ```
 
+Please check [documentation](http://tensorlayer.readthedocs.io/en/latest/user/installation.html) for detailed instructions. 
 
 
 # Examples and Tutorials
@@ -121,18 +123,21 @@ TensorLayer provides two set of Convolutional layer APIs, see [(Professional)](h
 -->
 
 
-# Library Features
+# Features
 
-As deep learning practitioners, we have been looking for a library that can serve for various development phases. This library shall be easy for beginners by providing rich neural network reference implementations. Later, it can be extended to address *real-world problems* by controlling training backends to exhibit low-level cognitive behaviours. In the end, it shall be able to serve in challenging *production environments*. TensorLayer is developed with the following goals:
+## Design Philosophy
 
-- *Simplicity* : TensorLayer lifts the low-level dataflow abstraction of TensorFlow to **high-level** deep learning modules. 
-A user often find it easy to bootstrap with TensorLayer, and then dive into low-level implementation only if need. 
+As deep learning practitioners, we have been looking for a library that can serve for various development phases. This library is easy for beginners by offering rich neural network 
+examples and tutorials. Later, its APIs can be extended to address *real-world problems* by allowing users to carefully manipulating TensorFlow. In the end, it is able to serve in challenging *production environments*. TensorLayer is a novel library that aims to satisfy development requirements throughout these phases. It has three key features:
+
+- *Simplicity* : TensorLayer lifts the low-level dataflow abstraction of TensorFlow to **high-level** deep learning modules. It also
+provides users with massive examples and tutorials to help bootstrap.
 - *Transparency* : TensorLayer provides access to the **native APIs** of TensorFlow. This helps users achieve flexible controls within the training engine.
-- *Performance* : TensorLayer provides **zero-cost** abstraction (see Benchmark below). It can run on distributed and heterogeneous TensorFlow platforms with full power.
+- *Performance* : TensorLayer provides **zero-cost** abstraction. It can run on distributed and heterogeneous TensorFlow platforms with full power.
 
-## Low Runtime Overhead
+## Negligible Overhead
 
-TensorLayer has negligible overhead. We show this by benchmarking classic deep learning
+TensorLayer has negligible performance overhead. We benchmark classic deep learning
 models using TensorLayer and native TensorFlow implementations
 on a Titan X Pascal GPU. Here are the training speeds of respective tasks:
 
@@ -141,14 +146,18 @@ on a Titan X Pascal GPU. Here are the training speeds of respective tasks:
 | TensorLayer 	| 2528 images/s 	| 18063 words/s 	| 58167 words/s 	|
 | TensorFlow  	| 2530 images/s 	| 18075 words/s 	| 58181 words/s 	|
 
-## Comparing TensorLayer with Keras and TFLearn
+## Compared with Keras and TFLearn
 
-A frequent question regarding TensorLayer is what is the different with other libraries like Keras and Tflearn. 
-These libraries are comfortable to start with. They provide imperative abstractions to lower adoption barrier; 
-but in turn mask the underlying engine from users. Though good for bootstrap, 
-it becomes hard to leverage the advanced features of TensorFlow, which is quite necessary in tackling real-world problems. 
+Similar to TensorLayer, Keras and TFLearn are also popular TensorFlow wrapper libraries.
+These libraries are comfortable to start with. They provide high-level abstractions; 
+but in turn mask the underlying engine features from users. Though good for bootstrap, 
+it becomes hard to manipulate the low-level powerful features of TensorFlow, which we find quite necessary 
+in tackling many real-world problems. 
 Without compromise in simplicity, TensorLayer APIs are generally more flexible and transparent.
-Users often find it easy to start with the examples and tutorials of TensorLayer, then dive into the TensorFlow low-level APIs only if need through the transparent API design.
+Users often find it easy to start with the examples and tutorials of TensorLayer, and 
+then dive into the TensorFlow low-level APIs only if need through the transparent API design.
+TensorLayer does not intend to create library lock-in. Users can easily import models from Keras, TFSlim and TFLearn into
+a TensorLayer environment.
 
 
 # Documentation
@@ -229,7 +238,7 @@ We provide many helper functions (like `fit()` , `test()`) that is similar to Ke
 
 # Academic and Industry Users
 
-TensorLayer is in an active development stage and has received numerous contributions from an open community. 
+TensorLayer is in an active development stage and has an open community. 
 It has been widely used by researchers from Imperial College London, Carnegie Mellon University, Stanford University, 
 Tsinghua University, UCLA, Linköping University and etc., 
 as well as engineers from Google, Microsoft, Alibaba, Tencent, Penguins Innovate, ReFULE4, Bloomberg, GoodAILab and many others.

docs/modules/visualize.rst

@@ -17,7 +17,7 @@ to visualize the model, activations etc. Here we provide more functions for data
    frame
    images2d
    tsne_embedding
-
+   draw_weights
 
 Save and read images
 ----------------------
@@ -65,3 +65,7 @@ Images by matplotlib
 Visualize embeddings
 --------------------
 .. autofunction:: tsne_embedding
+
+Visualize weights
+--------------------
+.. autofunction:: draw_weights

example/tutorial_mnist.py

@@ -144,7 +144,7 @@ def main_test_layers(model='relu'):
             print("   val acc: %f" % (val_acc / n_batch))
             try:
                 # You can visualize the weight of 1st hidden layer as follow.
-                tl.vis.W(network.all_params[0].eval(), second=10, saveable=True, shape=[28, 28], name='w1_' + str(epoch + 1), fig_idx=2012)
+                tl.vis.draw_weights(network.all_params[0].eval(), second=10, saveable=True, shape=[28, 28], name='w1_' + str(epoch + 1), fig_idx=2012)
                 # You can also save the weight of 1st hidden layer to .npz file.
                 # tl.files.save_npz([network.all_params[0]] , name='w1'+str(epoch+1)+'.npz')
             except:
@@ -370,7 +370,7 @@ def main_test_stacked_denoise_AE(model='relu'):
             print("   val acc: %f" % (val_acc / n_batch))
             try:
                 # visualize the 1st hidden layer during fine-tune
-                tl.vis.W(network.all_params[0].eval(), second=10, saveable=True, shape=[28, 28], name='w1_' + str(epoch + 1), fig_idx=2012)
+                tl.vis.draw_weights(network.all_params[0].eval(), second=10, saveable=True, shape=[28, 28], name='w1_' + str(epoch + 1), fig_idx=2012)
             except:
                 print("You should change vis.W(), if you want to save the feature images for different dataset")
 

tensorlayer/cost.py

@@ -513,7 +513,7 @@ def li_regularizer(scale, scope=None):
             logging.info('Scale of 0 disables regularizer.')
             return lambda _, name=None: None
 
-    def li(weights, name=None):
+    def li(weights):
         """Applies li regularization to weights."""
         with tf.name_scope('li_regularizer') as scope:
             my_scale = ops.convert_to_tensor(scale, dtype=weights.dtype.base_dtype, name='scale')
@@ -526,7 +526,7 @@ def li(weights, name=None):
     return li
 
 
-def lo_regularizer(scale, scope=None):
+def lo_regularizer(scale):
     """Lo regularization removes the neurons of current layer. The `o` represents `outputs`
     Returns a function that can be used to apply group lo regularization to weights.
     The implementation follows `TensorFlow contrib <https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/layers/python/layers/regularizers.py>`__.
@@ -535,8 +535,6 @@ def lo_regularizer(scale, scope=None):
     ----------
     scale : float
         A scalar multiplier `Tensor`. 0.0 disables the regularizer.
-    scope: str
-        An optional scope name for this function.
 
     Returns
     -------
@@ -576,7 +574,7 @@ def lo(weights, name='lo_regularizer'):
     return lo
 
 
-def maxnorm_regularizer(scale=1.0, scope=None):
+def maxnorm_regularizer(scale=1.0):
     """Max-norm regularization returns a function that can be used to apply max-norm regularization to weights.
 
     More about max-norm, see `wiki-max norm <https://en.wikipedia.org/wiki/Matrix_norm#Max_norm>`_.
@@ -586,8 +584,6 @@ def maxnorm_regularizer(scale=1.0, scope=None):
     ----------
     scale : float
         A scalar multiplier `Tensor`. 0.0 disables the regularizer.
-    scope: str
-        An optional scope name for this function.
 
     Returns
     ---------
@@ -627,7 +623,7 @@ def mn(weights, name='max_regularizer'):
     return mn
 
 
-def maxnorm_o_regularizer(scale, scope):
+def maxnorm_o_regularizer(scale):
     """Max-norm output regularization removes the neurons of current layer.
     Returns a function that can be used to apply max-norm regularization to each column of weight matrix.
     The implementation follows `TensorFlow contrib <https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/layers/python/layers/regularizers.py>`__.
@@ -636,8 +632,6 @@ def maxnorm_o_regularizer(scale, scope):
     ----------
     scale : float
         A scalar multiplier `Tensor`. 0.0 disables the regularizer.
-    scope: str
-        An optional scope name for this function.
 
     Returns
     ---------
@@ -677,7 +671,7 @@ def mn_o(weights, name='maxnorm_o_regularizer'):
     return mn_o
 
 
-def maxnorm_i_regularizer(scale, scope=None):
+def maxnorm_i_regularizer(scale):
     """Max-norm input regularization removes the neurons of previous layer.
     Returns a function that can be used to apply max-norm regularization to each row of weight matrix.
     The implementation follows `TensorFlow contrib <https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/layers/python/layers/regularizers.py>`__.
@@ -686,8 +680,6 @@ def maxnorm_i_regularizer(scale, scope=None):
     ----------
     scale : float
         A scalar multiplier `Tensor`. 0.0 disables the regularizer.
-    scope: str
-        An optional scope name for this function.
 
     Returns
     ---------
@@ -725,6 +717,3 @@ def mn_i(weights, name='maxnorm_i_regularizer'):
             return standard_ops_fn(my_scale, standard_ops.reduce_sum(standard_ops.reduce_max(standard_ops.abs(weights), 1)), name=scope)
 
     return mn_i
-
-
-#

tensorlayer/db.py

@@ -418,119 +418,3 @@ def __str__(self):
         _s = "[TensorDB] Info:\n"
         _t = _s + "    " + str(self.db)
         return _t
-
-    # def save_bulk_data(self, data=None, filename='filename'):
-    #     """ Put bulk data into TensorDB.datafs, return file ID.
-    #     When you have a very large data, you may like to save it into GridFS Buckets
-    #     instead of Collections, then when you want to load it, XXXX
-    #
-    #     Parameters
-    #     -----------
-    #     data : serialized data.
-    #     filename : string, GridFS Buckets.
-    #
-    #     References
-    #     -----------
-    #     - MongoDB find, xxxxx
-    #     """
-    #     s = time.time()
-    #     f_id = self.datafs.put(data, filename=filename)
-    #     print("[TensorDB] save_bulk_data: {} took: {}s".format(filename, round(time.time()-s, 2)))
-    #     return f_id
-    #
-    # def save_collection(self, data=None, collect_name='collect_name'):
-    #     """ Insert data into MongoDB Collections, return xx.
-    #
-    #     Parameters
-    #     -----------
-    #     data : serialized data.
-    #     collect_name : string, MongoDB collection name.
-    #
-    #     References
-    #     -----------
-    #     - MongoDB find, xxxxx
-    #     """
-    #     s = time.time()
-    #     rl = self.db[collect_name].insert_many(data)
-    #     print("[TensorDB] save_collection: {} took: {}s".format(collect_name, round(time.time()-s, 2)))
-    #     return rl
-    #
-    # def find(self, args={}, collect_name='collect_name'):
-    #     """ Find data from MongoDB Collections.
-    #
-    #     Parameters
-    #     -----------
-    #     args : dictionary, arguments for finding.
-    #     collect_name : string, MongoDB collection name.
-    #
-    #     References
-    #     -----------
-    #     - MongoDB find, xxxxx
-    #     """
-    #     s = time.time()
-    #
-    #     pc = self.db[collect_name].find(args)  # pymongo.cursor.Cursor object
-    #     flist = pc.distinct('f_id')
-    #     fldict = {}
-    #     for f in flist: # you may have multiple Buckets files
-    #         # fldict[f] = pickle.loads(self.datafs.get(f).read())
-    #         # s2 = time.time()
-    #         tmp = self.datafs.get(f).read()
-    #         # print(time.time()-s2)
-    #         fldict[f] = pickle.loads(tmp)
-    #         # print(time.time()-s2)
-    #         # exit()
-    #     # print(round(time.time()-s, 2))
-    #     data = [fldict[x['f_id']][x['id']] for x in pc]
-    #     data = np.asarray(data)
-    #     print("[TensorDB] find: {} get: {} took: {}s".format(collect_name, pc.count(), round(time.time()-s, 2)))
-    #     return data
-
-
-class DBLogger:
-    """ """
-
-    def __init__(self, db, model):
-        self.db = db
-        self.model = model
-
-    def on_train_begin(self, logs={}):
-        print("start")
-
-    def on_train_end(self, logs={}):
-        print("end")
-
-    def on_epoch_begin(self, epoch, logs={}):
-        self.epoch = epoch
-        self.et = time.time()
-        return
-
-    def on_epoch_end(self, epoch, logs={}):
-        self.et = time.time() - self.et
-        print("ending")
-        print(epoch)
-        logs['epoch'] = epoch
-        logs['time'] = datetime.utcnow()
-        logs['stepTime'] = self.et
-        logs['acc'] = np.asscalar(logs['acc'])
-        print(logs)
-
-        w = self.model.Params
-        fid = self.db.save_params(w, logs)
-        logs.update({'params': fid})
-        self.db.valid_log(logs)
-
-    def on_batch_begin(self, batch, logs={}):
-        self.t = time.time()
-        self.losses = []
-        self.batch = batch
-
-    def on_batch_end(self, batch, logs={}):
-        self.t2 = time.time() - self.t
-        logs['acc'] = np.asscalar(logs['acc'])
-        #logs['loss']=np.asscalar(logs['loss'])
-        logs['step_time'] = self.t2
-        logs['time'] = datetime.utcnow()
-        logs['epoch'] = self.epoch
-        logs['batch'] = self.batch
-        self.db.train_log(logs)

tensorlayer/files.py

@@ -117,7 +117,7 @@ def load_mnist_labels(path, filename):
     return X_train, y_train, X_val, y_val, X_test, y_test
 
 
-def load_cifar10_dataset(shape=(-1, 32, 32, 3), path='data', plotable=False, second=3):
+def load_cifar10_dataset(shape=(-1, 32, 32, 3), path='data', plotable=False):
     """Load CIFAR-10 dataset.
 
     It consists of 60000 32x32 colour images in 10 classes, with
@@ -137,8 +137,6 @@ def load_cifar10_dataset(shape=(-1, 32, 32, 3), path='data', plotable=False, sec
         The path that the data is downloaded to, defaults is ``data/cifar10/``.
     plotable : boolean
         Whether to plot some image examples, False as default.
-    second : int
-        If ``plotable`` is True, it is the display time.
 
     Examples
     --------
@@ -1791,4 +1789,4 @@ def npz_to_W_pdf(path=None, regx='w1pre_[0-9]+\.(npz)'):
     for f in file_list:
         W = load_npz(path, f)[0]
         logging.info("%s --> %s" % (f, f.split('.')[0] + '.pdf'))
-        visualize.W(W, second=10, saveable=True, name=f.split('.')[0], fig_idx=2012)
+        visualize.draw_weights(W, second=10, saveable=True, name=f.split('.')[0], fig_idx=2012)