Skip to content

Commit 82849ad

Browse files
authored
Merge pull request #5678 from ranqiu92/doc
Update annotations of layers.py.
2 parents 0ed5a55 + bf4b532 commit 82849ad

File tree

1 file changed

+85
-81
lines changed
  • python/paddle/trainer_config_helpers

1 file changed

+85
-81
lines changed

python/paddle/trainer_config_helpers/layers.py

Lines changed: 85 additions & 81 deletions
Original file line numberDiff line numberDiff line change
@@ -2987,8 +2987,10 @@ def img_cmrnorm_layer(input,
29872987
layer_attr=None):
29882988
"""
29892989
Response normalization across feature maps.
2990-
The details please refer to
2991-
`Alex's paper <http://www.cs.toronto.edu/~fritz/absps/imagenet.pdf>`_.
2990+
2991+
Reference:
2992+
ImageNet Classification with Deep Convolutional Neural Networks
2993+
http://www.cs.toronto.edu/~fritz/absps/imagenet.pdf
29922994
29932995
The example usage is:
29942996
@@ -2997,7 +2999,7 @@ def img_cmrnorm_layer(input,
29972999
norm = img_cmrnorm_layer(input=net, size=5)
29983000
29993001
:param name: The name of this layer. It is optional.
3000-
:type name: None | basestring
3002+
:type name: basestring
30013003
:param input: The input of this layer.
30023004
:type input: LayerOutput
30033005
:param size: Normalize in number of :math:`size` feature maps.
@@ -3006,9 +3008,11 @@ def img_cmrnorm_layer(input,
30063008
:type scale: float
30073009
:param power: The hyper-parameter.
30083010
:type power: float
3009-
:param num_channels: input layer's filers number or channels. If
3010-
num_channels is None, it will be set automatically.
3011-
:param layer_attr: Extra Layer Attribute.
3011+
:param num_channels: The number of input channels. If the parameter is not set or
3012+
set to None, its actual value will be automatically set to
3013+
the channels number of the input.
3014+
:param layer_attr: The extra layer attributes. See ExtraLayerAttribute for
3015+
details.
30123016
:type layer_attr: ExtraLayerAttribute
30133017
:return: LayerOutput object.
30143018
:rtype: LayerOutput
@@ -3036,7 +3040,7 @@ def batch_norm_layer(input,
30363040
use_global_stats=None,
30373041
mean_var_names=None):
30383042
"""
3039-
Batch Normalization Layer. The notation of this layer as follow.
3043+
Batch Normalization Layer. The notation of this layer is as follows.
30403044
30413045
:math:`x` is the input features over a mini-batch.
30423046
@@ -3050,8 +3054,10 @@ def batch_norm_layer(input,
30503054
\\sigma_{\\beta}^{2} + \\epsilon}} \\qquad &//\ normalize \\\\
30513055
y_i &\\gets \\gamma \\hat{x_i} + \\beta \\qquad &//\ scale\ and\ shift
30523056
3053-
The details of batch normalization please refer to this
3054-
`paper <http://arxiv.org/abs/1502.03167>`_.
3057+
Reference:
3058+
Batch Normalization: Accelerating Deep Network Training by Reducing
3059+
Internal Covariate Shift
3060+
http://arxiv.org/abs/1502.03167
30553061
30563062
The example usage is:
30573063
@@ -3061,48 +3067,47 @@ def batch_norm_layer(input,
30613067
30623068
:param name: The name of this layer. It is optional.
30633069
:type name: basestring
3064-
:param input: batch normalization input. Better be linear activation.
3065-
Because there is an activation inside batch_normalization.
3070+
:param input: This layer's input which is to be performed batch normalization on.
30663071
:type input: LayerOutput
30673072
:param batch_norm_type: We have batch_norm, mkldnn_batch_norm and cudnn_batch_norm.
30683073
batch_norm supports CPU, MKLDNN and GPU. cudnn_batch_norm
30693074
requires cuDNN version greater or equal to v4 (>=v4).
30703075
But cudnn_batch_norm is faster and needs less
30713076
memory than batch_norm. mkldnn_batch_norm requires
3072-
enable use_mkldnn. By default (None), we will
3073-
automaticly select cudnn_batch_norm for GPU,
3077+
use_mkldnn is enabled. By default (None), we will
3078+
automatically select cudnn_batch_norm for GPU,
30743079
mkldnn_batch_norm for MKLDNN and batch_norm for CPU.
3075-
Otherwise, select batch norm type based on the
3076-
specified type. If you use cudnn_batch_norm,
3077-
we suggested you use latest version, such as v5.1.
3080+
Users can specify the batch norm type. If you use
3081+
cudnn_batch_norm, we suggested you use latest version,
3082+
such as v5.1.
30783083
:type batch_norm_type: None | string, None or "batch_norm" or "cudnn_batch_norm"
30793084
or "mkldnn_batch_norm"
3080-
:param act: Activation Type. Better be relu. Because batch
3081-
normalization will normalize input near zero.
3085+
:param act: Activation type. ReluActivation is the default activation.
30823086
:type act: BaseActivation
3083-
:param num_channels: num of image channels or previous layer's number of
3084-
filters. None will automatically get from layer's
3085-
input.
3087+
:param num_channels: The number of input channels. If the parameter is not set or
3088+
set to None, its actual value will be automatically set to
3089+
the channels number of the input.
30863090
:type num_channels: int
3087-
:param bias_attr: :math:`\\beta`, better be zero when initialize. So the
3088-
initial_std=0, initial_mean=1 is best practice.
3091+
:param bias_attr: :math:`\\beta`. The bias attribute. If the parameter is set to
3092+
False or an object whose type is not ParameterAttribute, no
3093+
bias is defined. If the parameter is set to True, the bias is
3094+
initialized to zero.
30893095
:type bias_attr: ParameterAttribute | None | bool | Any
3090-
:param param_attr: :math:`\\gamma`, better be one when initialize. So the
3091-
initial_std=0, initial_mean=1 is best practice.
3096+
:param param_attr: :math:`\\gamma`. The parameter attribute. See ParameterAttribute
3097+
for details.
30923098
:type param_attr: ParameterAttribute
3093-
:param layer_attr: Extra Layer Attribute.
3099+
:param layer_attr: The extra layer attribute. See ExtraLayerAttribute for
3100+
details.
30943101
:type layer_attr: ExtraLayerAttribute
3095-
:param use_global_stats: whether use moving mean/variance statistics
3096-
during testing peroid. If None or True,
3097-
it will use moving mean/variance statistics during
3098-
testing. If False, it will use the mean
3099-
and variance of current batch of test data for
3100-
testing.
3102+
:param use_global_stats: Whether use moving mean/variance statistics during
3103+
testing peroid. If the parameter is set to None or
3104+
True, it will use moving mean/variance statistics
3105+
during testing. If the parameter is set to False, it
3106+
will use the mean and variance of the current batch
3107+
of test data.
31013108
:type use_global_stats: bool | None.
3102-
:param moving_average_fraction: Factor used in the moving average
3103-
computation, referred to as facotr,
3104-
:math:`runningMean = newMean*(1-factor)
3105-
+ runningMean*factor`
3109+
:param moving_average_fraction: Factor used in the moving average computation.
3110+
:math:`runningMean = newMean*(1-factor) + runningMean*factor`
31063111
:type moving_average_fraction: float.
31073112
:param mean_var_names: [mean name, variance name]
31083113
:type mean_var_names: string list
@@ -3164,8 +3169,9 @@ def sum_to_one_norm_layer(input, name=None, layer_attr=None):
31643169
:type input: LayerOutput
31653170
:param name: The name of this layer. It is optional.
31663171
:type name: basestring
3167-
:param layer_attr: extra layer attributes.
3168-
:type layer_attr: ExtraLayerAttribute.
3172+
:param layer_attr: The extra layer attribute. See ExtraLayerAttribute
3173+
for details.
3174+
:type layer_attr: ExtraLayerAttribute
31693175
:return: LayerOutput object.
31703176
:rtype: LayerOutput
31713177
"""
@@ -3200,7 +3206,8 @@ def row_l2_norm_layer(input, name=None, layer_attr=None):
32003206
:type input: LayerOutput
32013207
:param name: The name of this layer. It is optional.
32023208
:type name: basestring
3203-
:param layer_attr: extra layer attributes.
3209+
:param layer_attr: The extra layer attribute. See ExtraLayerAttribute
3210+
for details.
32043211
:type layer_attr: ExtraLayerAttribute.
32053212
:return: LayerOutput object.
32063213
:rtype: LayerOutput
@@ -3237,22 +3244,17 @@ def addto_layer(input, act=None, name=None, bias_attr=None, layer_attr=None):
32373244
act=ReluActivation(),
32383245
bias_attr=False)
32393246
3240-
This layer just simply add all input layers together, then activate the sum
3241-
inputs. Each input of this layer should be the same size, which is also the
3242-
output size of this layer.
3247+
This layer just simply adds all input layers together, then activates the
3248+
sum. All inputs should share the same dimension, which is also the dimension
3249+
of this layer's output.
32433250
32443251
There is no weight matrix for each input, because it just a simple add
32453252
operation. If you want a complicated operation before add, please use
32463253
mixed_layer.
32473254
3248-
It is a very good way to set dropout outside the layers. Since not all
3249-
PaddlePaddle layer support dropout, you can add an add_to layer, set
3250-
dropout here.
3251-
Please refer to dropout_layer for details.
3252-
32533255
:param name: The name of this layer. It is optional.
32543256
:type name: basestring
3255-
:param input: Input layers. It could be a LayerOutput or list/tuple of
3257+
:param input: The input layers. It could be a LayerOutput or list/tuple of
32563258
LayerOutput.
32573259
:type input: LayerOutput | list | tuple
32583260
:param act: Activation Type. LinearActivation is the default activation.
@@ -3261,7 +3263,8 @@ def addto_layer(input, act=None, name=None, bias_attr=None, layer_attr=None):
32613263
whose type is not ParameterAttribute, no bias is defined. If the
32623264
parameter is set to True, the bias is initialized to zero.
32633265
:type bias_attr: ParameterAttribute | None | bool | Any
3264-
:param layer_attr: Extra Layer attribute.
3266+
:param layer_attr: The extra layer attribute. See ExtraLayerAttribute for
3267+
details.
32653268
:type layer_attr: ExtraLayerAttribute
32663269
:return: LayerOutput object.
32673270
:rtype: LayerOutput
@@ -3300,8 +3303,8 @@ def addto_layer(input, act=None, name=None, bias_attr=None, layer_attr=None):
33003303
@layer_support(DROPOUT, ERROR_CLIPPING)
33013304
def concat_layer(input, act=None, name=None, layer_attr=None, bias_attr=None):
33023305
"""
3303-
Concat all input vector into one huge vector.
3304-
Inputs can be list of LayerOutput or list of projection.
3306+
Concatenate all input vectors to one vector.
3307+
Inputs can be a list of LayerOutput or a list of projection.
33053308
33063309
The example usage is:
33073310
@@ -3311,11 +3314,12 @@ def concat_layer(input, act=None, name=None, layer_attr=None, bias_attr=None):
33113314
33123315
:param name: The name of this layer. It is optional.
33133316
:type name: basestring
3314-
:param input: input layers or projections
3317+
:param input: The input layers or projections
33153318
:type input: list | tuple | collections.Sequence
33163319
:param act: Activation type. IdentityActivation is the default activation.
33173320
:type act: BaseActivation
3318-
:param layer_attr: Extra Layer Attribute.
3321+
:param layer_attr: The extra layer attribute. See ExtraLayerAttribute for
3322+
details.
33193323
:type layer_attr: ExtraLayerAttribute
33203324
:return: LayerOutput object.
33213325
:rtype: LayerOutput
@@ -3385,7 +3389,7 @@ def __reduce_concat_type__(a, b):
33853389
def seq_concat_layer(a, b, act=None, name=None, layer_attr=None,
33863390
bias_attr=None):
33873391
"""
3388-
Concat sequence a with sequence b.
3392+
Concatenate sequence a and sequence b.
33893393
33903394
Inputs:
33913395
- a = [a1, a2, ..., am]
@@ -3404,13 +3408,14 @@ def seq_concat_layer(a, b, act=None, name=None, layer_attr=None,
34043408
34053409
:param name: The name of this layer. It is optional.
34063410
:type name: basestring
3407-
:param a: input sequence layer
3411+
:param a: The first input sequence layer
34083412
:type a: LayerOutput
3409-
:param b: input sequence layer
3413+
:param b: The second input sequence layer
34103414
:type b: LayerOutput
34113415
:param act: Activation type. IdentityActivation is the default activation.
34123416
:type act: BaseActivation
3413-
:param layer_attr: Extra Layer Attribute.
3417+
:param layer_attr: The extra layer attribute. See ExtraLayerAttribute for
3418+
details.
34143419
:type layer_attr: ExtraLayerAttribute
34153420
:param bias_attr: The bias attribute. If the parameter is set to False or an object
34163421
whose type is not ParameterAttribute, no bias is defined. If the
@@ -3447,58 +3452,57 @@ def memory(name,
34473452
boot_bias_active_type=None,
34483453
boot_with_const_id=None):
34493454
"""
3450-
The memory layers is a layer cross each time step. Reference this output
3451-
as previous time step layer :code:`name` 's output.
3455+
The memory takes a layer's output at previous time step as its own output.
34523456
3453-
The default memory is zero in first time step, previous time step's
3454-
output in the rest time steps.
3457+
If boot_bias, the activation of the bias is the initial value of the memory.
34553458
3456-
If boot_bias, the first time step value is this bias and
3457-
with activation.
3459+
If boot_with_const_id is set, then the memory's output at the first time step
3460+
is a IndexSlot, the Arguments.ids()[0] is this :code:`cost_id`.
34583461
3459-
If boot_with_const_id, then the first time stop is a IndexSlot, the
3460-
Arguments.ids()[0] is this :code:`cost_id`.
3462+
If boot_layer is specified, the memory's output at the first time step will
3463+
be the boot_layer's output.
34613464
3462-
If boot_layer is not null, the memory is just the boot_layer's output.
3463-
Set :code:`is_seq` is true boot layer is sequence.
3464-
3465-
The same name layer in recurrent group will set memory on each time
3466-
step.
3465+
In other case, the default memory's output at the first time step is zero.
34673466
34683467
.. code-block:: python
34693468
34703469
mem = memory(size=256, name='state')
34713470
state = fc_layer(input=mem, size=256, name='state')
34723471
3473-
If you do not want to specify the name, you can equivalently use set_input()
3474-
to specify the layer needs to be remembered as the following:
3472+
If you do not want to specify the name, you can also use set_input()
3473+
to specify the layer to be remembered as the following:
34753474
34763475
.. code-block:: python
34773476
34783477
mem = memory(size=256)
34793478
state = fc_layer(input=mem, size=256)
34803479
mem.set_input(mem)
34813480
3482-
:param name: the name of the layer which this memory remembers.
3481+
:param name: The name of the layer which this memory remembers.
34833482
If name is None, user should call set_input() to specify the
34843483
name of the layer which this memory remembers.
34853484
:type name: basestring
3486-
:param size: size of memory.
3485+
:param size: The dimensionality of memory.
34873486
:type size: int
3488-
:param memory_name: the name of the memory.
3489-
It is ignored when name is provided.
3487+
:param memory_name: The name of the memory. It is ignored when name is provided.
34903488
:type memory_name: basestring
34913489
:param is_seq: DEPRECATED. is sequence for boot_layer
34923490
:type is_seq: bool
3493-
:param boot_layer: boot layer of memory.
3491+
:param boot_layer: This parameter specifies memory's output at the first time
3492+
step and the output is boot_layer's output.
34943493
:type boot_layer: LayerOutput | None
3495-
:param boot_bias: boot layer's bias
3494+
:param boot_bias: The bias attribute of memory's output at the first time step.
3495+
If the parameter is set to False or an object whose type is not
3496+
ParameterAttribute, no bias is defined. If the parameter is set
3497+
to True, the bias is initialized to zero.
34963498
:type boot_bias: ParameterAttribute | None
3497-
:param boot_bias_active_type: boot layer's active type.
3499+
:param boot_bias_active_type: Activation type for memory's bias at the first time
3500+
step. LinearActivation is the default activation.
34983501
:type boot_bias_active_type: BaseActivation
3499-
:param boot_with_const_id: boot layer's id.
3502+
:param boot_with_const_id: This parameter specifies memory's output at the first
3503+
time step and the output is an index.
35003504
:type boot_with_const_id: int
3501-
:return: LayerOutput object which is a memory.
3505+
:return: LayerOutput object.
35023506
:rtype: LayerOutput
35033507
"""
35043508
if boot_bias_active_type is None:

0 commit comments

Comments
 (0)