Skip to content

Commit e973541

Browse files
peterzhang2029peterzhang2029
committed
unify the bias
1 parent 2b93934 commit e973541

File tree

2 files changed

+77
-60
lines changed

2 files changed

+77
-60
lines changed

‎python/paddle/trainer_config_helpers/layers.py

Lines changed: 28 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -3649,9 +3649,10 @@ def gru_step_layer(input,
36493649
:param name: The name of this layer. It is optional.
36503650
:param gate_act: Activation type of this layer's two gates. Default is Sigmoid.
36513651
:type gate_act: BaseActivation
3652-
:param bias_attr: The bias attribute. If the parameter is set to False or an object
3653-
whose type is not ParameterAttribute, no bias is defined. If the
3654-
parameter is set to True, the bias is initialized to zero.
3652+
:param bias_attr: The parameter attribute for bias. If this parameter is set to
3653+
False or an object whose type is not ParameterAttribute, no bias
3654+
is defined. If this parameter is set to True,
3655+
the bias is initialized to zero.
36553656
:type bias_attr: ParameterAttribute | None | bool | Any
36563657
:param param_attr: the parameter_attribute for transforming the output_mem
36573658
from previous step.
@@ -3711,9 +3712,10 @@ def gru_step_naive_layer(input,
37113712
:type act: BaseActivation
37123713
:param gate_act: Activation type of this layer's two gates. Default is Sigmoid.
37133714
:type gate_act: BaseActivation
3714-
:param bias_attr: The bias attribute. If the parameter is set to False or an object
3715-
whose type is not ParameterAttribute, no bias is defined. If the
3716-
parameter is set to True, the bias is initialized to zero.
3715+
:param bias_attr: The parameter attribute for bias. If this parameter is set to
3716+
False or an object whose type is not ParameterAttribute, no bias
3717+
is defined. If this parameter is set to True,
3718+
the bias is initialized to zero.
37173719
:type bias_attr: ParameterAttribute | None | bool | Any
37183720
:param param_attr:
37193721
:param layer_attr:
@@ -3843,9 +3845,10 @@ def recurrent_layer(input,
38433845
:type input: LayerOutput
38443846
:param act: Activation type. TanhActivation is the default.
38453847
:type act: BaseActivation
3846-
:param bias_attr: The bias attribute. If the parameter is set to False or an object
3847-
whose type is not ParameterAttribute, no bias is defined. If the
3848-
parameter is set to True, the bias is initialized to zero.
3848+
:param bias_attr: The parameter attribute for bias. If this parameter is set to
3849+
False or an object whose type is not ParameterAttribute,
3850+
no bias is defined. If the parameter is set to True,
3851+
the bias is initialized to zero.
38493852
:type bias_attr: ParameterAttribute | None | bool | Any
38503853
:param param_attr: parameter attribute.
38513854
:type param_attr: ParameterAttribute
@@ -4835,9 +4838,10 @@ def tensor_layer(a,
48354838
:type act: BaseActivation
48364839
:param param_attr: The Parameter Attribute.
48374840
:type param_attr: ParameterAttribute
4838-
:param bias_attr: The bias attribute. If the parameter is set to False or an object
4839-
whose type is not ParameterAttribute, no bias is defined. If the
4840-
parameter is set to True, the bias is initialized to zero.
4841+
:param bias_attr: The parameter attribute for bias. If this parameter is set to
4842+
False or an object whose type is not ParameterAttribute,
4843+
no bias is defined. If this parameter is set to True,
4844+
the bias is initialized to zero.
48414845
:type bias_attr: ParameterAttribute | None | bool | Any
48424846
:param layer_attr: Extra Layer config.
48434847
:type layer_attr: ExtraLayerAttribute | None
@@ -4899,9 +4903,10 @@ def selective_fc_layer(input,
48994903
:type act: BaseActivation
49004904
:param param_attr: The Parameter Attribute.
49014905
:type param_attr: ParameterAttribute
4902-
:param bias_attr: The bias attribute. If the parameter is set to False or an object
4903-
whose type is not ParameterAttribute, no bias is defined. If the
4904-
parameter is set to True, the bias is initialized to zero.
4906+
:param bias_attr: The parameter attribute for bias. If this parameter is set to
4907+
False or an object whose type is not ParameterAttribute,
4908+
no bias is defined. If this parameter is set to True,
4909+
the bias is initialized to zero.
49054910
:type bias_attr: ParameterAttribute | None | bool | Any
49064911
:param layer_attr: Extra Layer config.
49074912
:type layer_attr: ExtraLayerAttribute | None
@@ -5584,10 +5589,10 @@ def nce_layer(input,
55845589
to the num_classes. Each member of the list defines
55855590
the probability of a class given input x.
55865591
:type neg_distribution: list | tuple | collections.Sequence | None
5587-
:param bias_attr: The attribute for bias. If this parameter is set False or
5588-
any object whose type is not ParameterAttribute, no bias
5589-
is added. If this parameter is set True, the bias is
5590-
initialized to zero.
5592+
:param bias_attr: The parameter attribute for bias. If this parameter is set to
5593+
False or an object whose type is not ParameterAttribute,
5594+
no bias is defined. If this parameter is set to True,
5595+
the bias is initialized to zero.
55915596
:type bias_attr: ParameterAttribute | None | bool | Any
55925597
:param layer_attr: Extra Layer Attribute.
55935598
:type layer_attr: ExtraLayerAttribute
@@ -6497,19 +6502,19 @@ def gated_unit_layer(input,
64976502
:param gate_param_attr: The parameter attribute of the gate. See ParameterAttribute
64986503
for details.
64996504
:type gate_param_attr: ParameterAttribute
6500-
:param gate_bias_attr: The bias attribute of the gate. If the parameter is set to False or
6505+
:param gate_bias_attr: The bias attribute of the gate. If this parameter is set to False or
65016506
an object whose type is not ParameterAttribute, no bias is defined.
6502-
If the parameter is set to True, the bias is initialized to zero.
6507+
If this parameter is set to True, the bias is initialized to zero.
65036508
:type gate_bias_attr: ParameterAttribute | bool | None | Any
65046509
:param inproj_attr: Extra layer attributes of the projection. See ExtraLayerAttribute for
65056510
details.
65066511
:type inproj_attr: ExtraLayerAttribute | None
65076512
:param inproj_param_attr: The parameter attribute of the projection. See ParameterAttribute
65086513
for details.
65096514
:type inproj_param_attr: ParameterAttribute
6510-
:param inproj_bias_attr: The bias attribute of the projection. If the parameter is set to False
6515+
:param inproj_bias_attr: The bias attribute of the projection. If this parameter is set to False
65116516
or an object whose type is not ParameterAttribute, no bias is defined.
6512-
If the parameter is set to True, the bias is initialized to zero.
6517+
If this parameter is set to True, the bias is initialized to zero.
65136518
:type inproj_bias_attr: ParameterAttribute | bool | None | Any
65146519
:param layer_attr: Extra layer attribute of the product. See ExtraLayerAttribute for
65156520
details.

‎python/paddle/trainer_config_helpers/networks.py

Lines changed: 49 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -681,36 +681,42 @@ def lstmemory_unit(input,
681681
state_act=TanhActivation())
682682
683683
684-
:param input: input layer.
684+
:param input: Input layer.
685685
:type input: LayerOutput
686-
:param out_memory: output of previous time step
686+
:param out_memory: The output of previous time step.
687687
:type out_memory: LayerOutput | None
688-
:param name: lstmemory unit name.
688+
:param name: The lstmemory unit name.
689689
:type name: basestring
690-
:param size: lstmemory unit size.
690+
:param size: The lstmemory unit size.
691691
:type size: int
692-
:param param_attr: parameter attribute, None means default attribute.
692+
:param param_attr: The parameter attribute for the weights in
693+
input to hidden projection.
694+
None means default attribute.
693695
:type param_attr: ParameterAttribute
694-
:param act: last activiation type of lstm.
696+
:param act: The last activiation type of lstm.
695697
:type act: BaseActivation
696-
:param gate_act: gate activiation type of lstm.
698+
:param gate_act: The gate activiation type of lstm.
697699
:type gate_act: BaseActivation
698-
:param state_act: state activiation type of lstm.
700+
:param state_act: The state activiation type of lstm.
699701
:type state_act: BaseActivation
700-
:param input_proj_bias_attr: bias attribute for input to hidden projection.
701-
False or None means no bias. If the parameter is set to True,
702-
the bias is initialized to zero.
702+
:param input_proj_bias_attr: The parameter attribute for the bias in
703+
input to hidden projection.
704+
False or None means no bias.
705+
If this parameter is set to True,
706+
the bias is initialized to zero.
703707
:type input_proj_bias_attr: ParameterAttribute|bool|None
704-
:param input_proj_layer_attr: extra layer attribute for input to hidden
705-
projection of the LSTM unit, such as dropout, error clipping.
708+
:param input_proj_layer_attr: The extra layer attribute for
709+
input to hidden projection of the LSTM unit,
710+
such as dropout, error clipping.
706711
:type input_proj_layer_attr: ExtraLayerAttribute
707-
:param lstm_bias_attr: bias parameter attribute of lstm layer.
708-
If the parameter is set to True or None,
709-
the bias is initialized to zero.
712+
:param lstm_bias_attr: The parameter attribute for the bias in lstm layer.
713+
False or None means no bias.
714+
If this parameter is set to True,
715+
the bias is initialized to zero.
710716
:type lstm_bias_attr: ParameterAttribute|True|None
711-
:param lstm_layer_attr: extra attribute of lstm layer.
717+
:param lstm_layer_attr: The extra attribute of lstm layer.
712718
:type lstm_layer_attr: ExtraLayerAttribute
713-
:return: lstmemory unit name.
719+
:return: The lstmemory unit name.
714720
:rtype: LayerOutput
715721
"""
716722
if size is None:
@@ -788,36 +794,42 @@ def lstmemory_group(input,
788794
gate_act=SigmoidActivation(),
789795
state_act=TanhActivation())
790796
791-
:param input: input layer.
797+
:param input: Input layer.
792798
:type input: LayerOutput
793-
:param size: lstmemory group size.
799+
:param size: The lstmemory group size.
794800
:type size: int
795-
:param name: name of lstmemory group.
801+
:param name: The name of lstmemory group.
796802
:type name: basestring
797-
:param out_memory: output of previous time step.
803+
:param out_memory: The output of previous time step.
798804
:type out_memory: LayerOutput | None
799-
:param reverse: process the input in a reverse order or not.
805+
:param reverse: Process the input in a reverse order or not.
800806
:type reverse: bool
801-
:param param_attr: parameter attribute, None means default attribute.
807+
:param param_attr: The parameter attribute for the weights in
808+
input to hidden projection.
809+
None means default attribute.
802810
:type param_attr: ParameterAttribute
803-
:param act: last activiation type of lstm.
811+
:param act: The last activiation type of lstm.
804812
:type act: BaseActivation
805-
:param gate_act: gate activiation type of lstm.
813+
:param gate_act: The gate activiation type of lstm.
806814
:type gate_act: BaseActivation
807-
:param state_act: state activiation type of lstm.
815+
:param state_act: The state activiation type of lstm.
808816
:type state_act: BaseActivation
809-
:param lstm_bias_attr: bias parameter attribute of lstm layer.
810-
If the parameter is set to True or None, the bias is
811-
initialized to zero.
812-
:type lstm_bias_attr: ParameterAttribute|True|None
813-
:param input_proj_bias_attr: bias attribute for input to hidden projection.
814-
False or None means no bias. If the parameter is set to True,
815-
the bias is initialized to zero.
817+
:param input_proj_bias_attr: The parameter attribute for the bias in
818+
input to hidden projection.
819+
False or None means no bias.
820+
If this parameter is set to True,
821+
the bias is initialized to zero.
816822
:type input_proj_bias_attr: ParameterAttribute|bool|None
817-
:param input_proj_layer_attr: extra layer attribute for input to hidden
818-
projection of the LSTM unit, such as dropout, error clipping.
823+
:param input_proj_layer_attr: The extra layer attribute for
824+
input to hidden projection of the LSTM unit,
825+
such as dropout, error clipping.
819826
:type input_proj_layer_attr: ExtraLayerAttribute
820-
:param lstm_layer_attr: lstm layer's extra attribute.
827+
:param lstm_bias_attr: The parameter attribute for the bias in lstm layer.
828+
False or None means no bias.
829+
If this parameter is set to True,
830+
the bias is initialized to zero.
831+
:type lstm_bias_attr: ParameterAttribute|True|None
832+
:param lstm_layer_attr: The extra attribute of lstm layer.
821833
:type lstm_layer_attr: ExtraLayerAttribute
822834
:return: the lstmemory group.
823835
:rtype: LayerOutput

0 commit comments

Comments
 (0)