Skip to content

Commit 081fd41

Browse files
authored
Merge pull request #13944 from sneaxiy/add_layer_norm_doc
Add layer_norm doc to illustrate param_attr and bias_attr
2 parents fe6c24a + acd77e6 commit 081fd41

File tree

1 file changed

+16
-7
lines changed
  • python/paddle/fluid/layers

1 file changed

+16
-7
lines changed

python/paddle/fluid/layers/nn.py

Lines changed: 16 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -2317,19 +2317,28 @@ def layer_norm(input,
23172317
Args:
23182318
input(Variable): The input tensor variable.
23192319
scale(bool): Whether to learn the adaptive gain :math:`g` after
2320-
normalization.
2320+
normalization. Default True.
23212321
shift(bool): Whether to learn the adaptive bias :math:`b` after
2322-
normalization.
2323-
begin_norm_axis(bool): The normalization will be performed along
2322+
normalization. Default True.
2323+
begin_norm_axis(int): The normalization will be performed along
23242324
dimensions from :attr:`begin_norm_axis` to :attr:`rank(input)`.
2325+
Default 1.
23252326
epsilon(float): The small value added to the variance to prevent
2326-
division by zero.
2327+
division by zero. Default 1e-05.
23272328
param_attr(ParamAttr|None): The parameter attribute for the learnable
2328-
gain :math:`g`.
2329+
gain :math:`g`. If :attr:`scale` is False, :attr:`param_attr` is
2330+
omitted. If :attr:`scale` is True and :attr:`param_attr` is None,
2331+
a default :code:`ParamAttr` would be added as scale. The
2332+
:attr:`param_attr` is initialized as 1 if it is added. Default None.
23292333
bias_attr(ParamAttr|None): The parameter attribute for the learnable
2330-
bias :math:`b`.
2334+
bias :math:`b`. If :attr:`shift` is False, :attr:`bias_attr` is
2335+
omitted. If :attr:`shift` is True and :attr:`param_attr` is None,
2336+
a default :code:`ParamAttr` would be added as bias. The
2337+
:attr:`bias_attr` is initialized as 0 if it is added. Default None.
23312338
act(str): Activation to be applied to the output of layer normalizaiton.
2332-
name (str): The name of this layer. It is optional.
2339+
Default None.
2340+
name(str): The name of this layer. It is optional. Default None, and a
2341+
unique name would be generated automatically.
23332342
23342343
Returns:
23352344
${y_comment}

0 commit comments

Comments
 (0)