Skip to content

Commit c05e0c7

Browse files
authored
Align subclassing guide with docstring of Layer.add_loss() (#1144)
* Align subclassing guide with docstring of Layer.add_loss() which uses `reduce_mean()`, not `reduce_sum()`, to achieve independence of batch sizes, consistent with Keras' default loss reduction by SUM_OVER_BATCH_SIZE. Along the way, clarify the difference between `add_loss` and `Loss`. * Update making_new_layers_and_models_via_subclassing.py Addresses review comment.
1 parent d3bec2e commit c05e0c7

File tree

1 file changed

+4
-1
lines changed

1 file changed

+4
-1
lines changed

guides/making_new_layers_and_models_via_subclassing.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -236,11 +236,14 @@ def __init__(self, rate=1e-2):
236236
self.rate = rate
237237

238238
def call(self, inputs):
239-
self.add_loss(self.rate * tf.reduce_sum(inputs))
239+
self.add_loss(self.rate * tf.reduce_mean(inputs))
240240
return inputs
241241

242242

243243
"""
244+
Notice that `add_loss()` can take the result of plain TensorFlow operations.
245+
There is no need to call a `Loss` object here.
246+
244247
These losses (including those created by any inner layer) can be retrieved via
245248
`layer.losses`. This property is reset at the start of every `__call__()` to
246249
the top-level layer, so that `layer.losses` always contains the loss values

0 commit comments

Comments
 (0)