Skip to content
This repository was archived by the owner on Jul 10, 2025. It is now read-only.

Commit 2ff42e2

Browse files
committed
[determinism] Improve list of ops in RFC
1 parent 41a8fbf commit 2ff42e2

File tree

1 file changed

+15
-8
lines changed

1 file changed

+15
-8
lines changed

rfcs/20210119-determinism.md

Lines changed: 15 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -101,15 +101,22 @@ We must ensure that every op will either run deterministically or raise an error
101101

102102
As part of the implementation, we will review all ops to make a determination of their behavior (deterministic vs nondeterministic). Ops that are known to operate nondeterministically, at least when running on a GPU, include the following:
103103

104-
* `tf.nn.softmax_cross_entropy_with_logits`
105-
* `tf.nn.sparse_softmax_cross_entropy_with_logits`
106-
* `tf.image.resize` gradient with `method=ResizeMethod.NEAREST`
107-
* `tf.math.segment_sum`, `tf.math.unsorted_segment_sum` forward
108-
* `tf.image.crop_and_resize` gradient to both image and boxes
109-
* `tf.math.unsorted_segment_mean`, `tf.math.unsorted_segment_prod` and `tf.math.unsorted_segment_sqrt`; all forward
110-
* `tf.sparse.sparse_dense_matmul`
111-
* `tf.image.adjust_contrast`
104+
* `tfa.image.dense_image_warp` gradient w.r.t `image`
112105
* `tf.compat.v1.nn.fused_batch_norm` gradient w.r.t. `offset`
106+
* `tf.convert_to_tensor` forward, when `value` is of type `tf.IndexedSlices`
107+
* `tf.gather` gradient w.r.t dense `params`
108+
* `tf.image.resize` gradient w.r.t `image`, when `method=ResizeMethod.NEAREST`
109+
* `tf.image.crop_and_resize` gradient w.r.t both `image` and `boxes`
110+
* `tf.image.adjust_contrast` forward
111+
* `tf.math_segment_prod` forward
112+
* `tf.math.segment_sum` forward
113+
* `tf.math.unsorted_segment_mean` forward
114+
* `tf.math.unsorted_segment_sqrt_n` forward
115+
* `tf.math.unsorted_segment_prod` forward
116+
* `tf.math.unsorted_segment_sum` forward
117+
* `tf.nn.softmax_cross_entropy_with_logits` forward
118+
* `tf.nn.sparse_softmax_cross_entropy_with_logits` forward
119+
* `tf.sparse.sparse_dense_matmul` forward
113120

114121
We have a list of other ops that use CUDA's `atomicAdd` and are therefore likely to be sources of nondeterminism. Once it has been confirmed that those ops function nondeterministically, they will be made to throw errors when determinism is enabled. In the long term, we can add a deterministic implementation to such ops.
115122

0 commit comments

Comments
 (0)