TensorFlow lesser known features

I sometimes stumble upon features in TensorFlow which look potentially powerful or interesting, which I did not know about before. This wiki page is supposed to be a collection of such features. (Once some of them becomes more central to RETURNN, we might want to have a separate wiki page for it. E.g. Distributed TensorFlow. To find them, it is interesting to just browse through tf.raw_ops.

Operations / functions

• tf.math.divide_no_nan/tf.div_no_nan. Computes x/y and return 0 if y==0 (no nan).
• tf.math.reduce_euclidean_norm
• tf.switch_case and tf.case
• tf.strings.to_hash_bucket
• tf.strings.ngrams
• tf.compat.v1.get_session_handle. keep tensor alive in the session, and access it in future calls via get_session_tensor, or use it for feed_dict

Resources

• DenseHashTable (and other (hash) tables)
• TensorList. list of tensors.
• Stack. stack of tensors.
• TensorArray (probably more well known). array of tensors.
• ConditionalAccumulator. accumulate / aggregate gradients
• TemporaryVariable. variable which only lives within a single step.
• Barrier / Mutex