TensorFlow lesser known features

I sometimes stumble upon features in TensorFlow which look potentially powerful or interesting, which I did not know about before. This wiki page is supposed to be a collection of such features. (Once some of them becomes more central to RETURNN, we might want to have a separate wiki page for it. E.g. Distributed TensorFlow. To find them, it is interesting to just browse through tf.raw_ops.

Also note that it is actually quite simple to write native simple ops to return custom information (e.g. like the OS pid).

Operations / functions

• tf.math.divide_no_nan/tf.div_no_nan. Computes x/y and return 0 if y==0 (no nan).
• tf.math.reduce_euclidean_norm
• tf.switch_case and tf.case
• tf.strings.to_hash_bucket
• tf.strings.ngrams
• tf.compat.v1.get_session_handle. keep tensor alive in the session, and access it in future calls via get_session_tensor, or use it for feed_dict
• tf.searchsorted, also lower_bound or LowerBound
• tf.timestamp. Provides the time since epoch in seconds.

Resources

• DenseHashTable and other (hash) tables, LookupTable, ...
• MapStagingArea
• TensorList. list of tensors.
• Stack. stack of tensors.
• TensorArray (probably more well known). array of tensors.
• BoostedTrees
• ConditionalAccumulator. accumulate / aggregate gradients
• TemporaryVariable. variable which only lives within a single step.
• Barrier / Mutex